Compare commits

..
9 Commits
Author SHA1 Message Date
sudacode 176edd67f1 chore(release): v0.19.1 2026-08-01 23:59:20 -07:00
sudacode 4d65dec340 fix(youtube): prevent playlist URLs from stalling yt-dlp probes (#180) 2026-08-01 22:56:28 -07:00
sudacode 6607c333bc fix(overlay): strip spinner frame from subsync overlay card
- Add overlayBody override to ConfiguredStatusNotificationOptions so overlay/OSD/desktop can diverge
- Extract getSubsyncStatusNotificationOptions() to strip the ASCII spinner frame from the overlay card (OSD keeps it since it renders the raw spinner)
- Add tests for spinner stripping and subsync result notifications
2026-07-31 18:03:26 -07:00
sudacode b2bbf1ae12 chore: regenerate config example artifacts 2026-07-31 17:49:02 -07:00
sudacode b204d4dd6e feat(anki): add configurable word card type for Kiku/Lapis (#175) 2026-07-31 17:17:29 -07:00
sudacode 89ed675935 fix(overlay): keep Yomitan popup interactive on macOS/Windows (#177) 2026-07-30 19:47:08 -07:00
sudacode 1995200e76 chore(release): prepare v0.19.0 2026-07-29 22:26:03 -07:00
sudacode e876e483d6 chore(release): prepare v0.19.0-beta.5 2026-07-29 02:09:01 -07:00
sudacode 6d2a72e13b feat(stats): add library entry deletion and app-wide delete progress (#174) 2026-07-29 02:00:16 -07:00
104 changed files with 3086 additions and 486 deletions

No files matched your search

+45
View File
@@ -1,5 +1,50 @@
# Changelog
## v0.19.1 (2026-08-01)
### Added
- Word Card Type: Adds a setting (Settings > Mining/Anki > Kiku/Lapis Features > "Word Card Type") to choose which card-type flag SubMiner marks on Kiku/Lapis word cards — `word-and-sentence` (default), `click`, `sentence`, `audio`, or `none`. Click cards (`IsClickCard`) can now be flagged, and setting any card-type flag clears the others so a note can't claim two types at once.
### Fixed
- Yomitan Popup: Fixes the macOS Yomitan popup going inert after mining a card — clicks outside the popup no longer pass through to mpv, and scrolling over the popup scrolls its definitions instead of seeking playback.
- YouTube Playlist Links: Fixes opening a video from a playlist URL (e.g. a Watch Later link with `list=`/`index=`) timing out while probing subtitles, metadata, or the playback URL.
## v0.19.0 (2026-07-29)
### Added
- Anki Maturity Highlighting: Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature), similar to asbplayer. Tier thresholds and colors are configurable, with a runtime toggle and an updated help legend.
- Post-Playback Menu: After a watch-history episode ends, the fzf/rofi launcher returns to that series with options to play the previous or next episode, rewatch, pick another episode, or quit. The pre-playback series menu now offers the previous episode too.
- Delete Library Entries: The stats Library detail view can now delete an entire title in one step (episodes, sessions, subtitle lines, rollups, cover art, and vocabulary counts). Delete progress is now shown app-wide via a progress bar and status toast instead of disappearing when you switch tabs.
- Cross-Machine Sync: Added SSH-based syncing of stats and watch history between machines, available from the tray ("Sync Stats & History") or `subminer sync`, with saved devices, per-host sync direction, background auto-sync, connection testing, manual snapshots, and support for Windows remotes.
- TsukiHime Subtitle Downloads: Added subtitle downloads for the current video via TsukiHime, loading Japanese as the primary track and your configured secondary language directly into mpv.
### Changed
- Clipboard-Video Shortcut: The "append clipboard video to queue" shortcut is now configurable.
### Fixed
- AniList Season Resolution: Season 2+ files now resolve to the correct AniList entry instead of silently falling back to season 1 (which mismatched character dictionaries and watch progress). Manual overrides now stay scoped per season, fix both the dictionary and progress tracking together, and also correct per-season cover art.
- Subtitle Annotation Accuracy: Fixed several annotation edge cases, including inconsistent POS exclusions on merged quote-particle tokens, dropped annotations on supplementary-plane kanji, katakana punctuation wrongly treated as noise, and certain kanji vocabulary losing N+1 highlighting eligibility.
- AnkiConnect Proxy Port Conflict: Video startup no longer crashes when another process already holds the configured AnkiConnect proxy port; a notification now explains how to resolve it.
- AppImage Quit Crash: Fixed a "Service Crash" desktop notification appearing after closing a video when running the Linux AppImage.
- Autoplay Pause Timing: Fixed playback resuming a few seconds before subtitle tokenization warmup finished, most noticeable when resuming mid-episode or when a cue starts within the first two seconds.
- Stats Known-Word Count: Fixed stats reporting 0 known words for every session after the known-word cache format changed.
- Stats Library Cover Art: Relinking a title to a different AniList entry now updates its cover in the Library grid immediately instead of leaving a stale, mismatched cover cached.
- Rofi Prompt Spacing: Rofi menu prompts now keep a space before the input field instead of running into the placeholder text.
- Stats Settings & Reliability: Hardened stats settings validation (nested/legacy AnkiConnect config now falls back safely instead of breaking) and stats routes against malformed requests and other edge cases.
- Stats Delete Performance: Deleting sessions, episodes, and library entries is now dramatically faster and no longer stalls playback (e.g. a 12-episode title dropped from about a minute to under a second on a large library); the Vocabulary tab also loads much faster. The first launch after upgrading runs a one-time database migration.
<details>
<summary>Internal changes</summary>
### Internal
- Added a golden-file regression test corpus for the tokenizer/annotation pipeline, plus scripts to record new fixtures and diff against stock Yomitan.
- Consolidated renderer modal state handling into a descriptor registry.
- Consolidated CI quality checks (PR, stable, and prerelease) into one reusable workflow with mpv plugin tests and dependency audits.
- Removed the unused stats IPC transport and unified stats dashboard HTTP types with the backend contract.
- Added a script to verify known-word highlight tiers against live Anki data outside of playback.
</details>
## v0.18.0 (2026-07-10)
### Added
@@ -1,5 +0,0 @@
type: added
area: overlay
- Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature) like asbplayer. Enable with `ankiConnect.knownWords.maturityEnabled`; the mature interval threshold (`matureThresholdDays`, default 21) and the four tier colors (`subtitleStyle.knownWordMaturityColors`) are configurable, and a runtime option toggles it in-session. The session help color legend shows the four tier colors while maturity highlighting is on.
- Tiers follow Anki's own card counts: the interval tiers exclude cards in the learning/relearning queue, so a lapsed card shows the learning color instead of young (its interval is reset to at least 1 day, which previously made the learning tier unreachable). A note with a mature card alongside a relearning card still shows mature.
-5
View File
@@ -1,5 +0,0 @@
type: fixed
area: overlay
- Applied configured primary POS exclusions consistently to merged trailing quote-particle tokens, preserved annotations for supplementary-plane kanji, and stopped treating katakana punctuation as kana-only annotation noise.
- Kept kanji vocabulary tagged `名詞/非自立` eligible for N+1 highlighting, consistent with frequency, JLPT, and vocabulary persistence.
@@ -1,4 +0,0 @@
type: changed
area: shortcuts
- Made the clipboard-video playlist shortcut configurable through `shortcuts.appendClipboardVideoToQueue`.
-8
View File
@@ -1,8 +0,0 @@
type: fixed
area: anilist
- Season 2 and later files now resolve to the right AniList entry. AniList has no numbered seasons (sequels are separate entries titled `Zoku`, `Kan`, `2nd Season`), so the old `<title> Season N` search returned nothing and silently fell back to the season 1 entry: an Oregairu S03 episode loaded the season 1 character dictionary and pushed watch progress onto season 1. SubMiner now searches the base title and walks AniList `SEQUEL` relations to reach the requested season, ordering the franchise's TV entries by air date when the relation chain is incomplete.
- When the season still cannot be located, SubMiner no longer writes progress to the season 1 entry; it skips the update and points at the manual AniList override instead. The character dictionary logs the same warning and states which entry it fell back to.
- Character dictionary overrides are now scoped by detected season as well as directory, so seasons kept in one flat folder no longer share (or overwrite) each other's override, and an override now also pins the entry used for AniList watch progress: correcting a wrong match once fixes both the dictionary and progress tracking.
- Cover art lookup uses the same season-aware resolution, so per-season art no longer falls back to the season 1 cover.
- An unresolvable season is no longer cached as if it were a normal match, so the fallback cannot silently stick for every later episode of that series.
-4
View File
@@ -1,4 +0,0 @@
type: fixed
area: anki
- Prevented video startup from crashing when another process already owns the configured AnkiConnect proxy port, and added a notification explaining how to resolve the conflict.
@@ -1,4 +0,0 @@
type: fixed
area: app
- Fixed "Service Crash" desktop notifications (KDE DrKonqi) after closing a video when running the Linux AppImage: the short-lived background bootstrap spawned a Chromium GPU child that outlived it (surviving `app.exit`) and died with SIGBUS at session end when the bootstrap's FUSE mount was finally released. The bootstrap now runs with the GPU in-process so it leaves no children behind, and the detached app's mount remains supervised until its Chromium children finish. Set `SUBMINER_NO_APPIMAGE_MOUNT_KEEPALIVE=1` to disable the detached-app mount supervisor.
@@ -1,4 +0,0 @@
type: fixed
area: character dictionary
- Kept manual AniList overrides active across episodes in the same season directory when filename guesses differ.
@@ -1,4 +0,0 @@
type: fixed
area: overlay
- Fixed `mpv.pauseUntilOverlayReady` releasing playback seconds before tokenization warmup finished: startup subtitle priming emits the current cue untokenized so the overlay can paint early, and that emission was treated as the autoplay-readiness signal as soon as the overlay window loaded. The autoplay gate now ignores untokenized subtitle payloads while tokenization warmup is pending, so playback resumes only after the first tokenized delivery (or the post-warmup release). Most visible when resuming mid-episode or when a subtitle cue starts within the first two seconds.
@@ -1,5 +0,0 @@
type: fixed
area: stats
- Stats reported 0 known words for every session after the known-word cache gained maturity tiers. The stats server carried its own copy of the cache parser that only recognized versions up to 3, so the new v4 file was read as "no cache" rather than as a format it should understand.
- The cache format, its parser, and the derived known-word set now live in one module that both the cache manager and the stats server read, and the version dispatch ends in an exhaustive check so a future format bump fails the build instead of silently reporting zero. A cache that exists but does not parse now logs a warning rather than passing for an empty one.
-7
View File
@@ -1,7 +0,0 @@
type: internal
area: tokenizer
- Added a golden-file regression corpus for the tokenizer/annotation pipeline: recorded Yomitan backend responses and MeCab tokens replay through the real tokenizeSubtitle pipeline in bun tests without Electron or dictionaries.
- Added `record-tokenizer-fixture:electron` script to capture new fixtures from a live Yomitan/MeCab session, with flags for known words, JLPT levels, and annotation toggles.
- Seeded eleven fixtures covering the #147–#156 regression classes (grammar-helper suppression, lexical くれる, kanji non-independent nouns, N+1 targeting, reading collisions, unparsed runs, ordinal/honorific prefixes).
- Added `compare-yomitan-api:electron` script that diffs SubMiner tokenization against a stock Yomitan instance via the yomitan-api bridge (segmentation, readings, headword forms).
-5
View File
@@ -1,5 +0,0 @@
type: added
area: launcher
- After a watch-history episode ends or mpv closes, the fzf or rofi launcher returns to that series with options to play the previous episode, rewatch, play the next episode, select another episode, or quit SubMiner. Previous and Next continue across season directories.
- The action menu shown right after picking a series from `subminer -H` now also offers the previous episode, matching the menu shown after playback.
-4
View File
@@ -1,4 +0,0 @@
type: internal
area: overlay
- Consolidated renderer modal state handling into a descriptor registry.
-4
View File
@@ -1,4 +0,0 @@
type: internal
area: release
- Consolidated pull request, stable release, and prerelease quality checks in one reusable workflow, with Lua mpv plugin tests and blocking high-severity dependency audits running in every gate.
-4
View File
@@ -1,4 +0,0 @@
type: fixed
area: launcher
- Rofi menu prompts now keep a space between the prompt text and the input field instead of running into the search placeholder.
-5
View File
@@ -1,5 +0,0 @@
type: fixed
area: stats
- Validated nested and legacy AnkiConnect settings after splitting the resolver, preserving valid modern overrides while warning and falling back for invalid primitive values.
- Hardened stats routes against malformed IDs and static paths, stalled AniList searches, word-mining media collisions, missing Yomitan bridges, and throwing timing observers.
-8
View File
@@ -1,8 +0,0 @@
type: added
area: sync
- Added cross-machine immersion sync for stats and watch history over SSH, available as a window (**Sync Stats & History** in the tray menu, or `subminer sync --ui`) and as a command (`subminer sync <host>`, with `--push` / `--pull` for one-way insert-only transfers). The window keeps saved devices with per-host direction, one-click sync with live stage-by-stage progress and separate merge summaries for each machine, connection testing for first-time setup, cancellable runs while the app/stats server/playback is active, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Hosts with auto-sync enabled sync in the background on a configurable interval, including during playback, with results reported as overlay notifications; hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and appear in the window automatically.
- Merges are an insert-only union keyed on session UUIDs / video keys / series title keys, so re-syncing is idempotent and nothing is double-counted: each side snapshots its database (`VACUUM INTO`) from a consistent WAL point, snapshots are exchanged with `scp`, and each machine merges the other's data transactionally. Lifetime totals and daily/monthly rollups are updated incrementally (history older than the session retention window is preserved), unfinished sessions are excluded until a later sync sees them finalized, and remote-only historical rollups are copied only when they do not conflict with retained local session history. Sync aborts on stats schema version mismatches and refuses to run while the stats daemon or a live mpv session is active (`--force` overrides).
- The sync engine runs only inside the app: the sync window and the `subminer sync` command both delegate to `SubMiner --sync-cli` (headless, works over SSH with no display), so neither machine needs bun or the command-line launcher. A remote machine only needs SubMiner itself, found automatically as the app binary or via the launcher proxy.
- Windows remotes are supported: sync detects the remote shell (POSIX, cmd, or PowerShell) and manages remote temp files through SubMiner itself (`sync --make-temp` / `--remove-temp`) instead of `mktemp` / `rm`, so a Windows machine with the built-in OpenSSH Server works as a sync remote, found in its default Windows install location automatically.
- Added supporting flags: `subminer sync <host> --check` tests the SSH connection and remote launcher availability without syncing, `subminer sync --snapshot <file>` and `--merge <file>` expose the underlying steps for manual transfers, and `subminer sync --json` emits machine-readable NDJSON progress (the protocol the sync window consumes).
-4
View File
@@ -1,4 +0,0 @@
type: added
area: overlay
- Added TsukiHime subtitle downloads for the current video, with Japanese primary and configured secondary-language tracks loaded directly into mpv.
-4
View File
@@ -1,4 +0,0 @@
type: internal
area: stats
- Removed the unused stats IPC data transport and unified the stats dashboard's HTTP wire types with the backend contract.
@@ -1,7 +0,0 @@
type: internal
area: tokenizer
- Added `verify-known-word-highlights:electron` script: tokenizes a real subtitle file through the app's Yomitan/MeCab pipeline with the live known-word cache, prints each line in the configured tier colors, and summarizes the tier counts so highlighting can be checked outside of playback.
- Added `--audit`, which re-derives every highlighted tier from live Anki card data (`notesInfo` + `cardsInfo` intervals) and reports each token whose rendered tier disagrees, catching both stale cache entries and tier-classification bugs.
- Added `--profile-copy` so the check can run while SubMiner is open (Electron locks the Yomitan userData dir), plus `--refresh`, `--limit`, `--json`, and `--quiet`.
- Added `KnownWordCacheManager.getKnownWordMatchNoteIds`, exposing the note ids behind a known-word match so an audit can trace a rendered tier back to the exact Anki notes.
+5 -2
View File
@@ -523,7 +523,7 @@
// ==========================================
// AnkiConnect Integration
// Automatic Anki updates and media generation options.
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.
// Shared AI provider transport settings are read from top-level ai and typically require restart.
// Most other AnkiConnect settings still require restart.
// ==========================================
@@ -605,7 +605,10 @@
"enabled": false, // Enable Kiku-specific mining behaviors (duplicate handling, field grouping). Values: true | false
"fieldGrouping": "disabled", // Kiku duplicate-card field grouping mode. Values: auto | manual | disabled
"deleteDuplicateInAuto": true // When Kiku field grouping is "auto", delete the duplicate source card after grouping completes. Values: true | false
} // Is kiku setting.
}, // Is kiku setting.
"lapisKiku": {
"wordCardKind": "word-and-sentence" // Card-type flag SubMiner marks on Kiku/Lapis word cards. Only one flag is set at a time; the others are cleared. Requires isKiku.enabled or isLapis.enabled. Values: word-and-sentence | click | sentence | audio | none
} // Lapis kiku setting.
}, // Automatic Anki updates and media generation options.
// ==========================================
+2
View File
@@ -69,6 +69,8 @@ SubMiner fetches cover art from AniList for display in the stats dashboard. When
A no-match result is cached for 5 minutes before SubMiner retries, preventing repeated API calls for unrecognized media.
If the automatic match is wrong, use **Change AniList Entry** on a title in the stats Library. Relinking rewrites the cached art for every episode of that title, and both the detail view and the Library grid pick up the new cover right away: the grid refetches after a relink, and cover responses carry an ETag and are revalidated on each request instead of being cached for a day.
## Rate Limiting
All AniList API calls go through a shared rate limiter that enforces a sliding window of 20 requests per minute. The limiter also reads AniList's `X-RateLimit-Remaining` and `Retry-After` response headers and pauses requests when the server signals throttling. This applies to both episode tracking and cover art fetching.
+19 -4
View File
@@ -289,6 +289,21 @@ Trigger with the mine sentence shortcut (`Ctrl/Cmd+S` by default). The card is c
To mine multiple subtitle lines as one sentence card, use `Ctrl/Cmd+Shift+S` followed by a digit (1–9) to select how many recent lines to combine.
## Word Card Type (Kiku/Lapis)
Word cards get a card-type flag when SubMiner fills their sentence, whether that comes from Yomitan auto-enrichment, a manual clipboard update, or stats-dashboard word mining. By default the flag is `IsWordAndSentenceCard`; pick a different one with `ankiConnect.lapisKiku.wordCardKind`.
```jsonc
"ankiConnect": {
"isKiku": { "enabled": true },
"lapisKiku": {
"wordCardKind": "click" // word-and-sentence (default), click, sentence, audio, none
}
}
```
`click` marks `IsClickCard`, `sentence` marks `IsSentenceCard`, `audio` marks `IsAudioCard`, and `none` leaves the flags untouched for templates that manage them elsewhere. Whichever flag is chosen, the other card-type flags are cleared so the note never claims two card types. The setting is only read when `isKiku` or `isLapis` is enabled, and cards mined with Mine Sentence or Mine Audio keep their own flag.
## Field Grouping (Kiku)
When you mine the same word multiple times, SubMiner can merge the cards instead of creating duplicates. This is designed for note types like [Kiku](https://github.com/youyoumu/kiku) that support grouped sentence/audio/image fields.
@@ -313,11 +328,11 @@ When you mine the same word multiple times, SubMiner can merge the cards instead
### What Gets Merged
| Field | Merge behavior |
| -------- | ---------------------------------------- |
| Field | Merge behavior |
| -------- | --------------------------------------------- |
| Sentence | Both cards' sentences kept as grouped entries |
| Audio | Both cards' `[sound:...]` entries kept |
| Image | Both cards' images kept |
| Audio | Both cards' `[sound:...]` entries kept |
| Image | Both cards' images kept |
Identical values from both cards are kept as separate grouped entries; the merge does not deduplicate.
+52 -2
View File
@@ -1,6 +1,56 @@
# Changelog
## v0.18.0 (2026-07-10)
## v0.19.1 (2026-08-01)
**Added**
- Word Card Type: Adds a setting (Settings > Mining/Anki > Kiku/Lapis Features > "Word Card Type") to choose which card-type flag SubMiner marks on Kiku/Lapis word cards — `word-and-sentence` (default), `click`, `sentence`, `audio`, or `none`. Click cards (`IsClickCard`) can now be flagged, and setting any card-type flag clears the others so a note can't claim two types at once.
**Fixed**
- Yomitan Popup: Fixes the macOS Yomitan popup going inert after mining a card — clicks outside the popup no longer pass through to mpv, and scrolling over the popup scrolls its definitions instead of seeking playback.
- YouTube Playlist Links: Fixes opening a video from a playlist URL (e.g. a Watch Later link with `list=`/`index=`) timing out while probing subtitles, metadata, or the playback URL.
## v0.19.0 (2026-07-29)
**Added**
- Anki Maturity Highlighting: Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature), similar to asbplayer. Tier thresholds and colors are configurable, with a runtime toggle and an updated help legend.
- Post-Playback Menu: After a watch-history episode ends, the fzf/rofi launcher returns to that series with options to play the previous or next episode, rewatch, pick another episode, or quit. The pre-playback series menu now offers the previous episode too.
- Delete Library Entries: The stats Library detail view can now delete an entire title in one step (episodes, sessions, subtitle lines, rollups, cover art, and vocabulary counts). Delete progress is now shown app-wide via a progress bar and status toast instead of disappearing when you switch tabs.
- Cross-Machine Sync: Added SSH-based syncing of stats and watch history between machines, available from the tray ("Sync Stats & History") or `subminer sync`, with saved devices, per-host sync direction, background auto-sync, connection testing, manual snapshots, and support for Windows remotes.
- TsukiHime Subtitle Downloads: Added subtitle downloads for the current video via TsukiHime, loading Japanese as the primary track and your configured secondary language directly into mpv.
**Changed**
- Clipboard-Video Shortcut: The "append clipboard video to queue" shortcut is now configurable.
**Fixed**
- AniList Season Resolution: Season 2+ files now resolve to the correct AniList entry instead of silently falling back to season 1 (which mismatched character dictionaries and watch progress). Manual overrides now stay scoped per season, fix both the dictionary and progress tracking together, and also correct per-season cover art.
- Subtitle Annotation Accuracy: Fixed several annotation edge cases, including inconsistent POS exclusions on merged quote-particle tokens, dropped annotations on supplementary-plane kanji, katakana punctuation wrongly treated as noise, and certain kanji vocabulary losing N+1 highlighting eligibility.
- AnkiConnect Proxy Port Conflict: Video startup no longer crashes when another process already holds the configured AnkiConnect proxy port; a notification now explains how to resolve it.
- AppImage Quit Crash: Fixed a "Service Crash" desktop notification appearing after closing a video when running the Linux AppImage.
- Autoplay Pause Timing: Fixed playback resuming a few seconds before subtitle tokenization warmup finished, most noticeable when resuming mid-episode or when a cue starts within the first two seconds.
- Stats Known-Word Count: Fixed stats reporting 0 known words for every session after the known-word cache format changed.
- Stats Library Cover Art: Relinking a title to a different AniList entry now updates its cover in the Library grid immediately instead of leaving a stale, mismatched cover cached.
- Rofi Prompt Spacing: Rofi menu prompts now keep a space before the input field instead of running into the placeholder text.
- Stats Settings & Reliability: Hardened stats settings validation (nested/legacy AnkiConnect config now falls back safely instead of breaking) and stats routes against malformed requests and other edge cases.
- Stats Delete Performance: Deleting sessions, episodes, and library entries is now dramatically faster and no longer stalls playback (e.g. a 12-episode title dropped from about a minute to under a second on a large library); the Vocabulary tab also loads much faster. The first launch after upgrading runs a one-time database migration.
<details>
<summary>Internal changes</summary>
**Internal**
- Added a golden-file regression test corpus for the tokenizer/annotation pipeline, plus scripts to record new fixtures and diff against stock Yomitan.
- Consolidated renderer modal state handling into a descriptor registry.
- Consolidated CI quality checks (PR, stable, and prerelease) into one reusable workflow with mpv plugin tests and dependency audits.
- Removed the unused stats IPC transport and unified stats dashboard HTTP types with the backend contract.
- Added a script to verify known-word highlight tiers against live Anki data outside of playback.
</details>
## Previous Versions
<details>
<summary>v0.18.x</summary>
<h2>v0.18.0 (2026-07-10)</h2>
**Added**
- Sentence Audio Normalization: Generated sentence audio is now normalized to -23 LUFS by default, and clips mined from playback mirror mpv's software volume curve with a limiter to prevent clipping. Both behaviors are configurable independently.
@@ -31,7 +81,7 @@
</details>
## Previous Versions
</details>
<details>
<summary>v0.17.x</summary>
+72 -54
View File
@@ -398,30 +398,30 @@ See `config.example.jsonc` for detailed configuration options.
}
```
| Option | Values | Description |
| ---------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `primaryDefaultMode` | string | Default primary subtitle bar visibility mode: `"hidden"`, `"visible"`, or `"hover"` (default: `"visible"`) |
| `subtitleStyle.css` | object | CSS declaration object applied to primary subtitles after normal style defaults. Use CSS property names such as `font-size`. |
| `secondary.css` | object | CSS declaration object applied to secondary subtitles after normal secondary style defaults. |
| `enableJlpt` | boolean | Enable JLPT level underline styling (`false` by default) |
| `preserveLineBreaks` | boolean | Preserve line breaks in visible overlay subtitle rendering (`false` by default). Enable to mirror mpv line layout. |
| `autoPauseVideoOnHover` | boolean | Pause playback while mouse hovers subtitle text, then resume on leave (`true` by default). |
| `autoPauseVideoOnYomitanPopup` | boolean | Pause playback while the Yomitan popup is open, then resume when the popup closes (`true` by default). |
| `primaryVisibleOnYomitanPopup` | boolean | Keep hover-mode primary subtitles visible while the Yomitan popup is open (`true` by default). |
| `nameMatchEnabled` | boolean | Enable character dictionary sync and subtitle token coloring for character-name matches (`false` by default) |
| `nameMatchImagesEnabled` | boolean | Show small cached AniList character portraits beside matched character-name tokens (`false` by default) |
| `nameMatchColor` | string | Hex color used for subtitle tokens matched from the SubMiner character dictionary (default: `#f5bde6`) |
| `knownWordColor` | string | Hex color used for known-word subtitle highlights (default: `#a6da95`) |
| Option | Values | Description |
| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `primaryDefaultMode` | string | Default primary subtitle bar visibility mode: `"hidden"`, `"visible"`, or `"hover"` (default: `"visible"`) |
| `subtitleStyle.css` | object | CSS declaration object applied to primary subtitles after normal style defaults. Use CSS property names such as `font-size`. |
| `secondary.css` | object | CSS declaration object applied to secondary subtitles after normal secondary style defaults. |
| `enableJlpt` | boolean | Enable JLPT level underline styling (`false` by default) |
| `preserveLineBreaks` | boolean | Preserve line breaks in visible overlay subtitle rendering (`false` by default). Enable to mirror mpv line layout. |
| `autoPauseVideoOnHover` | boolean | Pause playback while mouse hovers subtitle text, then resume on leave (`true` by default). |
| `autoPauseVideoOnYomitanPopup` | boolean | Pause playback while the Yomitan popup is open, then resume when the popup closes (`true` by default). |
| `primaryVisibleOnYomitanPopup` | boolean | Keep hover-mode primary subtitles visible while the Yomitan popup is open (`true` by default). |
| `nameMatchEnabled` | boolean | Enable character dictionary sync and subtitle token coloring for character-name matches (`false` by default) |
| `nameMatchImagesEnabled` | boolean | Show small cached AniList character portraits beside matched character-name tokens (`false` by default) |
| `nameMatchColor` | string | Hex color used for subtitle tokens matched from the SubMiner character dictionary (default: `#f5bde6`) |
| `knownWordColor` | string | Hex color used for known-word subtitle highlights (default: `#a6da95`) |
| `knownWordMaturityColors` | object | Per-tier known-word colors used when `ankiConnect.knownWords.maturityEnabled` is on: `new` (`#ee99a0`), `learning` (`#b7bdf8`), `young` (`#91d7e3`), `mature` (`#a6da95`) |
| `nPlusOneColor` | string | Hex color used for the single N+1 target subtitle highlight (default: `#c6a0f6`) |
| `frequencyDictionary.enabled` | boolean | Enable frequency highlighting from dictionary lookups (`false` by default) |
| `frequencyDictionary.sourcePath` | string | Path to a local frequency dictionary root. Leave empty or omit to use installed/default frequency-dictionary search paths. |
| `frequencyDictionary.topX` | number | Only color tokens whose frequency rank is `<= topX` (`10000` by default) |
| `frequencyDictionary.mode` | string | `"single"` or `"banded"` (`"single"` by default) |
| `frequencyDictionary.matchMode` | string | `"headword"` or `"surface"` (`"headword"` by default) |
| `frequencyDictionary.singleColor` | string | Color used for all highlighted tokens in single mode |
| `frequencyDictionary.bandedColors` | string[] | Array of five hex colors used for ranked bands in banded mode |
| `jlptColors` | object | JLPT level underline colors object (`N1`..`N5`) |
| `nPlusOneColor` | string | Hex color used for the single N+1 target subtitle highlight (default: `#c6a0f6`) |
| `frequencyDictionary.enabled` | boolean | Enable frequency highlighting from dictionary lookups (`false` by default) |
| `frequencyDictionary.sourcePath` | string | Path to a local frequency dictionary root. Leave empty or omit to use installed/default frequency-dictionary search paths. |
| `frequencyDictionary.topX` | number | Only color tokens whose frequency rank is `<= topX` (`10000` by default) |
| `frequencyDictionary.mode` | string | `"single"` or `"banded"` (`"single"` by default) |
| `frequencyDictionary.matchMode` | string | `"headword"` or `"surface"` (`"headword"` by default) |
| `frequencyDictionary.singleColor` | string | Color used for all highlighted tokens in single mode |
| `frequencyDictionary.bandedColors` | string[] | Array of five hex colors used for ranked bands in banded mode |
| `jlptColors` | object | JLPT level underline colors object (`N1`..`N5`) |
Subtitle CSS custom properties:
@@ -555,11 +555,11 @@ Secondary subtitles do **not** auto-load by default. To turn them on for local a
}
```
| Option | Values | Description |
| ----------------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `secondarySubLanguages` | string[] | Language codes to auto-load (e.g., `["eng", "en"]`); non-Signs/Songs tracks are preferred when several tracks match. Default is empty (`[]`). |
| `autoLoadSecondarySub` | `true`, `false` | Auto-detect and load a matching secondary subtitle track for local/Jellyfin sidecar files (default: `false`) |
| `defaultMode` | `"hidden"`, `"visible"`, `"hover"` | Initial display mode (default: `"hover"`) |
| Option | Values | Description |
| ----------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `secondarySubLanguages` | string[] | Language codes to auto-load (e.g., `["eng", "en"]`); non-Signs/Songs tracks are preferred when several tracks match. Default is empty (`[]`). |
| `autoLoadSecondarySub` | `true`, `false` | Auto-detect and load a matching secondary subtitle track for local/Jellyfin sidecar files (default: `false`) |
| `defaultMode` | `"hidden"`, `"visible"`, `"hover"` | Initial display mode (default: `"hover"`) |
These two settings apply to local and Jellyfin playback only. YouTube secondary selection is fixed to English and ignores them; see [YouTube Integration](/youtube-integration#secondary-subtitle-languages). `defaultMode` still controls how the loaded secondary bar is displayed in every case.
@@ -1043,7 +1043,7 @@ This example is intentionally compact. The option table below documents availabl
| `ankiConnect.knownWords.refreshMinutes` | number | Minutes between known-word cache refreshes (default: `1440`) |
| `ankiConnect.knownWords.decks` | object | Deck→fields mapping used for known-word cache query scope (e.g. `{ "Kaishi 1.5k": ["Word"] }`). |
| `ankiConnect.knownWords.maturityEnabled` | `true`, `false` | Color known words by Anki card maturity (new/learning/young/mature) instead of one color. Requires `knownWords.highlightEnabled` (default: `false`). Tier colors come from `subtitleStyle.knownWordMaturityColors`. |
| `ankiConnect.knownWords.matureThresholdDays` | number | Card interval in days at which a known word counts as mature (default: `21`, matching Anki's own convention) |
| `ankiConnect.knownWords.matureThresholdDays` | number | Card interval in days at which a known word counts as mature (default: `21`, matching Anki's own convention) |
| `ankiConnect.nPlusOne.enabled` | `true`, `false` | Enable N+1 subtitle highlighting (highlights the one unknown word in a sentence). Independent from `knownWords.highlightEnabled`. Requires known-word cache data (default: `false`). |
| `ankiConnect.nPlusOne.minSentenceWords` | number | Minimum number of words required in a sentence before single unknown-word N+1 highlighting can trigger (default: `3`). |
| `behavior.notificationType` | `"overlay"`, `"system"`, `"both"`, `"none"` | Notification type on card update (default: `"overlay"`). `"both"` means overlay + system. `osd` and `osd-system` are legacy config-file-only values; use `"osd-system"` to keep the old OSD + system behavior. |
@@ -1069,6 +1069,9 @@ SubMiner is intentionally built for [Kiku](https://kiku.youyoumu.my.id/) and [La
"enabled": true,
"fieldGrouping": "manual",
"deleteDuplicateInAuto": true
},
"lapisKiku": {
"wordCardKind": "word-and-sentence"
}
}
```
@@ -1077,6 +1080,21 @@ SubMiner is intentionally built for [Kiku](https://kiku.youyoumu.my.id/) and [La
- Enable `isKiku` to turn on duplicate merge behavior for mined Word/Expression hits.
- When both are enabled, Kiku behavior is applied for grouping while sentence-card model settings are still read from `isLapis`.
- `isKiku.fieldGrouping` supports `disabled`, `auto`, and `manual` merge modes; see [Field Grouping Modes](#field-grouping-modes).
- `lapisKiku.wordCardKind` picks the card-type flag set on word cards; see [Word Card Type](#word-card-type). It is read only while `isLapis` or `isKiku` is enabled.
### Word Card Type
When SubMiner fills the sentence on a mined word card - from Yomitan auto-enrichment, a manual clipboard update, or stats-dashboard word mining - it marks which card that note should generate. `ankiConnect.lapisKiku.wordCardKind` chooses the flag:
| Value | Flag set |
| ----------------------------- | ----------------------- |
| `word-and-sentence` (default) | `IsWordAndSentenceCard` |
| `click` | `IsClickCard` |
| `sentence` | `IsSentenceCard` |
| `audio` | `IsAudioCard` |
| `none` | none; flags left as-is |
The other card-type flags are cleared so a note never claims two card types at once. Notes are skipped when the note type has no field for the chosen flag, and when the note was already mined as a sentence or audio card. Cards created by Mine Sentence and Mine Audio keep their own flag regardless of this setting.
### N+1 Word Highlighting
@@ -1167,10 +1185,10 @@ TsukiHime subtitle search works out of the box and needs no account or API key.
}
```
| Option | Values | Description |
| ---------------------------- | ------------ | ---------------------------------------------------------------------------------------------------- |
| Option | Values | Description |
| ---------------------------- | ------------ | ----------------------------------------------------------------------------------------------------- |
| `tsukihime.apiBaseUrl` | string (URL) | Base URL of the TsukiHime API (default: `https://api.tsukihime.org/v1`). Only change it for a mirror. |
| `tsukihime.maxSearchResults` | number | Maximum releases returned per search (default: `10`; the API caps this at 100) |
| `tsukihime.maxSearchResults` | number | Maximum releases returned per search (default: `10`; the API caps this at 100) |
The keyboard shortcut lives under `shortcuts.openTsukihime` (default `Ctrl+Shift+T`; set to `null` to disable). The older `animetosho` section and `shortcuts.openAnimetosho` are still accepted as deprecated aliases, with the current names taking precedence when both are set.
@@ -1228,17 +1246,17 @@ AniList integration is opt-in and disabled by default. Enable it to allow SubMin
}
```
| Option | Values | Description |
| -------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------- |
| `anilist.enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
| `accessToken` | string | Optional explicit AniList access token override (default: empty string) |
| `characterDictionary.maxLoaded` | number | Maximum number of most-recently-used AniList media snapshots included in the merged dictionary (default: `3`) |
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
| `characterDictionary.collapsibleSections.description` | `true`, `false` | Open the Description section by default in generated dictionary entries |
| `characterDictionary.collapsibleSections.characterInformation` | `true`, `false` | Open the Character Information section by default in generated dictionary entries |
| `characterDictionary.collapsibleSections.voicedBy` | `true`, `false` | Open the Voiced by section by default in generated dictionary entries |
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary settings updates to all Yomitan profiles or only active profile |
| Option | Values | Description |
| -------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------- |
| `anilist.enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
| `accessToken` | string | Optional explicit AniList access token override (default: empty string) |
| `characterDictionary.maxLoaded` | number | Maximum number of most-recently-used AniList media snapshots included in the merged dictionary (default: `3`) |
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
| `characterDictionary.collapsibleSections.description` | `true`, `false` | Open the Description section by default in generated dictionary entries |
| `characterDictionary.collapsibleSections.characterInformation` | `true`, `false` | Open the Character Information section by default in generated dictionary entries |
| `characterDictionary.collapsibleSections.voicedBy` | `true`, `false` | Open the Voiced by section by default in generated dictionary entries |
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary settings updates to all Yomitan profiles or only active profile |
When `enabled` is `true` and `accessToken` is empty, SubMiner opens an AniList setup helper window. Keep `enabled` as `false` to disable all AniList setup/update behavior.
@@ -1539,18 +1557,18 @@ Configure the mpv executable, profile, and window state for SubMiner-managed mpv
}
```
| Option | Values | Description |
| ------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `executablePath` | string | Absolute path to `mpv.exe` for Windows launch flows. Leave empty to auto-discover from `SUBMINER_MPV_PATH` or `PATH` (default `""`) |
| `profile` | string | mpv profile name passed as `--profile=<name>`. Leave empty to pass no profile (default `""`) |
| `launchMode` | `"normal"` \| `"maximized"` \| `"fullscreen"` | Window state when SubMiner spawns mpv (default `"normal"`) |
| Option | Values | Description |
| ------------------------ | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `executablePath` | string | Absolute path to `mpv.exe` for Windows launch flows. Leave empty to auto-discover from `SUBMINER_MPV_PATH` or `PATH` (default `""`) |
| `profile` | string | mpv profile name passed as `--profile=<name>`. Leave empty to pass no profile (default `""`) |
| `launchMode` | `"normal"` \| `"maximized"` \| `"fullscreen"` | Window state when SubMiner spawns mpv (default `"normal"`) |
| `socketPath` | string | mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin (platform-dependent default: `/tmp/subminer-socket`, or `\\\\.\\pipe\\subminer-socket` on Windows) |
| `backend` | `"auto"` \| `"hyprland"` \| `"sway"` \| `"x11"` \| `"macos"` \| `"windows"` | Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform (default: `"auto"`) |
| `autoStartSubMiner` | `true`, `false` | Start SubMiner in the background when SubMiner-managed mpv loads a file (default: `true`) |
| `pauseUntilOverlayReady` | `true`, `false` | Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness, with a 30-second fallback (default: `true`) |
| `subminerBinaryPath` | string | SubMiner app binary path passed to the bundled mpv plugin. Leave empty to use the launcher-detected app path (default: `""`) |
| `aniskipEnabled` | `true`, `false` | Enable AniSkip intro detection, chapter markers, and the skip-intro key (default: `true`) |
| `aniskipButtonKey` | string | mpv key used to skip the detected intro while the skip prompt is visible (default: `"TAB"`) |
| `backend` | `"auto"` \| `"hyprland"` \| `"sway"` \| `"x11"` \| `"macos"` \| `"windows"` | Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform (default: `"auto"`) |
| `autoStartSubMiner` | `true`, `false` | Start SubMiner in the background when SubMiner-managed mpv loads a file (default: `true`) |
| `pauseUntilOverlayReady` | `true`, `false` | Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness, with a 30-second fallback (default: `true`) |
| `subminerBinaryPath` | string | SubMiner app binary path passed to the bundled mpv plugin. Leave empty to use the launcher-detected app path (default: `""`) |
| `aniskipEnabled` | `true`, `false` | Enable AniSkip intro detection, chapter markers, and the skip-intro key (default: `true`) |
| `aniskipButtonKey` | string | mpv key used to skip the detected intro while the skip prompt is visible (default: `"TAB"`) |
If `mpv.profile` is configured and the launcher also receives `--profile`, SubMiner passes both as a comma-separated mpv profile list.
+2
View File
@@ -57,6 +57,8 @@ Jellyfin stream URLs are normalized to stable item links before stats titles are
When YouTube channel metadata is available, the Library tab groups videos by creator/channel and treats each tracked video as an episode-like entry inside that channel section.
Open a title and use **Delete Entry** in its header to remove a mistakenly tracked show outright. This deletes every episode of that title along with their sessions, subtitle lines, rollups and cover art, drops the words and kanji that were only seen there, and removes the card from the Library grid. Individual episodes and sessions can still be deleted on their own from the episode list and session rows. Entry deletion is refused while that title is the one currently playing.
![Stats Library](/screenshots/stats-library.png)
#### Trends
+5 -2
View File
@@ -523,7 +523,7 @@
// ==========================================
// AnkiConnect Integration
// Automatic Anki updates and media generation options.
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.
// Shared AI provider transport settings are read from top-level ai and typically require restart.
// Most other AnkiConnect settings still require restart.
// ==========================================
@@ -605,7 +605,10 @@
"enabled": false, // Enable Kiku-specific mining behaviors (duplicate handling, field grouping). Values: true | false
"fieldGrouping": "disabled", // Kiku duplicate-card field grouping mode. Values: auto | manual | disabled
"deleteDuplicateInAuto": true // When Kiku field grouping is "auto", delete the duplicate source card after grouping completes. Values: true | false
} // Is kiku setting.
}, // Is kiku setting.
"lapisKiku": {
"wordCardKind": "word-and-sentence" // Card-type flag SubMiner marks on Kiku/Lapis word cards. Only one flag is set at a time; the others are cleared. Requires isKiku.enabled or isLapis.enabled. Values: word-and-sentence | click | sentence | audio | none
} // Lapis kiku setting.
}, // Automatic Anki updates and media generation options.
// ==========================================
+1 -1
View File
@@ -11,7 +11,7 @@ SubMiner auto-loads Japanese subtitles when you play a YouTube URL, giving you t
When SubMiner detects a YouTube URL (or `ytsearch:` target), it pauses mpv at startup and runs a subtitle pipeline before resuming playback:
1. **Probe** --- `yt-dlp --dump-single-json` extracts all available subtitle tracks (manual uploads and auto-generated captions) along with video metadata.
1. **Probe** --- `yt-dlp --dump-single-json` extracts all available subtitle tracks (manual uploads and auto-generated captions) along with video metadata. Every yt-dlp call passes `--no-playlist`, so playlist links (for example a Watch Later URL with `list=`/`index=`) resolve to the single video instead of the whole playlist.
2. **Discover** --- Each track is normalized into a `YoutubeTrackOption` with language code, kind (`manual` or `auto`), display label, and direct download URL.
3. **Select** --- SubMiner picks the best primary track (Japanese, preferring manual over auto) and secondary track (English, preferring manual over auto).
4. **Download** --- Selected tracks are fetched via direct URL when available, falling back to `yt-dlp --write-subs` / `--write-auto-subs`. YouTube TimedText XML formats (`srv1`/`srv2`/`srv3`) are converted to VTT on the fly. Auto-generated VTT captions are normalized to remove rolling-caption duplication.
+4 -2
View File
@@ -185,6 +185,7 @@ export const IMMERSION_DB_FIXTURE_DDL = `
line_id INTEGER NOT NULL,
word_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, word_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
@@ -193,6 +194,7 @@ export const IMMERSION_DB_FIXTURE_DDL = `
line_id INTEGER NOT NULL,
kanji_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, kanji_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
@@ -313,8 +315,8 @@ export const IMMERSION_DB_FIXTURE_DDL = `
CREATE INDEX idx_subtitle_lines_session_line ON imm_subtitle_lines(session_id, line_index);
CREATE INDEX idx_subtitle_lines_video_line ON imm_subtitle_lines(video_id, line_index);
CREATE INDEX idx_subtitle_lines_anime_line ON imm_subtitle_lines(anime_id, line_index);
CREATE INDEX idx_word_line_occurrences_word ON imm_word_line_occurrences(word_id, line_id);
CREATE INDEX idx_kanji_line_occurrences_kanji ON imm_kanji_line_occurrences(kanji_id, line_id);
CREATE INDEX idx_word_line_occurrences_word_seen ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id);
CREATE INDEX idx_kanji_line_occurrences_kanji_seen ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id);
CREATE INDEX idx_media_art_cover_blob_hash ON imm_media_art(cover_blob_hash);
CREATE INDEX idx_media_art_anilist_id ON imm_media_art(anilist_id);
CREATE INDEX idx_media_art_cover_url ON imm_media_art(cover_url);
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "subminer",
"productName": "SubMiner",
"desktopName": "SubMiner.desktop",
"version": "0.19.0-beta.4",
"version": "0.19.1",
"description": "All-in-one sentence mining overlay with AnkiConnect and dictionary integration",
"packageManager": "bun@1.3.5",
"main": "dist/main-entry.js",
+13 -2
View File
@@ -24,6 +24,11 @@
- The mature-interval threshold and the four tier colors are configurable, and the in-session help legend shows the active tier colors while maturity highlighting is on.
- Tiers follow Anki's own card state: a lapsed card correctly shows as learning rather than young, and a note is treated as mature if any of its cards are mature. Stats and other known-word tools stay accurate with this new data.
- **Stats Library Entry Deletion**
- Added a "Delete Entry" action in the stats Library detail view that removes an entire title in one step: every episode, session, subtitle line, rollup, cover, and vocabulary count derived from it. Previously a mistaken entry had to be cleared episode by episode and still lingered in the Library.
- Delete progress (session, session group, episode, or full entry) now shows app-wide as a progress bar plus a status toast, staying visible across tabs and windows instead of disappearing when you switch away.
- Deletes are dramatically faster on large libraries, and opening the Vocabulary tab no longer stalls; the first launch after upgrading migrates the stats database in place to support this.
### Changed
- **Clipboard-Video Shortcut**
@@ -34,8 +39,9 @@
- **Word Highlighting Accuracy**
- Fixed several incorrect word highlighting and annotation cases: inconsistent part-of-speech exclusions on merged quote-particle tokens, missing annotations for rare kanji, katakana punctuation wrongly treated as non-kana noise, and certain kanji vocabulary skipped for next-level ("N+1") highlighting.
- **Character Dictionary Season Overrides**
- Manual AniList overrides for a series now stay in effect for every episode in the same season folder, even when individual episode filenames produce different automatic guesses.
- **AniList Season Resolution**
- Season 2 and later episodes now resolve to the correct AniList entry by walking sequel relations instead of guessing from the title, so watch progress, the character dictionary, and cover art for later seasons no longer silently fall back to season 1.
- Manual AniList overrides now stay in effect for every episode in the same season (by folder and detected season), and setting an override now fixes both the character dictionary and AniList watch progress together instead of needing separate corrections.
- **Startup Playback Pausing Too Early**
- Fixed playback resuming before subtitle processing finished warming up, which could briefly show untranslated subtitles right after opening a video.
@@ -53,6 +59,9 @@
- Hardened the stats server against malformed requests, stalled AniList lookups, media mismatches during word mining, and missing Yomitan connections.
- AnkiConnect settings validation now preserves valid custom configurations while safely falling back on invalid values instead of failing.
- **Stats Library Cover After Relink**
- Relinking a title to a different AniList entry now updates its cover art in the stats Library grid, not just the detail view, so unrelated titles no longer end up sharing the wrong cover.
- **Rofi Menu Prompt Spacing**
- Rofi menu prompts now keep a space between the prompt label and the input field instead of crowding the search placeholder text.
@@ -64,6 +73,8 @@
- refactor: split anki-connect and stats-server resolvers into modules by @ksyasuda in #169
- feat(launcher): add post-playback history menu with previous episode by @ksyasuda in #170
- Anki maturity-based known-word highlighting by @ksyasuda in #172
- fix(anilist): resolve later seasons via sequel relations, not title guessing by @ksyasuda in #173
- feat(stats): add library entry deletion and app-wide delete progress by @ksyasuda in #174
## Installation
+32
View File
@@ -0,0 +1,32 @@
## Highlights
### Added
- Kiku/Lapis Word Card Type Setting
- A new setting (Mining/Anki > Kiku/Lapis Features > "Word Card Type") lets you choose which card-type flag gets marked on Kiku/Lapis word cards, including a click-card option that SubMiner couldn't set before.
- Handy if you only want click cards flagged instead of the default word-and-sentence marking.
- Choosing a card type now clears any other flags automatically, so a note can't end up marked as two types at once.
### Fixed
- Yomitan Popup on macOS
- Fixed the popup going unresponsive after mining a card — clicks outside it no longer leak through to mpv, and the overlay no longer flickers hidden and shown.
- Scrolling over the popup now scrolls its definitions instead of seeking the video.
- YouTube Playlist Links
- Opening a video from a playlist URL (like a Watch Later link with `list=`/`index=`) no longer times out while loading subtitles, metadata, or playback info.
## What's Changed
- feat(anki): add configurable word card type for Kiku/Lapis by @ksyasuda in #175
- fix(overlay): keep Yomitan popup interactive on macOS/Windows by @ksyasuda in #177
- fix(youtube): prevent playlist URLs from stalling yt-dlp probes by @ksyasuda in #180
## Installation
See the README and docs/installation guide for full setup steps.
## Assets
- Linux: `SubMiner.AppImage`
- macOS: `SubMiner-*.dmg` and `SubMiner-*.zip`
- Windows: `SubMiner-*.exe` and `SubMiner-*-win.zip`
- Optional extras: `subminer-assets.tar.gz` and the `subminer` launcher
Note: the `subminer` wrapper script uses Bun (`#!/usr/bin/env bun`), so `bun` must be installed and on `PATH`.
+7 -74
View File
@@ -22,10 +22,12 @@ import { MediaGenerator } from './media-generator';
import path from 'path';
import {
AnkiConnectConfig,
type CardKind,
KikuDuplicateCardInfo,
KikuFieldGroupingChoice,
KikuMergePreviewResponse,
NotificationOptions,
type WordCardKind,
} from './types/anki';
import { AiConfig } from './types/integrations';
import type { KnownWordMaturityTier } from './types/subtitle';
@@ -50,6 +52,7 @@ import {
withUpdateProgress,
UiFeedbackState,
} from './anki-integration/ui-feedback';
import { applyCardKindFlagFields, resolveWordCardKindSetting } from './anki-integration/card-kinds';
import { KnownWordCacheManager } from './anki-integration/known-word-cache';
import { PollingRunner } from './anki-integration/polling';
import type { AnkiConnectProxyServer } from './anki-integration/anki-connect-proxy';
@@ -83,8 +86,6 @@ interface NoteInfo {
fields: Record<string, { value: string }>;
}
type CardKind = 'sentence' | 'audio' | 'word-and-sentence';
function trimToNonEmptyString(value: unknown): string | null {
if (typeof value !== 'string') return null;
const trimmed = value.trim();
@@ -840,6 +841,7 @@ export class AnkiIntegration {
kikuEnabled: boolean;
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
kikuDeleteDuplicateInAuto: boolean;
wordCardKind: WordCardKind;
} {
const lapis = this.getLapisConfig();
const kiku = this.getKikuConfig();
@@ -852,6 +854,7 @@ export class AnkiIntegration {
kikuEnabled: kiku.enabled,
kikuFieldGrouping: (kiku.fieldGrouping || 'disabled') as 'auto' | 'manual' | 'disabled',
kikuDeleteDuplicateInAuto: kiku.deleteDuplicateInAuto !== false,
wordCardKind: resolveWordCardKindSetting(this.config.lapisKiku?.wordCardKind),
};
}
@@ -1315,79 +1318,9 @@ export class AnkiIntegration {
availableFieldNames: string[],
cardKind: CardKind,
): void {
const audioFlagNames = ['IsAudioCard'];
if (cardKind === 'word-and-sentence') {
const wordAndSentenceFlag = this.resolveFieldName(
availableFieldNames,
'IsWordAndSentenceCard',
);
if (!wordAndSentenceFlag) {
return;
}
updatedFields[wordAndSentenceFlag] = 'x';
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
if (sentenceFlag && sentenceFlag !== wordAndSentenceFlag) {
updatedFields[sentenceFlag] = '';
}
for (const audioFlagName of audioFlagNames) {
const resolved = this.resolveFieldName(availableFieldNames, audioFlagName);
if (resolved && resolved !== wordAndSentenceFlag) {
updatedFields[resolved] = '';
}
}
return;
}
if (cardKind === 'sentence') {
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
if (sentenceFlag) {
updatedFields[sentenceFlag] = 'x';
}
for (const audioFlagName of audioFlagNames) {
const resolved = this.resolveFieldName(availableFieldNames, audioFlagName);
if (resolved && resolved !== sentenceFlag) {
updatedFields[resolved] = '';
}
}
const wordAndSentenceFlag = this.resolveFieldName(
availableFieldNames,
'IsWordAndSentenceCard',
);
if (wordAndSentenceFlag && wordAndSentenceFlag !== sentenceFlag) {
updatedFields[wordAndSentenceFlag] = '';
}
return;
}
const resolvedAudioFlags = Array.from(
new Set(
audioFlagNames
.map((name) => this.resolveFieldName(availableFieldNames, name))
.filter((name): name is string => Boolean(name)),
),
applyCardKindFlagFields(updatedFields, cardKind, (preferredName) =>
this.resolveFieldName(availableFieldNames, preferredName),
);
const audioFlagName = resolvedAudioFlags[0] || null;
if (audioFlagName) {
updatedFields[audioFlagName] = 'x';
}
for (const extraAudioFlag of resolvedAudioFlags.slice(1)) {
updatedFields[extraAudioFlag] = '';
}
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
if (sentenceFlag && sentenceFlag !== audioFlagName) {
updatedFields[sentenceFlag] = '';
}
const wordAndSentenceFlag = this.resolveFieldName(availableFieldNames, 'IsWordAndSentenceCard');
if (wordAndSentenceFlag && wordAndSentenceFlag !== audioFlagName) {
updatedFields[wordAndSentenceFlag] = '';
}
}
private async showNotification(
@@ -4,29 +4,23 @@ import test from 'node:test';
import { CardCreationService } from './card-creation';
import { toMpvEdlValue } from './mpv-edl-test-utils';
import type { MediaInput } from '../media-generator';
import type { AnkiConnectConfig } from '../types/anki';
import type { AnkiConnectConfig, CardKind } from '../types/anki';
import { applyCardKindFlagFields } from './card-kinds';
type CardCreationDeps = ConstructorParameters<typeof CardCreationService>[0];
function setWordAndSentenceCardTypeFields(
function setCardTypeFields(
updatedFields: Record<string, string>,
availableFieldNames: string[],
cardKind: 'sentence' | 'audio' | 'word-and-sentence',
cardKind: CardKind,
): void {
if (cardKind !== 'word-and-sentence') return;
const resolveFieldName = (preferredName: string): string | null =>
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
const wordAndSentenceFlag = resolveFieldName('IsWordAndSentenceCard');
if (!wordAndSentenceFlag) return;
updatedFields[wordAndSentenceFlag] = 'x';
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
const resolved = resolveFieldName(flagName);
if (resolved && resolved !== wordAndSentenceFlag) {
updatedFields[resolved] = '';
}
}
applyCardKindFlagFields(
updatedFields,
cardKind,
(preferredName) =>
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ??
null,
);
}
function createManualUpdateService(overrides: Partial<CardCreationDeps> = {}): {
@@ -217,7 +211,7 @@ test('manual clipboard subtitle update marks Kiku word cards as word-and-sentenc
kikuFieldGrouping: 'disabled',
kikuDeleteDuplicateInAuto: false,
}),
setCardTypeFields: setWordAndSentenceCardTypeFields,
setCardTypeFields,
});
await service.updateLastAddedFromClipboard('字幕');
+6 -10
View File
@@ -3,7 +3,7 @@ import {
getConfiguredWordFieldName,
getPreferredWordValueFromExtractedFields,
} from '../anki-field-config';
import { AnkiConnectConfig } from '../types/anki';
import { AnkiConnectConfig, type CardKind, type WordCardKind } from '../types/anki';
import { createLogger } from '../logger';
import type { MediaInput } from '../media-input';
import { SubtitleTimingTracker } from '../subtitle-timing-tracker';
@@ -15,7 +15,7 @@ import {
resolveAudioStreamIndexForMediaGeneration,
type MediaGenerationInputResolverOptions,
} from './media-source';
import { shouldMarkWordAndSentenceCard } from './note-field-utils';
import { resolveWordCardKind } from './note-field-utils';
import type { PendingYoutubeMediaUpdate } from './pending-youtube-media';
import { resolveMpvVolumeScale } from './mpv-volume';
@@ -42,8 +42,6 @@ export interface CardCreationNoteInfo {
fields: Record<string, { value: string }>;
}
type CardKind = 'sentence' | 'audio' | 'word-and-sentence';
interface CardCreationClient {
addNote(
deck: string,
@@ -136,6 +134,7 @@ interface CardCreationDeps {
kikuEnabled: boolean;
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
kikuDeleteDuplicateInAuto: boolean;
wordCardKind?: WordCardKind;
};
getFallbackDurationSeconds: () => number;
appendKnownWordsFromNoteInfo: (noteInfo: CardCreationNoteInfo) => void;
@@ -274,12 +273,9 @@ export class CardCreationService {
if (sentenceField) {
const processedSentence = this.deps.processSentence(sentence, fields);
updatedFields[sentenceField] = processedSentence;
if (shouldMarkWordAndSentenceCard(noteInfo, sentenceCardConfig)) {
this.deps.setCardTypeFields(
updatedFields,
Object.keys(noteInfo.fields),
'word-and-sentence',
);
const wordCardKind = resolveWordCardKind(noteInfo, sentenceCardConfig);
if (wordCardKind) {
this.deps.setCardTypeFields(updatedFields, Object.keys(noteInfo.fields), wordCardKind);
}
updatePerformed = true;
}
+64
View File
@@ -0,0 +1,64 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { applyCardKindFlagFields } from './card-kinds';
function resolverFor(availableFieldNames: string[]) {
return (preferredName: string): string | null =>
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
}
const KIKU_FLAG_FIELDS = ['IsWordAndSentenceCard', 'IsClickCard', 'IsSentenceCard', 'IsAudioCard'];
test('flags the requested card kind and clears the others', () => {
const fields: Record<string, string> = {};
applyCardKindFlagFields(fields, 'click', resolverFor(KIKU_FLAG_FIELDS));
assert.deepEqual(fields, {
IsClickCard: 'x',
IsWordAndSentenceCard: '',
IsSentenceCard: '',
IsAudioCard: '',
});
});
test('matches flag fields case-insensitively', () => {
const fields: Record<string, string> = {};
applyCardKindFlagFields(fields, 'word-and-sentence', resolverFor(['iswordandsentencecard']));
assert.deepEqual(fields, { iswordandsentencecard: 'x' });
});
test('leaves flags untouched when the note type has no flag for a word card kind', () => {
const fields: Record<string, string> = {};
applyCardKindFlagFields(
fields,
'click',
resolverFor(['IsWordAndSentenceCard', 'IsSentenceCard']),
);
assert.deepEqual(fields, {});
});
test('clears stale flags for explicit mine actions even without the target flag', () => {
const fields: Record<string, string> = {};
applyCardKindFlagFields(
fields,
'audio',
resolverFor(['IsWordAndSentenceCard', 'IsSentenceCard']),
);
assert.deepEqual(fields, { IsWordAndSentenceCard: '', IsSentenceCard: '' });
});
test('does not blank the target flag it just set', () => {
const fields: Record<string, string> = {};
applyCardKindFlagFields(fields, 'sentence', resolverFor(['IsSentenceCard']));
assert.deepEqual(fields, { IsSentenceCard: 'x' });
});
+63
View File
@@ -0,0 +1,63 @@
import type { CardKind, WordCardKind } from '../types/anki';
/**
* Kiku/Lapis note types decide which card a note generates from mutually exclusive
* `Is...Card` flag fields. Setting one always means clearing the others.
*/
export const CARD_KIND_FLAG_FIELDS: Record<CardKind, string> = {
'word-and-sentence': 'IsWordAndSentenceCard',
click: 'IsClickCard',
sentence: 'IsSentenceCard',
audio: 'IsAudioCard',
};
export const WORD_CARD_KINDS: readonly WordCardKind[] = [
'word-and-sentence',
'click',
'sentence',
'audio',
'none',
];
export const DEFAULT_WORD_CARD_KIND: WordCardKind = 'word-and-sentence';
/**
* Card kinds SubMiner marks on its own initiative (word cards). They are only applied
* when the note type actually carries the matching flag field, so plain note types keep
* their fields untouched.
*/
const IMPLICIT_CARD_KINDS = new Set<CardKind>(['word-and-sentence', 'click']);
export function isWordCardKind(value: unknown): value is WordCardKind {
return typeof value === 'string' && WORD_CARD_KINDS.includes(value as WordCardKind);
}
export function resolveWordCardKindSetting(value: unknown): WordCardKind {
return isWordCardKind(value) ? value : DEFAULT_WORD_CARD_KIND;
}
/**
* Flags `cardKind` on the note and clears every other card-kind flag it has, so the note
* never ends up claiming to be two kinds of card at once.
*/
export function applyCardKindFlagFields(
updatedFields: Record<string, string>,
cardKind: CardKind,
resolveFieldName: (preferredName: string) => string | null,
): void {
const targetFlag = resolveFieldName(CARD_KIND_FLAG_FIELDS[cardKind]);
if (!targetFlag && IMPLICIT_CARD_KINDS.has(cardKind)) {
return;
}
if (targetFlag) {
updatedFields[targetFlag] = 'x';
}
for (const [kind, flagName] of Object.entries(CARD_KIND_FLAG_FIELDS)) {
if (kind === cardKind) continue;
const resolved = resolveFieldName(flagName);
if (resolved && resolved !== targetFlag) {
updatedFields[resolved] = '';
}
}
}
@@ -0,0 +1,118 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { resolveWordCardKind, type NoteFieldValueInfo } from './note-field-utils';
function kikuNote(values: Record<string, string> = {}): NoteFieldValueInfo {
const defaults: Record<string, string> = {
Expression: '単語',
Sentence: '',
IsWordAndSentenceCard: '',
IsClickCard: '',
IsSentenceCard: '',
IsAudioCard: '',
};
return {
fields: Object.fromEntries(
Object.entries({ ...defaults, ...values }).map(([name, value]) => [name, { value }]),
),
};
}
test('marks word-and-sentence cards by default when Kiku is enabled', () => {
assert.equal(
resolveWordCardKind(kikuNote(), { lapisEnabled: false, kikuEnabled: true }),
'word-and-sentence',
);
});
test('honors the configured word card kind', () => {
assert.equal(
resolveWordCardKind(kikuNote(), {
lapisEnabled: false,
kikuEnabled: true,
wordCardKind: 'click',
}),
'click',
);
});
test('marks nothing when neither Kiku nor Lapis is enabled', () => {
assert.equal(
resolveWordCardKind(kikuNote(), {
lapisEnabled: false,
kikuEnabled: false,
wordCardKind: 'click',
}),
null,
);
});
test('marks nothing when the word card kind is "none"', () => {
assert.equal(
resolveWordCardKind(kikuNote(), {
lapisEnabled: true,
kikuEnabled: false,
wordCardKind: 'none',
}),
null,
);
});
test('falls back to the default kind for an unrecognized setting', () => {
assert.equal(
resolveWordCardKind(kikuNote(), {
lapisEnabled: false,
kikuEnabled: true,
wordCardKind: 'bogus' as never,
}),
'word-and-sentence',
);
});
test('marks nothing when the note type lacks the configured flag field', () => {
const note: NoteFieldValueInfo = {
fields: { Expression: { value: '単語' }, Sentence: { value: '' } },
};
assert.equal(
resolveWordCardKind(note, { lapisEnabled: false, kikuEnabled: true, wordCardKind: 'click' }),
null,
);
});
test('leaves cards already mined as sentence or audio cards alone', () => {
for (const flagField of ['IsSentenceCard', 'IsAudioCard']) {
assert.equal(
resolveWordCardKind(kikuNote({ [flagField]: 'x' }), {
lapisEnabled: false,
kikuEnabled: true,
wordCardKind: 'click',
}),
null,
flagField,
);
}
});
test('re-affirms the configured kind when the note already carries its flag', () => {
assert.equal(
resolveWordCardKind(kikuNote({ IsSentenceCard: 'x' }), {
lapisEnabled: false,
kikuEnabled: true,
wordCardKind: 'sentence',
}),
'sentence',
);
});
test('overrides a differently flagged word card', () => {
assert.equal(
resolveWordCardKind(kikuNote({ IsWordAndSentenceCard: 'x' }), {
lapisEnabled: false,
kikuEnabled: true,
wordCardKind: 'click',
}),
'click',
);
});
+60 -15
View File
@@ -1,3 +1,13 @@
import type { CardKind, WordCardKind } from '../types/anki';
import { createLogger } from '../logger';
import {
CARD_KIND_FLAG_FIELDS,
DEFAULT_WORD_CARD_KIND,
resolveWordCardKindSetting,
} from './card-kinds';
const log = createLogger('anki').child('integration.note-fields');
export interface NoteFieldValueInfo {
fields: Record<string, { value: string }>;
}
@@ -16,22 +26,57 @@ export function hasNoteFieldValue(noteInfo: NoteFieldValueInfo, preferredName: s
return (getNoteFieldValue(noteInfo, preferredName) ?? '').trim().length > 0;
}
export function shouldMarkWordAndSentenceCard(
noteInfo: NoteFieldValueInfo,
sentenceCardConfig: { lapisEnabled: boolean; kikuEnabled: boolean },
): boolean {
if (!sentenceCardConfig.lapisEnabled && !sentenceCardConfig.kikuEnabled) {
return false;
}
/** Flags set only by an explicit mine action; a note carrying one is not a word card. */
const EXPLICIT_CARD_FLAG_FIELDS = [CARD_KIND_FLAG_FIELDS.sentence, CARD_KIND_FLAG_FIELDS.audio];
const wordAndSentenceValue = getNoteFieldValue(noteInfo, 'IsWordAndSentenceCard');
if (wordAndSentenceValue === null) {
return false;
const warnedMissingFlagFields = new Set<CardKind>();
function warnMissingFlagFieldOnce(wordCardKind: CardKind, flagField: string): void {
if (wordCardKind === DEFAULT_WORD_CARD_KIND || warnedMissingFlagFields.has(wordCardKind)) {
// The default kind is also the fallback for plain note types, so its absence is expected.
return;
}
if (wordAndSentenceValue.trim().length > 0) {
return true;
}
return (
!hasNoteFieldValue(noteInfo, 'IsSentenceCard') && !hasNoteFieldValue(noteInfo, 'IsAudioCard')
warnedMissingFlagFields.add(wordCardKind);
log.warn(
`Word card type "${wordCardKind}" is configured but the note has no ${flagField} field; leaving card type flags unchanged.`,
);
}
/**
* Card kind to flag when SubMiner fills a word card's sentence, or null to leave the
* card-kind flags alone. Kiku/Lapis only: other note types have no such fields.
*/
export function resolveWordCardKind(
noteInfo: NoteFieldValueInfo,
sentenceCardConfig: {
lapisEnabled: boolean;
kikuEnabled: boolean;
wordCardKind?: WordCardKind;
},
): CardKind | null {
if (!sentenceCardConfig.lapisEnabled && !sentenceCardConfig.kikuEnabled) {
return null;
}
const wordCardKind = resolveWordCardKindSetting(sentenceCardConfig.wordCardKind);
if (wordCardKind === 'none') {
return null;
}
const flagField = CARD_KIND_FLAG_FIELDS[wordCardKind];
const flagValue = getNoteFieldValue(noteInfo, flagField);
if (flagValue === null) {
// Note type has no flag field for the configured kind.
warnMissingFlagFieldOnce(wordCardKind, flagField);
return null;
}
if (flagValue.trim().length > 0) {
return wordCardKind;
}
const alreadyExplicitCard = EXPLICIT_CARD_FLAG_FIELDS.some(
(fieldName) =>
fieldName.toLowerCase() !== flagField.toLowerCase() && hasNoteFieldValue(noteInfo, fieldName),
);
return alreadyExplicitCard ? null : wordCardKind;
}
@@ -6,26 +6,21 @@ import {
type NoteUpdateWorkflowNoteInfo,
} from './note-update-workflow';
import type { SubtitleMiningContext } from '../types/subtitle';
import type { CardKind } from '../types/anki';
import { applyCardKindFlagFields } from './card-kinds';
function setWordAndSentenceCardTypeFields(
function setCardTypeFields(
updatedFields: Record<string, string>,
availableFieldNames: string[],
cardKind: 'word-and-sentence',
cardKind: CardKind,
): void {
assert.equal(cardKind, 'word-and-sentence');
const resolveFieldName = (preferredName: string): string | null =>
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
const wordAndSentenceFlag = resolveFieldName('IsWordAndSentenceCard');
if (!wordAndSentenceFlag) return;
updatedFields[wordAndSentenceFlag] = 'x';
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
const resolved = resolveFieldName(flagName);
if (resolved && resolved !== wordAndSentenceFlag) {
updatedFields[resolved] = '';
}
}
applyCardKindFlagFields(
updatedFields,
cardKind,
(preferredName) =>
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ??
null,
);
}
function createWorkflowHarness() {
@@ -79,7 +74,7 @@ function createWorkflowHarness() {
handleFieldGroupingManual: async (_originalNoteId, _newNoteId, _newNoteInfo, _expression) =>
false,
processSentence: (text: string, _noteFields: Record<string, string>) => text,
setCardTypeFields: setWordAndSentenceCardTypeFields,
setCardTypeFields,
resolveConfiguredFieldName: (noteInfo: NoteUpdateWorkflowNoteInfo, preferred?: string) => {
if (!preferred) return null;
const names = Object.keys(noteInfo.fields);
@@ -183,6 +178,73 @@ test('NoteUpdateWorkflow marks enriched Kiku word cards as word-and-sentence car
});
});
test('NoteUpdateWorkflow marks the configured word card kind instead of word-and-sentence', async () => {
const harness = createWorkflowHarness();
harness.deps.getEffectiveSentenceCardConfig = () => ({
sentenceField: 'Sentence',
lapisEnabled: false,
kikuEnabled: true,
kikuFieldGrouping: 'manual',
wordCardKind: 'click',
});
harness.deps.client.notesInfo = async () =>
[
{
noteId: 42,
fields: {
Expression: { value: 'taberu' },
Sentence: { value: '' },
IsWordAndSentenceCard: { value: 'x' },
IsClickCard: { value: '' },
IsSentenceCard: { value: '' },
IsAudioCard: { value: '' },
},
},
] satisfies NoteUpdateWorkflowNoteInfo[];
await harness.workflow.execute(42);
assert.equal(harness.updates.length, 1);
assert.deepEqual(harness.updates[0]?.fields, {
Sentence: 'subtitle-text',
IsClickCard: 'x',
IsWordAndSentenceCard: '',
IsSentenceCard: '',
IsAudioCard: '',
});
});
test('NoteUpdateWorkflow leaves card type flags alone when the word card kind is none', async () => {
const harness = createWorkflowHarness();
harness.deps.getEffectiveSentenceCardConfig = () => ({
sentenceField: 'Sentence',
lapisEnabled: false,
kikuEnabled: true,
kikuFieldGrouping: 'manual',
wordCardKind: 'none',
});
harness.deps.client.notesInfo = async () =>
[
{
noteId: 42,
fields: {
Expression: { value: 'taberu' },
Sentence: { value: '' },
IsWordAndSentenceCard: { value: '' },
IsSentenceCard: { value: '' },
IsAudioCard: { value: '' },
},
},
] satisfies NoteUpdateWorkflowNoteInfo[];
await harness.workflow.execute(42);
assert.equal(harness.updates.length, 1);
assert.deepEqual(harness.updates[0]?.fields, {
Sentence: 'subtitle-text',
});
});
test('NoteUpdateWorkflow does not set Kiku card flags when Lapis and Kiku are disabled', async () => {
const harness = createWorkflowHarness();
harness.deps.client.notesInfo = async () =>
+7 -8
View File
@@ -1,7 +1,8 @@
import { DEFAULT_ANKI_CONNECT_CONFIG } from '../config';
import { getPreferredWordValueFromExtractedFields } from '../anki-field-config';
import type { SubtitleMiningContext } from '../types/subtitle';
import { shouldMarkWordAndSentenceCard } from './note-field-utils';
import type { CardKind, WordCardKind } from '../types/anki';
import { resolveWordCardKind } from './note-field-utils';
export interface NoteUpdateWorkflowNoteInfo {
noteId: number;
@@ -39,6 +40,7 @@ export interface NoteUpdateWorkflowDeps {
lapisEnabled: boolean;
kikuEnabled: boolean;
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
wordCardKind?: WordCardKind;
};
appendKnownWordsFromNoteInfo: (noteInfo: NoteUpdateWorkflowNoteInfo) => void;
extractFields: (fields: Record<string, { value: string }>) => Record<string, string>;
@@ -67,7 +69,7 @@ export interface NoteUpdateWorkflowDeps {
setCardTypeFields: (
updatedFields: Record<string, string>,
availableFieldNames: string[],
cardKind: 'word-and-sentence',
cardKind: CardKind,
) => void;
resolveConfiguredFieldName: (
noteInfo: NoteUpdateWorkflowNoteInfo,
@@ -207,12 +209,9 @@ export class NoteUpdateWorkflow {
if (sentenceField && currentSubtitleText) {
const processedSentence = this.deps.processSentence(currentSubtitleText, fields);
updatedFields[sentenceField] = processedSentence;
if (shouldMarkWordAndSentenceCard(noteInfo, sentenceCardConfig)) {
this.deps.setCardTypeFields(
updatedFields,
Object.keys(noteInfo.fields),
'word-and-sentence',
);
const wordCardKind = resolveWordCardKind(noteInfo, sentenceCardConfig);
if (wordCardKind) {
this.deps.setCardTypeFields(updatedFields, Object.keys(noteInfo.fields), wordCardKind);
}
updatePerformed = true;
}
+8
View File
@@ -116,6 +116,10 @@ export function normalizeAnkiIntegrationConfig(config: AnkiConnectConfig): AnkiC
...DEFAULT_ANKI_CONNECT_CONFIG.isKiku,
...(config.isKiku ?? {}),
},
lapisKiku: {
...DEFAULT_ANKI_CONNECT_CONFIG.lapisKiku,
...(config.lapisKiku ?? {}),
},
} as AnkiConnectConfig;
}
@@ -205,6 +209,10 @@ export class AnkiIntegrationRuntime {
patch.isKiku !== undefined
? { ...this.config.isKiku, ...patch.isKiku }
: this.config.isKiku,
lapisKiku:
patch.lapisKiku !== undefined
? { ...this.config.lapisKiku, ...patch.lapisKiku }
: this.config.lapisKiku,
};
this.config = normalizeAnkiIntegrationConfig(mergedConfig);
this.deps.onConfigChanged?.(this.config);
+37
View File
@@ -2738,6 +2738,43 @@ test('ignores deprecated isLapis sentence-card field overrides', () => {
);
});
test('accepts a Kiku/Lapis word card kind and warns on an unknown one', () => {
const dir = makeTempDir();
fs.writeFileSync(
path.join(dir, 'config.jsonc'),
`{
"ankiConnect": {
"isKiku": { "enabled": true },
"lapisKiku": { "wordCardKind": "click" }
}
}`,
'utf-8',
);
const service = new ConfigService(dir);
assert.equal(service.getConfig().ankiConnect.lapisKiku.wordCardKind, 'click');
assert.equal(service.getWarnings().length, 0);
const invalidDir = makeTempDir();
fs.writeFileSync(
path.join(invalidDir, 'config.jsonc'),
`{
"ankiConnect": {
"lapisKiku": { "wordCardKind": "isClickCard" }
}
}`,
'utf-8',
);
const invalidService = new ConfigService(invalidDir);
assert.equal(invalidService.getConfig().ankiConnect.lapisKiku.wordCardKind, 'word-and-sentence');
assert.ok(
invalidService
.getWarnings()
.some((warning) => warning.path === 'ankiConnect.lapisKiku.wordCardKind'),
);
});
test('accepts valid ankiConnect knownWords deck object', () => {
const dir = makeTempDir();
fs.writeFileSync(
@@ -91,6 +91,9 @@ export const INTEGRATIONS_DEFAULT_CONFIG: Pick<
fieldGrouping: 'disabled',
deleteDuplicateInAuto: true,
},
lapisKiku: {
wordCardKind: 'word-and-sentence',
},
},
jimaku: {
apiBaseUrl: 'https://jimaku.cc',
@@ -1,4 +1,5 @@
import { ResolvedConfig } from '../../types/config';
import { WORD_CARD_KINDS } from '../../anki-integration/card-kinds';
import { MPV_LAUNCH_MODE_VALUES } from '../../shared/mpv-launch-mode';
import {
NOTIFICATION_TYPE_VALUES,
@@ -374,6 +375,21 @@ export function buildIntegrationConfigOptionRegistry(
defaultValue: defaultConfig.ankiConnect.isLapis.sentenceCardModel,
description: 'Note type name used by Lapis sentence cards.',
},
{
path: 'ankiConnect.lapisKiku.wordCardKind',
kind: 'enum',
enumValues: WORD_CARD_KINDS,
enumLabels: {
'word-and-sentence': 'Word and sentence card (IsWordAndSentenceCard)',
click: 'Click card (IsClickCard)',
sentence: 'Sentence card (IsSentenceCard)',
audio: 'Audio card (IsAudioCard)',
none: 'Leave card type flags untouched',
},
defaultValue: defaultConfig.ankiConnect.lapisKiku.wordCardKind,
description:
'Card-type flag SubMiner marks on Kiku/Lapis word cards. Only one flag is set at a time; the others are cleared. Requires isKiku.enabled or isLapis.enabled.',
},
{
path: 'ankiConnect.metadata.pattern',
kind: 'string',
+1 -1
View File
@@ -135,7 +135,7 @@ const INTEGRATION_TEMPLATE_SECTIONS: ConfigTemplateSection[] = [
title: 'AnkiConnect Integration',
description: ['Automatic Anki updates and media generation options.'],
notes: [
'Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.',
'Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.',
'Shared AI provider transport settings are read from top-level ai and typically require restart.',
'Most other AnkiConnect settings still require restart.',
],
+2
View File
@@ -1,6 +1,7 @@
import type { ResolveContext } from './context';
import { initializeAnkiConnectResolution } from './anki-connect/initialize';
import { applyAnkiKikuResolution } from './anki-connect/kiku';
import { applyAnkiLapisKikuResolution } from './anki-connect/lapis-kiku';
import { applyAnkiKnownWordsResolution } from './anki-connect/known-words';
import { applyAnkiLegacyResolution } from './anki-connect/legacy';
import { applyAnkiModernResolution } from './anki-connect/modern';
@@ -22,4 +23,5 @@ export function applyAnkiConnectResolution(context: ResolveContext): void {
applyAnkiLegacyResolution(context, ankiConnect, behavior, fields, media, metadata);
applyAnkiKnownWordsResolution(context, ankiConnect, behavior);
applyAnkiKikuResolution(context);
applyAnkiLapisKikuResolution(context, ankiConnect);
}
@@ -77,5 +77,8 @@ export function initializeAnkiConnectResolution(
? (ankiConnect.isKiku as (typeof context.resolved)['ankiConnect']['isKiku'])
: {}),
},
lapisKiku: {
...context.resolved.ankiConnect.lapisKiku,
},
};
}
@@ -0,0 +1,39 @@
import { isWordCardKind, WORD_CARD_KINDS } from '../../../anki-integration/card-kinds';
import { DEFAULT_CONFIG } from '../../definitions';
import type { ResolveContext } from '../context';
import { isObject } from '../shared';
export function applyAnkiLapisKikuResolution(
context: ResolveContext,
ankiConnect: Record<string, unknown>,
): void {
if (!isObject(ankiConnect.lapisKiku)) {
if (ankiConnect.lapisKiku !== undefined) {
context.warn(
'ankiConnect.lapisKiku',
ankiConnect.lapisKiku,
context.resolved.ankiConnect.lapisKiku,
'Expected object.',
);
}
return;
}
const wordCardKind = ankiConnect.lapisKiku.wordCardKind;
if (wordCardKind === undefined) {
return;
}
if (isWordCardKind(wordCardKind)) {
context.resolved.ankiConnect.lapisKiku.wordCardKind = wordCardKind;
return;
}
context.warn(
'ankiConnect.lapisKiku.wordCardKind',
wordCardKind,
DEFAULT_CONFIG.ankiConnect.lapisKiku.wordCardKind,
`Expected one of ${WORD_CARD_KINDS.join(', ')}.`,
);
context.resolved.ankiConnect.lapisKiku.wordCardKind =
DEFAULT_CONFIG.ankiConnect.lapisKiku.wordCardKind;
}
+9 -1
View File
@@ -221,6 +221,7 @@ const LABEL_OVERRIDES: Record<string, string> = {
'ankiConnect.nPlusOne.enabled': 'Enabled',
'ankiConnect.isLapis.enabled': 'Enable Lapis Features',
'ankiConnect.isKiku.enabled': 'Enable Kiku Features',
'ankiConnect.lapisKiku.wordCardKind': 'Word Card Type',
'stats.toggleKey': 'Toggle Stats Overlay',
'shortcuts.openCharacterDictionaryManager': 'Open Character Dictionary Manager',
'subtitleSidebar.pauseVideoOnHover': 'Pause Video On Hover - Sidebar',
@@ -255,6 +256,8 @@ const DESCRIPTION_OVERRIDES: Record<string, string> = {
'Enable Lapis-specific mining behavior and sentence-card model targeting. When Kiku is enabled, Lapis features still work and Kiku-specific features are added on top.',
'ankiConnect.isLapis.sentenceCardModel':
'Anki note type used for Lapis sentence cards. Select from note types reported by AnkiConnect.',
'ankiConnect.lapisKiku.wordCardKind':
'Card-type flag marked on mined word cards. Only one flag is set at a time; the others are cleared. Requires Kiku or Lapis to be enabled.',
'subtitleStyle.css':
'CSS declarations applied to primary subtitles. Includes color, background-color, and all font properties.',
'subtitleStyle.secondary.css':
@@ -401,7 +404,11 @@ function categoryAndSection(path: string): { category: ConfigSettingsCategory; s
if (path.startsWith('ankiConnect.media.')) {
return { category: 'mining-anki', section: 'Media Capture' };
}
if (path.startsWith('ankiConnect.isKiku.') || path.startsWith('ankiConnect.isLapis.')) {
if (
path.startsWith('ankiConnect.isKiku.') ||
path.startsWith('ankiConnect.isLapis.') ||
path.startsWith('ankiConnect.lapisKiku.')
) {
return { category: 'mining-anki', section: 'Kiku/Lapis Features' };
}
if (path.startsWith('ankiConnect.ai.')) {
@@ -702,6 +709,7 @@ function restartBehaviorForPath(path: string): ConfigSettingsRestartBehavior {
path === 'ankiConnect.fields.miscInfo' ||
path === 'ankiConnect.isLapis.sentenceCardModel' ||
path === 'ankiConnect.isKiku.fieldGrouping' ||
path === 'ankiConnect.lapisKiku.wordCardKind' ||
path === 'mpv.aniskipEnabled' ||
path === 'mpv.aniskipButtonKey' ||
path === 'stats.toggleKey' ||
@@ -1100,12 +1100,56 @@ describe('stats server API routes', () => {
assert.deepEqual(assignments, [{ animeId: 1, body }]);
});
it('GET /api/stats/anime/:animeId/cover returns cover art', async () => {
it('GET /api/stats/anime/:animeId/cover returns revalidated cover art', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/1/cover');
assert.equal(res.status, 200);
assert.equal(res.headers.get('content-type'), 'image/jpeg');
assert.equal(res.headers.get('cache-control'), 'public, max-age=86400');
assert.equal(res.headers.get('cache-control'), 'no-cache');
assert.ok(res.headers.get('etag'));
});
it('GET /api/stats/anime/:animeId/cover answers 304 for a matching ETag', async () => {
const app = createStatsApp(createMockTracker());
const first = await app.request('/api/stats/anime/1/cover');
const etag = first.headers.get('etag');
assert.ok(etag);
const cached = await app.request('/api/stats/anime/1/cover', {
headers: { 'If-None-Match': `W/${etag}, "other"` },
});
assert.equal(cached.status, 304);
assert.equal(cached.headers.get('etag'), etag);
});
it('GET /api/stats/anime/:animeId/cover resends art when the ETag no longer matches', async () => {
// A relinked AniList entry swaps the bytes behind the same cover URL.
let coverBlob = Buffer.from([0xff, 0xd8, 0xff, 0xd9]);
const app = createStatsApp(
createMockTracker({
getAnimeCoverArt: async () => ({
videoId: 1,
anilistId: 21858,
coverUrl: 'https://example.com/cover.jpg',
coverBlob,
titleRomaji: 'Little Witch Academia',
titleEnglish: 'Little Witch Academia',
episodesTotal: 25,
fetchedAtMs: Date.now(),
}),
}),
);
const first = await app.request('/api/stats/anime/1/cover');
const staleEtag = first.headers.get('etag');
assert.ok(staleEtag);
coverBlob = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
const res = await app.request('/api/stats/anime/1/cover', {
headers: { 'If-None-Match': staleEtag },
});
assert.equal(res.status, 200);
assert.equal(res.headers.get('content-type'), 'image/png');
assert.notEqual(res.headers.get('etag'), staleEtag);
});
it('GET /api/stats/anime/:animeId/cover serves detected cover MIME type', async () => {
@@ -2410,6 +2454,80 @@ Aligned English subtitle
});
});
it('POST /api/stats/mine-card marks the configured Kiku word card kind', async () => {
await withTempDir(async (dir) => {
const sourcePath = path.join(dir, 'episode.mkv');
fs.writeFileSync(sourcePath, 'fake media');
await withFakeAnkiConnect(
async (requests, url) => {
const app = createStatsApp(createMockTracker(), {
addYomitanNote: async () => 777,
createMediaGenerator: () => ({
generateAudio: async () => null,
generateScreenshot: async () => null,
generateAnimatedImage: async () => null,
}),
ankiConnectConfig: {
url,
deck: 'Mining',
fields: {
image: 'Picture',
sentence: 'Sentence',
},
media: {
generateAudio: false,
generateImage: false,
},
isKiku: {
enabled: true,
fieldGrouping: 'disabled',
deleteDuplicateInAuto: true,
},
lapisKiku: {
wordCardKind: 'click',
},
},
});
const res = await app.request('/api/stats/mine-card?mode=word', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
sourcePath,
startMs: 1_000,
endMs: 2_000,
sentence: '猫を見た',
word: '猫',
videoTitle: 'Episode 1',
}),
});
const body = await res.json();
assert.equal(res.status, 200, JSON.stringify(body));
const updateRequest = requests.find((request) => request.action === 'updateNoteFields');
const fields = updateRequest?.params?.note?.fields ?? {};
assert.equal(fields.IsClickCard, 'x');
assert.equal(fields.IsWordAndSentenceCard, '');
assert.equal(fields.IsSentenceCard, '');
assert.equal(fields.IsAudioCard, '');
},
{
notesInfoFields: {
Expression: { value: '猫' },
Sentence: { value: '' },
Picture: { value: '' },
IsWordAndSentenceCard: { value: '' },
IsClickCard: { value: '' },
IsSentenceCard: { value: '' },
IsAudioCard: { value: '' },
},
},
);
});
});
it('POST /api/stats/mine-card writes word mining sentence audio and image together', async () => {
await withTempDir(async (dir) => {
const sourcePath = path.join(dir, 'episode.mkv');
@@ -2873,6 +2991,39 @@ Aligned English subtitle
assert.equal(deleteCalls, 0);
});
it('DELETE /api/stats/anime/:animeId deletes the whole library entry', async () => {
let deletedAnimeId: number | null = null;
const app = createStatsApp(
createMockTracker({
deleteAnime: async (animeId: number) => {
deletedAnimeId = animeId;
},
} as Partial<ImmersionTrackerService>),
);
const res = await app.request('/api/stats/anime/7', { method: 'DELETE' });
assert.equal(res.status, 200);
assert.equal(deletedAnimeId, 7);
assert.deepEqual(await res.json(), { ok: true });
});
it('DELETE /api/stats/anime/:animeId rejects non-positive anime ids', async () => {
let deleteCalls = 0;
const app = createStatsApp(
createMockTracker({
deleteAnime: async () => {
deleteCalls += 1;
},
} as Partial<ImmersionTrackerService>),
);
const res = await app.request('/api/stats/anime/0', { method: 'DELETE' });
assert.equal(res.status, 400);
assert.equal(deleteCalls, 0);
});
it('POST /api/stats/anki/browse returns 400 for missing noteId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anki/browse', { method: 'POST' });
+1
View File
@@ -85,6 +85,7 @@ const HOT_RELOAD_EXACT_OR_PREFIX_PATHS = [
'ankiConnect.fields.miscInfo',
'ankiConnect.isLapis.sentenceCardModel',
'ankiConnect.isKiku.fieldGrouping',
'ankiConnect.lapisKiku.wordCardKind',
] as const;
function hotReloadFieldForChangedPath(path: string): string | null {
@@ -1466,6 +1466,190 @@ test('deleteVideo ignores the currently active video and keeps new writes flusha
}
});
/**
* Attach the derived rows a real session would produce to every recorded line
* of `animeId`: word/kanji entries and their occurrences, monthly rollups, and
* cover art backed by a shared blob.
*/
function seedDerivedAnimeData(db: DatabaseSync, animeId: number): void {
const lines = db
.prepare(
'SELECT line_id AS lineId, CREATED_DATE AS seenMs FROM imm_subtitle_lines WHERE anime_id = ?',
)
.all(animeId) as Array<{ lineId: number; seenMs: number }>;
assert.ok(lines.length > 0, 'expected recorded subtitle lines to decorate');
db.prepare(
`INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
VALUES (9001, '天気', '天気', 'てんき', 'noun', '名詞', 0, 0, 0)`,
).run();
db.prepare(
`INSERT INTO imm_kanji(id, kanji, first_seen, last_seen, frequency) VALUES (9101, '気', 0, 0, 0)`,
).run();
const insertWordOccurrence = db.prepare(
'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, 9001, 1, ?)',
);
const insertKanjiOccurrence = db.prepare(
'INSERT INTO imm_kanji_line_occurrences(line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, 9101, 1, ?)',
);
for (const line of lines) {
insertWordOccurrence.run(line.lineId, line.seenMs);
insertKanjiOccurrence.run(line.lineId, line.seenMs);
}
db.prepare(
`UPDATE imm_words SET frequency = ?, first_seen = ?, last_seen = ? WHERE id = 9001`,
).run(lines.length, 0, 0);
const videoIds = (
db
.prepare('SELECT video_id AS videoId FROM imm_videos WHERE anime_id = ?')
.all(animeId) as Array<{ videoId: number }>
).map((row) => row.videoId);
const insertMonthlyRollup = db.prepare(
`INSERT INTO imm_monthly_rollups(rollup_month, video_id, total_sessions, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (202401, ?, 1, '0', '0')`,
);
const insertArt = db.prepare(
`INSERT INTO imm_media_art(video_id, anilist_id, cover_url, cover_blob, cover_blob_hash, fetched_at_ms, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (?, 4242, 'https://example.test/cover.jpg', NULL, 'deadbeef', '0', '0', '0')`,
);
db.prepare(
`INSERT INTO imm_cover_art_blobs(blob_hash, cover_blob, CREATED_DATE, LAST_UPDATE_DATE)
VALUES ('deadbeef', X'FFD8FFD9', '0', '0')`,
).run();
for (const videoId of videoIds) {
insertMonthlyRollup.run(videoId);
insertArt.run(videoId);
}
}
test('deleteAnime removes every episode, session and library row for the title', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
try {
const Ctor = await loadTrackerCtor();
for (const episode of ['S02E05', 'S02E06']) {
tracker = new Ctor({ dbPath });
tracker.handleMediaChange(`/tmp/Little Witch Academia ${episode}.mkv`, `Episode ${episode}`);
await waitForPendingAnimeMetadata(tracker);
tracker.recordSubtitleLine('今日は晴れです', 0, 1.2);
tracker.recordCardsMined(1);
tracker.destroy();
tracker = null;
}
tracker = new Ctor({ dbPath });
const privateApi = tracker as unknown as { db: DatabaseSync };
const animeId = (
privateApi.db.prepare('SELECT anime_id FROM imm_anime LIMIT 1').get() as {
anime_id: number;
} | null
)?.anime_id;
assert.ok(animeId);
// The tokenizer does not run in this harness, so attach vocabulary, kanji,
// rollups and cover art to the recorded lines by hand. Without them the
// "everything is gone" assertions below would pass against empty tables.
seedDerivedAnimeData(privateApi.db, animeId);
const countOf = (sql: string): number =>
(privateApi.db.prepare(sql).get() as { total: number }).total;
for (const table of [
'imm_words',
'imm_kanji',
'imm_word_line_occurrences',
'imm_kanji_line_occurrences',
'imm_daily_rollups',
'imm_monthly_rollups',
'imm_media_art',
'imm_cover_art_blobs',
]) {
assert.ok(
countOf(`SELECT COUNT(*) AS total FROM ${table}`) > 0,
`precondition: ${table} should hold rows before the delete`,
);
}
const libraryBefore = await tracker.getAnimeLibrary();
assert.equal(libraryBefore.length, 1);
assert.equal(libraryBefore[0]?.episodeCount, 2);
await tracker.deleteAnime(animeId);
const libraryAfter = await tracker.getAnimeLibrary();
assert.equal(libraryAfter.length, 0);
for (const table of [
'imm_anime',
'imm_lifetime_anime',
'imm_videos',
'imm_sessions',
'imm_subtitle_lines',
'imm_daily_rollups',
'imm_monthly_rollups',
'imm_lifetime_media',
'imm_words',
'imm_kanji',
'imm_word_line_occurrences',
'imm_kanji_line_occurrences',
'imm_media_art',
'imm_cover_art_blobs',
]) {
assert.equal(
countOf(`SELECT COUNT(*) AS total FROM ${table}`),
0,
`${table} should be empty after deleting the only title`,
);
}
} finally {
tracker?.destroy();
cleanupDbPath(dbPath);
}
});
test('deleteAnime ignores the title of the currently active session', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
try {
const Ctor = await loadTrackerCtor();
tracker = new Ctor({ dbPath });
tracker.handleMediaChange('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5');
await waitForPendingAnimeMetadata(tracker);
const privateApi = tracker as unknown as {
db: DatabaseSync;
sessionState: { sessionId: number; videoId: number } | null;
};
const videoId = privateApi.sessionState?.videoId;
assert.ok(videoId);
const animeId = (
privateApi.db.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?').get(videoId) as {
anime_id: number | null;
} | null
)?.anime_id;
assert.ok(animeId);
await tracker.deleteAnime(animeId);
const animeCountRow = privateApi.db
.prepare('SELECT COUNT(*) AS total FROM imm_anime WHERE anime_id = ?')
.get(animeId) as { total: number };
const videoCountRow = privateApi.db
.prepare('SELECT COUNT(*) AS total FROM imm_videos WHERE video_id = ?')
.get(videoId) as { total: number };
assert.equal(animeCountRow.total, 1);
assert.equal(videoCountRow.total, 1);
} finally {
tracker?.destroy();
cleanupDbPath(dbPath);
}
});
test('handleMediaChange links parsed anime metadata on the active video row', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
@@ -83,6 +83,7 @@ import {
} from './immersion-tracker/query-library';
import {
cleanupVocabularyStats,
deleteAnime as deleteAnimeQuery,
deleteSession as deleteSessionQuery,
deleteSessions as deleteSessionsQuery,
deleteVideo as deleteVideoQuery,
@@ -733,6 +734,28 @@ export class ImmersionTrackerService {
deleteVideoQuery(this.db, videoId);
}
async deleteAnime(animeId: number): Promise<void> {
// The active video's anime link is assigned asynchronously after the title
// is parsed, so a guard reading imm_videos too early sees a null and lets
// the delete through — then the late update recreates the anime row.
const pendingVideoId = this.sessionState?.videoId;
if (pendingVideoId !== undefined) {
await this.pendingAnimeMetadataUpdates.get(pendingVideoId);
}
const activeVideoId = this.sessionState?.videoId;
if (activeVideoId !== undefined) {
const activeAnime = this.db
.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?')
.get(activeVideoId) as { anime_id: number | null } | null;
if (activeAnime?.anime_id === animeId) {
this.logger.warn(`Ignoring delete request for active immersion anime ${animeId}`);
return;
}
}
deleteAnimeQuery(this.db, animeId);
}
async reassignAnimeAnilist(
animeId: number,
info: {
@@ -0,0 +1,331 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { Database } from '../sqlite.js';
import type { DatabaseSync } from '../sqlite.js';
import { ensureSchema } from '../storage.js';
import { deleteSession, deleteSessions, deleteVideo } from '../query-maintenance.js';
const DAY_MS = 86_400_000;
const BASE_MS = 1_700_000_000_000;
function makeDbPath(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-lexical-removal-test-'));
return path.join(dir, 'immersion.sqlite');
}
function cleanupDbPath(dbPath: string): void {
const dir = path.dirname(dbPath);
if (!fs.existsSync(dir)) return;
fs.rmSync(dir, { recursive: true, force: true });
}
/**
* Seed two episodes of one anime, each with one ended session.
*
* `lines` places a word occurrence on a specific day so tests can control which
* session holds a word's first/last occurrence.
*/
function seed(
db: DatabaseSync,
lines: Array<{ session: 1 | 2; wordId: number; dayOffset: number; count?: number }>,
options: { legacyRows?: boolean } = {},
): void {
db.exec(`
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 'show', 'Show', ${BASE_MS}, ${BASE_MS});
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 'v1', 1, 'Ep 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS}),
(2, 'v2', 1, 'Ep 2', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 's1', 1, '${BASE_MS}', '${BASE_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS}),
(2, 's2', 2, '${BASE_MS + DAY_MS}', '${BASE_MS + DAY_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS});
`);
const insertLine = db.prepare(
`INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (?, ?, ?, 1, ?, ?, ?, ?)`,
);
const insertWord = db.prepare(
`INSERT OR IGNORE INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
VALUES (?, ?, ?, '', 'noun', '名詞', 0, 0, 0)`,
);
// `legacyRows` reproduces databases written before the seen_ms column existed,
// where the timestamp has to be read back off the subtitle line.
const insertOccurrence = options.legacyRows
? db.prepare(
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (?, ?, ?)`,
)
: db.prepare(
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)`,
);
let lineId = 0;
for (const line of lines) {
lineId += 1;
const seenMs = BASE_MS + line.dayOffset * DAY_MS;
insertLine.run(lineId, line.session, line.session, lineId, `line ${lineId}`, seenMs, seenMs);
insertWord.run(line.wordId, `語${line.wordId}`, `語${line.wordId}`);
if (options.legacyRows) {
insertOccurrence.run(lineId, line.wordId, line.count ?? 1);
} else {
insertOccurrence.run(lineId, line.wordId, line.count ?? 1, seenMs);
}
}
// Match what the tracker maintains: aggregates derived from the occurrences.
db.exec(`
UPDATE imm_words SET
frequency = (
SELECT COALESCE(SUM(o.occurrence_count), 0)
FROM imm_word_line_occurrences o WHERE o.word_id = imm_words.id
),
first_seen = (
SELECT MIN(sl.CREATED_DATE) / 1000
FROM imm_word_line_occurrences o
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
WHERE o.word_id = imm_words.id
),
last_seen = (
SELECT MAX(sl.LAST_UPDATE_DATE) / 1000
FROM imm_word_line_occurrences o
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
WHERE o.word_id = imm_words.id
)
`);
}
function createDb(
lines: Parameters<typeof seed>[1],
options: Parameters<typeof seed>[2] = {},
): { db: DatabaseSync; dbPath: string } {
const dbPath = makeDbPath();
const db = new Database(dbPath);
ensureSchema(db);
seed(db, lines, options);
return { db, dbPath };
}
function readWord(
db: DatabaseSync,
wordId: number,
): { frequency: number; firstSeen: number; lastSeen: number } | null {
return (
(db
.prepare(
'SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen FROM imm_words WHERE id = ?',
)
.get(wordId) as { frequency: number; firstSeen: number; lastSeen: number } | null) ?? null
);
}
test('deleting a session subtracts only the occurrences it removed', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 10, dayOffset: 0, count: 3 },
{ session: 2, wordId: 10, dayOffset: 1, count: 4 },
]);
try {
assert.equal(readWord(db, 10)?.frequency, 7);
deleteSession(db, 1);
assert.equal(readWord(db, 10)?.frequency, 4, 'only session 1 occurrences are subtracted');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('deleting the session that held a word first drops the word entirely', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 11, dayOffset: 0 },
{ session: 2, wordId: 12, dayOffset: 1 },
]);
try {
deleteSession(db, 1);
assert.equal(readWord(db, 11), null, 'word seen only in the deleted session is removed');
assert.ok(readWord(db, 12), 'word seen elsewhere survives');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('deleting the earliest session moves first_seen forward to the surviving line', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 13, dayOffset: 0 },
{ session: 2, wordId: 13, dayOffset: 5 },
]);
try {
assert.equal(readWord(db, 13)?.firstSeen, Math.floor(BASE_MS / 1000));
deleteSession(db, 1);
const word = readWord(db, 13);
assert.equal(word?.frequency, 1);
assert.equal(
word?.firstSeen,
Math.floor((BASE_MS + 5 * DAY_MS) / 1000),
'first_seen advances to the remaining occurrence',
);
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 5 * DAY_MS) / 1000));
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('deleting the latest session moves last_seen back to the surviving line', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 14, dayOffset: 0 },
{ session: 2, wordId: 14, dayOffset: 5 },
]);
try {
deleteSession(db, 2);
const word = readWord(db, 14);
assert.equal(word?.frequency, 1);
assert.equal(word?.lastSeen, Math.floor(BASE_MS / 1000), 'last_seen falls back to session 1');
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('deleting an interior occurrence leaves the surrounding extremes untouched', () => {
// Session 2 carries the middle occurrence; sessions bracket it in time.
const { db, dbPath } = createDb([
{ session: 1, wordId: 15, dayOffset: 0 },
{ session: 2, wordId: 15, dayOffset: 3 },
{ session: 1, wordId: 15, dayOffset: 9 },
]);
try {
deleteSessions(db, [2]);
const word = readWord(db, 15);
assert.equal(word?.frequency, 2);
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 9 * DAY_MS) / 1000));
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('a stored frequency that has drifted low is repaired instead of dropping a live word', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 16, dayOffset: 0, count: 5 },
{ session: 2, wordId: 16, dayOffset: 4, count: 5 },
]);
try {
// Simulate drift: the stored total is lower than the occurrences justify, so
// naive subtraction would take the word to zero while rows still reference it.
db.prepare('UPDATE imm_words SET frequency = 5 WHERE id = ?').run(16);
deleteSession(db, 1);
const word = readWord(db, 16);
assert.ok(word, 'word with surviving occurrences is not deleted');
assert.equal(word?.frequency, 5, 'frequency is recomputed from the surviving occurrences');
assert.equal(word?.firstSeen, Math.floor((BASE_MS + 4 * DAY_MS) / 1000));
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('deleting a video subtracts every occurrence carried by its lines', () => {
const { db, dbPath } = createDb([
{ session: 1, wordId: 17, dayOffset: 0, count: 2 },
{ session: 1, wordId: 17, dayOffset: 1, count: 3 },
{ session: 2, wordId: 17, dayOffset: 2, count: 4 },
{ session: 2, wordId: 18, dayOffset: 2, count: 1 },
]);
try {
deleteVideo(db, 1);
assert.equal(readWord(db, 17)?.frequency, 4, 'both lines from video 1 are subtracted');
assert.equal(readWord(db, 18)?.frequency, 1, 'untouched video keeps its counts');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('occurrence rows written before the seen_ms column still resolve their dates', () => {
const { db, dbPath } = createDb(
[
{ session: 1, wordId: 20, dayOffset: 0 },
{ session: 2, wordId: 20, dayOffset: 6 },
],
{ legacyRows: true },
);
try {
assert.equal(
(
db
.prepare('SELECT COUNT(*) AS total FROM imm_word_line_occurrences WHERE seen_ms IS NULL')
.get() as { total: number }
).total,
2,
'precondition: rows carry no denormalised timestamp',
);
deleteSession(db, 1);
const word = readWord(db, 20);
assert.equal(word?.frequency, 1);
assert.equal(
word?.firstSeen,
Math.floor((BASE_MS + 6 * DAY_MS) / 1000),
'falls back to the subtitle line timestamp',
);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('upgrading an older database backfills seen_ms from the subtitle lines', () => {
const { db, dbPath } = createDb(
[
{ session: 1, wordId: 21, dayOffset: 0 },
{ session: 2, wordId: 21, dayOffset: 2 },
],
{ legacyRows: true },
);
try {
// Re-run ensureSchema the way a pre-19 database would be opened.
db.exec('DELETE FROM imm_schema_version');
db.exec(`INSERT INTO imm_schema_version(schema_version, applied_at_ms) VALUES (18, '0')`);
ensureSchema(db);
const rows = db
.prepare('SELECT line_id AS lineId, seen_ms AS seenMs FROM imm_word_line_occurrences')
.all() as Array<{ lineId: number; seenMs: number | null }>;
assert.equal(rows.length, 2);
for (const row of rows) {
assert.ok(row.seenMs, `line ${row.lineId} should have a backfilled timestamp`);
}
assert.deepEqual(
rows.map((row) => row.seenMs).sort((a, b) => Number(a) - Number(b)),
[BASE_MS, BASE_MS + 2 * DAY_MS],
);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
@@ -4993,3 +4993,145 @@ test('getTrendsDashboard librarySummary is empty when no rollups exist', () => {
cleanupDbPath(dbPath);
}
});
test('getVocabularyStats counts the distinct anime each word appeared in', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const startedAtMs = 1_700_000_000_000;
db.exec(`
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs}),
(22, 'vocab-b', 'Vocab B', ${startedAtMs}, ${startedAtMs});
`);
const videoIds = [1, 2].map((n) =>
getOrCreateVideoRecord(db, `local:/tmp/vocab-anime-${n}.mkv`, {
canonicalTitle: `Vocab Anime ${n}`,
sourcePath: `/tmp/vocab-anime-${n}.mkv`,
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
}),
);
const sessionIds = videoIds.map(
(videoId) => startSessionRecord(db, videoId, startedAtMs).sessionId,
);
// 猫 spans both titles, twice in the first, so the count has to deduplicate
// by anime rather than just tallying occurrence rows. 犬 stays in one title.
insertFilteredWordOccurrence(db, {
sessionId: sessionIds[0]!,
videoId: videoIds[0]!,
animeId: 11,
lineIndex: 1,
occurrenceCount: 5,
startedAtMs,
word: '猫',
reading: 'ねこ',
});
insertFilteredWordOccurrence(db, {
sessionId: sessionIds[0]!,
videoId: videoIds[0]!,
animeId: 11,
lineIndex: 2,
occurrenceCount: 1,
startedAtMs,
word: '猫',
reading: 'ねこ',
});
insertFilteredWordOccurrence(db, {
sessionId: sessionIds[1]!,
videoId: videoIds[1]!,
animeId: 22,
lineIndex: 3,
occurrenceCount: 4,
startedAtMs,
word: '猫',
reading: 'ねこ',
});
insertFilteredWordOccurrence(db, {
sessionId: sessionIds[0]!,
videoId: videoIds[0]!,
animeId: 11,
lineIndex: 4,
occurrenceCount: 2,
startedAtMs,
word: '犬',
reading: 'いぬ',
});
const rows = getVocabularyStats(db, 10);
const cat = rows.find((row) => row.headword === '猫');
const dog = rows.find((row) => row.headword === '犬');
assert.equal(cat?.animeCount, 2, '猫 was seen in two anime across three lines');
assert.equal(dog?.animeCount, 1, '犬 was seen in one');
assert.equal(cat?.frequency, 10, 'frequency still aggregates across both titles');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getVocabularyStats still applies part-of-speech exclusions', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const startedAtMs = 1_700_000_000_000;
db.exec(`
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs});
`);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/vocab-exclude.mkv', {
canonicalTitle: 'Vocab Exclude',
sourcePath: '/tmp/vocab-exclude.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
insertFilteredWordOccurrence(db, {
sessionId,
videoId,
animeId: 11,
lineIndex: 1,
occurrenceCount: 9,
startedAtMs,
word: '走る',
reading: 'はしる',
partOfSpeech: 'verb',
});
insertFilteredWordOccurrence(db, {
sessionId,
videoId,
animeId: 11,
lineIndex: 2,
occurrenceCount: 3,
startedAtMs,
word: '猫',
reading: 'ねこ',
partOfSpeech: 'noun',
});
const all = getVocabularyStats(db, 10);
assert.deepEqual(
all.map((row) => row.headword).sort(),
['猫', '走る'],
'precondition: both parts of speech are present',
);
const nounsOnly = getVocabularyStats(db, 10, ['verb']);
assert.deepEqual(
nounsOnly.map((row) => row.headword),
['猫'],
'excluded part of speech is filtered out even though it ranks higher',
);
assert.equal(nounsOnly[0]?.animeCount, 1, 'surviving rows keep their anime count');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
@@ -115,18 +115,29 @@ export function getVocabularyStats(
const whereClause = hasExclude
? `WHERE (part_of_speech IS NULL OR part_of_speech NOT IN (${placeholders}))`
: '';
// The page is selected before the join so `animeCount` is only computed for the
// rows being returned. Aggregating first made every request walk each word's
// entire occurrence history — seconds of blocked event loop on a large library,
// because only the ordering, not the aggregate, decides which rows survive.
const stmt = db.prepare(`
SELECT w.id AS wordId, w.headword, w.word, w.reading,
w.part_of_speech AS partOfSpeech, w.pos1, w.pos2, w.pos3,
w.frequency, w.frequency_rank AS frequencyRank,
w.first_seen AS firstSeen, w.last_seen AS lastSeen,
WITH page AS (
SELECT id, headword, word, reading, part_of_speech, pos1, pos2, pos3,
frequency, frequency_rank, first_seen, last_seen
FROM imm_words
${whereClause}
ORDER BY frequency DESC, id
LIMIT ? OFFSET ?
)
SELECT p.id AS wordId, p.headword, p.word, p.reading,
p.part_of_speech AS partOfSpeech, p.pos1, p.pos2, p.pos3,
p.frequency, p.frequency_rank AS frequencyRank,
p.first_seen AS firstSeen, p.last_seen AS lastSeen,
COUNT(DISTINCT sl.anime_id) AS animeCount
FROM imm_words w
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
FROM page p
LEFT JOIN imm_word_line_occurrences o ON o.word_id = p.id
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id AND sl.anime_id IS NOT NULL
${whereClause ? whereClause.replace('part_of_speech', 'w.part_of_speech') : ''}
GROUP BY w.id
ORDER BY w.frequency DESC LIMIT ? OFFSET ?
GROUP BY p.id
ORDER BY p.frequency DESC, p.id
`);
const visibleRows: VocabularyStatsRow[] = [];
let offset = 0;
@@ -9,14 +9,12 @@ import { PartOfSpeech, type MergedToken } from '../../../types';
import { shouldExcludeTokenFromVocabularyPersistence } from '../tokenizer/annotation-stage';
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
import {
applyLexicalRemovals,
cleanupUnusedCoverArtBlobHash,
deleteSessionsByIds,
findSharedCoverBlobHash,
getAffectedKanjiIdsForSessions,
getAffectedKanjiIdsForVideo,
getAffectedWordIdsForSessions,
getAffectedWordIdsForVideo,
refreshLexicalAggregates,
planLexicalRemovalsForSessions,
planLexicalRemovalsForVideos,
toDbMs,
toDbTimestamp,
} from './query-shared';
@@ -232,12 +230,13 @@ export async function cleanupVocabularyStats(
WHERE id = ?`,
);
const moveOccurrencesStmt = db.prepare(
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
SELECT line_id, ?, occurrence_count
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms)
SELECT line_id, ?, occurrence_count, seen_ms
FROM imm_word_line_occurrences
WHERE word_id = ?
ON CONFLICT(line_id, word_id) DO UPDATE SET
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count`,
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)`,
);
const deleteOccurrencesStmt = db.prepare(
'DELETE FROM imm_word_line_occurrences WHERE word_id = ?',
@@ -484,14 +483,16 @@ export function isVideoWatched(db: DatabaseSync, videoId: number): boolean {
export function deleteSession(db: DatabaseSync, sessionId: number): void {
const sessionIds = [sessionId];
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
db.exec('BEGIN IMMEDIATE');
try {
// Measured inside the write lock: the plan records what the delete removes,
// and applying a plan taken against a different snapshot would subtract the
// wrong totals from imm_words/imm_kanji.
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
deleteSessionsByIds(db, sessionIds);
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
applyLexicalRemovals(db, lexicalRemovals);
rebuildLifetimeSummariesInTransaction(db);
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
db.exec('COMMIT');
@@ -503,14 +504,13 @@ export function deleteSession(db: DatabaseSync, sessionId: number): void {
export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
if (sessionIds.length === 0) return;
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
db.exec('BEGIN IMMEDIATE');
try {
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
deleteSessionsByIds(db, sessionIds);
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
applyLexicalRemovals(db, lexicalRemovals);
rebuildLifetimeSummariesInTransaction(db);
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
db.exec('COMMIT');
@@ -520,24 +520,83 @@ export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
}
}
/**
* Delete an entire library entry: every episode of the anime, all of their
* sessions and derived stats, and the anime row itself.
*
* Mirrors {@link deleteVideo} per episode, but batches the lexical refresh and
* lifetime rebuild into a single transaction so a multi-episode title doesn't
* pay for one full rebuild per episode.
*/
export function deleteAnime(db: DatabaseSync, animeId: number): void {
db.exec('BEGIN IMMEDIATE');
try {
const videoIds = (
db.prepare('SELECT video_id FROM imm_videos WHERE anime_id = ?').all(animeId) as Array<{
video_id: number;
}>
).map((row) => row.video_id);
const lexicalRemovals = planLexicalRemovalsForVideos(db, videoIds);
const coverBlobHashes: string[] = [];
const sessionIds: number[] = [];
for (const videoId of videoIds) {
const artRow = db
.prepare('SELECT cover_blob_hash AS coverBlobHash FROM imm_media_art WHERE video_id = ?')
.get(videoId) as { coverBlobHash: string | null } | undefined;
if (artRow?.coverBlobHash) {
coverBlobHashes.push(artRow.coverBlobHash);
}
const sessions = db
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
.all(videoId) as Array<{ session_id: number }>;
sessionIds.push(...sessions.map((session) => session.session_id));
}
deleteSessionsByIds(db, sessionIds);
const deleteLinesStmt = db.prepare('DELETE FROM imm_subtitle_lines WHERE video_id = ?');
const deleteDailyStmt = db.prepare('DELETE FROM imm_daily_rollups WHERE video_id = ?');
const deleteMonthlyStmt = db.prepare('DELETE FROM imm_monthly_rollups WHERE video_id = ?');
const deleteArtStmt = db.prepare('DELETE FROM imm_media_art WHERE video_id = ?');
const deleteVideoStmt = db.prepare('DELETE FROM imm_videos WHERE video_id = ?');
for (const videoId of videoIds) {
deleteLinesStmt.run(videoId);
deleteDailyStmt.run(videoId);
deleteMonthlyStmt.run(videoId);
deleteArtStmt.run(videoId);
deleteVideoStmt.run(videoId);
}
for (const coverBlobHash of new Set(coverBlobHashes)) {
cleanupUnusedCoverArtBlobHash(db, coverBlobHash);
}
db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?').run(animeId);
db.prepare('DELETE FROM imm_anime WHERE anime_id = ?').run(animeId);
applyLexicalRemovals(db, lexicalRemovals);
rebuildLifetimeSummariesInTransaction(db);
db.exec('COMMIT');
} catch (error) {
db.exec('ROLLBACK');
throw error;
}
}
export function deleteVideo(db: DatabaseSync, videoId: number): void {
const artRow = db
.prepare(
`
db.exec('BEGIN IMMEDIATE');
try {
const artRow = db
.prepare(
`
SELECT cover_blob_hash AS coverBlobHash
FROM imm_media_art
WHERE video_id = ?
`,
)
.get(videoId) as { coverBlobHash: string | null } | undefined;
const affectedWordIds = getAffectedWordIdsForVideo(db, videoId);
const affectedKanjiIds = getAffectedKanjiIdsForVideo(db, videoId);
const sessions = db
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
.all(videoId) as Array<{ session_id: number }>;
)
.get(videoId) as { coverBlobHash: string | null } | undefined;
const lexicalRemovals = planLexicalRemovalsForVideos(db, [videoId]);
const sessions = db
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
.all(videoId) as Array<{ session_id: number }>;
db.exec('BEGIN IMMEDIATE');
try {
deleteSessionsByIds(
db,
sessions.map((session) => session.session_id),
@@ -548,7 +607,7 @@ export function deleteVideo(db: DatabaseSync, videoId: number): void {
db.prepare('DELETE FROM imm_media_art WHERE video_id = ?').run(videoId);
cleanupUnusedCoverArtBlobHash(db, artRow?.coverBlobHash ?? null);
db.prepare('DELETE FROM imm_videos WHERE video_id = ?').run(videoId);
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
applyLexicalRemovals(db, lexicalRemovals);
rebuildLifetimeSummariesInTransaction(db);
db.exec('COMMIT');
} catch (error) {
@@ -203,6 +203,167 @@ export function getAffectedKanjiIdsForVideo(db: DatabaseSync, videoId: number):
return getAffectedIdsForVideo(db, 'kanji', videoId);
}
/** Per-entity totals that a pending delete is about to remove. */
interface LexicalRemoval {
id: number;
removedFrequency: number;
removedFirstSeenMs: number | null;
removedLastSeenMs: number | null;
}
/** What a pending delete removes from `imm_words` and `imm_kanji`. */
export interface LexicalRemovalPlan {
words: LexicalRemoval[];
kanji: LexicalRemoval[];
}
export const EMPTY_LEXICAL_REMOVAL_PLAN: LexicalRemovalPlan = { words: [], kanji: [] };
function collectLexicalRemovals(
db: DatabaseSync,
entity: LexicalEntity,
lineScopeSql: string,
params: number[],
): LexicalRemoval[] {
const table = entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
const col = `${entity}_id`;
return db
.prepare(
`SELECT
o.${col} AS id,
COALESCE(SUM(o.occurrence_count), 0) AS removedFrequency,
MIN(COALESCE(o.seen_ms, sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS removedFirstSeenMs,
MAX(COALESCE(o.seen_ms, sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS removedLastSeenMs
FROM imm_subtitle_lines sl
JOIN ${table} o ON o.line_id = sl.line_id
WHERE ${lineScopeSql}
GROUP BY o.${col}`,
)
.all(...params) as LexicalRemoval[];
}
function planLexicalRemovals(
db: DatabaseSync,
lineScopeSql: string,
params: number[],
): LexicalRemovalPlan {
return {
words: collectLexicalRemovals(db, 'word', lineScopeSql, params),
kanji: collectLexicalRemovals(db, 'kanji', lineScopeSql, params),
};
}
/**
* Measure what deleting these sessions removes from the vocabulary tables.
*
* Must run before the rows are deleted. Reads only the subtitle lines in scope,
* unlike {@link refreshLexicalAggregates}, which re-reads every occurrence of
* every affected word across the whole library.
*/
export function planLexicalRemovalsForSessions(
db: DatabaseSync,
sessionIds: number[],
): LexicalRemovalPlan {
if (sessionIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
return planLexicalRemovals(db, `sl.session_id IN (${makePlaceholders(sessionIds)})`, sessionIds);
}
/** Measure what deleting these videos removes from the vocabulary tables. */
export function planLexicalRemovalsForVideos(
db: DatabaseSync,
videoIds: number[],
): LexicalRemovalPlan {
if (videoIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
return planLexicalRemovals(db, `sl.video_id IN (${makePlaceholders(videoIds)})`, videoIds);
}
function toStoredSeenSeconds(ms: number | null): number | null {
if (ms === null || !Number.isFinite(ms)) return null;
return Math.floor(ms / 1000);
}
/**
* Apply a removal plan to the vocabulary aggregates.
*
* Frequencies are adjusted by subtraction, which is exact and touches only the
* affected rows. `first_seen`/`last_seen` only need a rescan when the removed
* lines held the current extreme, and rows whose frequency reaches zero are
* verified against the surviving occurrences before deletion — so stored counts
* that have drifted still converge on the truth instead of dropping a live row.
*/
export function applyLexicalRemovals(db: DatabaseSync, plan: LexicalRemovalPlan): void {
applyRemovalsForEntity(db, 'word', plan.words);
applyRemovalsForEntity(db, 'kanji', plan.kanji);
}
function applyRemovalsForEntity(
db: DatabaseSync,
entity: LexicalEntity,
removals: LexicalRemoval[],
): void {
if (removals.length === 0) return;
const entityTable = entity === 'word' ? 'imm_words' : 'imm_kanji';
const occurrenceTable =
entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
const col = `${entity}_id`;
const selectStmt = db.prepare(
`SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen
FROM ${entityTable}
WHERE id = ?`,
);
const updateFrequencyStmt = db.prepare(`UPDATE ${entityTable} SET frequency = ? WHERE id = ?`);
const hasOccurrencesStmt = db.prepare(
`SELECT 1 AS found FROM ${occurrenceTable} WHERE ${col} = ? LIMIT 1`,
);
const deleteStmt = db.prepare(`DELETE FROM ${entityTable} WHERE id = ?`);
const needsExactRefresh: number[] = [];
for (const removal of removals) {
const current = selectStmt.get(removal.id) as {
frequency: number | null;
firstSeen: number | null;
lastSeen: number | null;
} | null;
if (!current) continue;
const nextFrequency = (current.frequency ?? 0) - removal.removedFrequency;
if (nextFrequency <= 0) {
// The rows in scope are already gone by now, so anything still pointing at
// this entity means the stored frequency was stale rather than exhausted.
if (hasOccurrencesStmt.get(removal.id)) {
needsExactRefresh.push(removal.id);
} else {
deleteStmt.run(removal.id);
}
continue;
}
const removedFirstSeen = toStoredSeenSeconds(removal.removedFirstSeenMs);
const removedLastSeen = toStoredSeenSeconds(removal.removedLastSeenMs);
const firstSeenMayHaveMoved =
current.firstSeen === null ||
(removedFirstSeen !== null && removedFirstSeen <= current.firstSeen);
const lastSeenMayHaveMoved =
current.lastSeen === null ||
(removedLastSeen !== null && removedLastSeen >= current.lastSeen);
if (firstSeenMayHaveMoved || lastSeenMayHaveMoved) {
needsExactRefresh.push(removal.id);
continue;
}
updateFrequencyStmt.run(nextFrequency, removal.id);
}
if (entity === 'word') {
refreshWordAggregates(db, needsExactRefresh);
} else {
refreshKanjiAggregates(db, needsExactRefresh);
}
}
function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
if (wordIds.length === 0) {
return;
@@ -214,11 +375,18 @@ function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
SELECT
w.id AS wordId,
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
MIN(COALESCE(o.seen_ms, (
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
FROM imm_subtitle_lines sl
WHERE sl.line_id = o.line_id
))) AS firstSeen,
MAX(COALESCE(o.seen_ms, (
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
FROM imm_subtitle_lines sl
WHERE sl.line_id = o.line_id
))) AS lastSeen
FROM imm_words w
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
WHERE w.id IN (${makePlaceholders(wordIds)})
GROUP BY w.id
`,
@@ -263,11 +431,18 @@ function refreshKanjiAggregates(db: DatabaseSync, kanjiIds: number[]): void {
SELECT
k.id AS kanjiId,
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
MIN(COALESCE(o.seen_ms, (
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
FROM imm_subtitle_lines sl
WHERE sl.line_id = o.line_id
))) AS firstSeen,
MAX(COALESCE(o.seen_ms, (
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
FROM imm_subtitle_lines sl
WHERE sl.line_id = o.line_id
))) AS lastSeen
FROM imm_kanji k
LEFT JOIN imm_kanji_line_occurrences o ON o.kanji_id = k.id
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
WHERE k.id IN (${makePlaceholders(kanjiIds)})
GROUP BY k.id
`,
+56 -12
View File
@@ -192,6 +192,33 @@ function addColumnIfMissing(
}
}
/**
* Copy each subtitle line's timestamp onto its word/kanji occurrence rows.
*
* Vocabulary aggregates used to be recomputed by joining every occurrence back
* to `imm_subtitle_lines` just to read two timestamps, which meant one random
* read into the widest table per occurrence — the reason deleting a session cost
* seconds on a large library. With the timestamp stored alongside the count, the
* covering index answers those aggregates on its own.
*/
function backfillLexicalOccurrenceSeenMs(db: DatabaseSync): void {
for (const table of ['imm_word_line_occurrences', 'imm_kanji_line_occurrences']) {
addColumnIfMissing(db, table, 'seen_ms', 'INTEGER');
db.exec(`
UPDATE ${table}
SET seen_ms = (
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
FROM imm_subtitle_lines sl
WHERE sl.line_id = ${table}.line_id
)
WHERE seen_ms IS NULL
`);
}
// Superseded by the covering (entity, seen_ms, occurrence_count, line_id) indexes.
db.exec('DROP INDEX IF EXISTS idx_word_line_occurrences_word');
db.exec('DROP INDEX IF EXISTS idx_kanji_line_occurrences_kanji');
}
function dropColumnIfExists(db: DatabaseSync, tableName: string, columnName: string): void {
if (hasColumn(db, tableName, columnName)) {
db.exec(`ALTER TABLE ${tableName} DROP COLUMN ${columnName}`);
@@ -923,6 +950,7 @@ export function ensureSchema(db: DatabaseSync): void {
line_id INTEGER NOT NULL,
word_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, word_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
@@ -933,6 +961,7 @@ export function ensureSchema(db: DatabaseSync): void {
line_id INTEGER NOT NULL,
kanji_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, kanji_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
@@ -1093,6 +1122,7 @@ export function ensureSchema(db: DatabaseSync): void {
line_id INTEGER NOT NULL,
word_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, word_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
@@ -1103,6 +1133,7 @@ export function ensureSchema(db: DatabaseSync): void {
line_id INTEGER NOT NULL,
kanji_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
seen_ms INTEGER,
PRIMARY KEY(line_id, kanji_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
@@ -1349,13 +1380,19 @@ export function ensureSchema(db: DatabaseSync): void {
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_anime_line
ON imm_subtitle_lines(anime_id, line_index)
`);
if (currentVersion?.schema_version && currentVersion.schema_version < 19) {
backfillLexicalOccurrenceSeenMs(db);
}
// Covering indexes: vocabulary aggregates (frequency, first/last seen) are
// answered from the index alone, without reading the wide subtitle-line rows.
db.exec(`
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word
ON imm_word_line_occurrences(word_id, line_id)
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word_seen
ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji
ON imm_kanji_line_occurrences(kanji_id, line_id)
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji_seen
ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_media_art_cover_blob_hash
@@ -1475,21 +1512,23 @@ export function createTrackerPreparedStatements(db: DatabaseSync): TrackerPrepar
`),
wordLineOccurrenceUpsertStmt: db.prepare(`
INSERT INTO imm_word_line_occurrences (
line_id, word_id, occurrence_count
line_id, word_id, occurrence_count, seen_ms
) VALUES (
?, ?, ?
?, ?, ?, ?
)
ON CONFLICT(line_id, word_id) DO UPDATE SET
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)
`),
kanjiLineOccurrenceUpsertStmt: db.prepare(`
INSERT INTO imm_kanji_line_occurrences (
line_id, kanji_id, occurrence_count
line_id, kanji_id, occurrence_count, seen_ms
) VALUES (
?, ?, ?
?, ?, ?, ?
)
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count,
seen_ms = COALESCE(imm_kanji_line_occurrences.seen_ms, excluded.seen_ms)
`),
videoAnimeIdSelectStmt: db.prepare(`
SELECT anime_id FROM imm_videos
@@ -1630,11 +1669,16 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
const lineId = Number(lineResult.lastInsertRowid);
for (const occurrence of write.wordOccurrences) {
const wordId = incrementWordAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount);
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount, currentMs);
}
for (const occurrence of write.kanjiOccurrences) {
const kanjiId = incrementKanjiAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
stmts.kanjiLineOccurrenceUpsertStmt.run(lineId, kanjiId, occurrence.occurrenceCount);
stmts.kanjiLineOccurrenceUpsertStmt.run(
lineId,
kanjiId,
occurrence.occurrenceCount,
currentMs,
);
}
return;
}
+1 -1
View File
@@ -1,4 +1,4 @@
export const SCHEMA_VERSION = 18;
export const SCHEMA_VERSION = 19;
export const DEFAULT_QUEUE_CAP = 1_000;
export const DEFAULT_BATCH_SIZE = 25;
export const DEFAULT_FLUSH_INTERVAL_MS = 500;
+28 -14
View File
@@ -1,3 +1,4 @@
import { createHash } from 'node:crypto';
import type { Hono } from 'hono';
import type { ImmersionTrackerService } from './immersion-tracker-service.js';
import { statsJson, type StatsCoverImagesRequest } from '../../types/stats-http-contract.js';
@@ -74,6 +75,31 @@ function detectImageContentType(bytes: Uint8Array): string {
return 'application/octet-stream';
}
function ifNoneMatchHits(header: string | undefined, etag: string): boolean {
if (!header) return false;
if (header.trim() === '*') return true;
return header
.split(',')
.map((candidate) => candidate.trim().replace(/^W\//, ''))
.some((candidate) => candidate === etag);
}
// Cover URLs are stable per anime/video, so relinking an AniList entry swaps the
// bytes behind an unchanged URL. Long max-age would keep serving the old art from
// the browser cache; revalidate instead and let the ETag skip the transfer.
function coverResponse(bytes: Uint8Array<ArrayBuffer>, ifNoneMatch: string | undefined): Response {
const etag = `"${createHash('sha256').update(bytes).digest('hex')}"`;
const headers: Record<string, string> = {
'Content-Type': detectImageContentType(bytes),
'Cache-Control': 'no-cache',
ETag: etag,
};
if (ifNoneMatchHits(ifNoneMatch, etag)) {
return new Response(null, { status: 304, headers });
}
return new Response(bytes, { headers });
}
function createLimitedTaskRunner(maxConcurrentTasks: number): (task: () => Promise<void>) => void {
const queue: Array<() => Promise<void>> = [];
let activeTasks = 0;
@@ -137,13 +163,7 @@ export function registerStatsCoverRoutes(app: Hono, tracker: ImmersionTrackerSer
art = await tracker.getAnimeCoverArt(animeId);
}
if (!art?.coverBlob) return c.body(null, 404);
const bytes = new Uint8Array(art.coverBlob);
return new Response(bytes, {
headers: {
'Content-Type': detectImageContentType(bytes),
'Cache-Control': 'public, max-age=86400',
},
});
return coverResponse(new Uint8Array(art.coverBlob), c.req.header('if-none-match'));
});
app.get('/api/stats/media/:videoId/cover', async (c) => {
@@ -155,12 +175,6 @@ export function registerStatsCoverRoutes(app: Hono, tracker: ImmersionTrackerSer
art = await tracker.getCoverArt(videoId);
}
if (!art?.coverBlob) return c.body(null, 404);
const bytes = new Uint8Array(art.coverBlob);
return new Response(bytes, {
headers: {
'Content-Type': detectImageContentType(bytes),
'Cache-Control': 'public, max-age=604800',
},
});
return coverResponse(new Uint8Array(art.coverBlob), c.req.header('if-none-match'));
});
}
@@ -190,4 +190,11 @@ export function registerStatsLibraryRoutes(
await tracker.deleteVideo(videoId);
return c.json(statsJson('deleteVideo', { ok: true }));
});
app.delete('/api/stats/anime/:animeId', async (c) => {
const animeId = parseIntQuery(c.req.param('animeId'), 0);
if (animeId <= 0) return c.body(null, 400);
await tracker.deleteAnime(animeId);
return c.json(statsJson('deleteAnime', { ok: true }));
});
}
@@ -11,7 +11,7 @@ import {
resolveSecondarySubtitleTextFromSidecar,
} from '../secondary-subtitle-sidecar.js';
import {
applyStatsWordAndSentenceCardFields,
applyStatsWordCardFields,
createStatsMiningContext,
getStatsDirectMiningAudioFieldNames,
getStatsWordMiningAudioFieldName,
@@ -272,7 +272,7 @@ export function registerStatsMiningRoutes(app: Hono, options?: StatsMiningRouteO
const imageFieldName = ankiConfig.fields?.image ?? 'Picture';
mediaFields[sentenceFieldName] = highlightedSentence;
applyStatsWordAndSentenceCardFields(mediaFields, noteInfo, ankiConfig);
applyStatsWordCardFields(mediaFields, noteInfo, ankiConfig);
if (audioBuffer) {
const audioFilename = `subminer_audio_${timestamp}_${noteId}.mp3`;
@@ -1,5 +1,7 @@
import type { MediaGenerator } from '../../../media-generator.js';
import type { AnkiConnectConfig } from '../../../types.js';
import { applyCardKindFlagFields } from '../../../anki-integration/card-kinds.js';
import { resolveWordCardKind } from '../../../anki-integration/note-field-utils.js';
import { createLogger } from '../../../logger.js';
import type { RetimedSecondarySubtitleInput } from '../secondary-subtitle-sidecar.js';
@@ -94,20 +96,22 @@ export function shouldUseStatsLapisKikuCardFields(ankiConfig: AnkiConnectConfig)
return ankiConfig.isLapis?.enabled === true || ankiConfig.isKiku?.enabled === true;
}
export function applyStatsWordAndSentenceCardFields(
export function applyStatsWordCardFields(
fields: Record<string, string>,
noteInfo: StatsServerNoteInfo | null,
ankiConfig: AnkiConnectConfig,
): void {
if (!shouldUseStatsLapisKikuCardFields(ankiConfig) || !noteInfo) return;
const wordAndSentenceFlag = resolveStatsNoteFieldName(noteInfo, 'IsWordAndSentenceCard');
if (!wordAndSentenceFlag) return;
if (!noteInfo) return;
const cardKind = resolveWordCardKind(noteInfo, {
lapisEnabled: ankiConfig.isLapis?.enabled === true,
kikuEnabled: ankiConfig.isKiku?.enabled === true,
wordCardKind: ankiConfig.lapisKiku?.wordCardKind,
});
if (!cardKind) return;
fields[wordAndSentenceFlag] = 'x';
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
const resolved = resolveStatsNoteFieldName(noteInfo, flagName);
if (resolved && resolved !== wordAndSentenceFlag) fields[resolved] = '';
}
applyCardKindFlagFields(fields, cardKind, (preferredName) =>
resolveStatsNoteFieldName(noteInfo, preferredName),
);
}
export function getStatsDirectMiningAudioFieldNames(
@@ -0,0 +1,100 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Database } from '../immersion-tracker/sqlite';
import { ensureSchema } from '../immersion-tracker/storage';
import { mergeSnapshotIntoDb } from './merge';
const BASE_MS = 1_700_000_000_000;
const DAY_MS = 86_400_000;
/**
* Build a database holding one anime, one episode, one ended session and a
* single word occurrence.
*
* `legacyOccurrences` reproduces a peer whose schema predates the denormalised
* `seen_ms` column, so the merge has to recover the timestamp from the line.
*/
function buildDb(
dir: string,
name: string,
options: { word: string; seenMs: number; legacyOccurrences: boolean },
): string {
const dbPath = path.join(dir, name);
const db = new Database(dbPath);
ensureSchema(db);
db.exec(`
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 'key-${name}', 'Show ${name}', ${BASE_MS}, ${BASE_MS});
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 'video-${name}', 1, 'Episode 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 'uuid-${name}', 1, '${options.seenMs}', '${options.seenMs + 1000}', 2, ${BASE_MS}, ${BASE_MS});
INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
VALUES (1, 1, 1, 1, 0, 'line', ${options.seenMs}, ${options.seenMs});
INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
VALUES (1, '${options.word}', '${options.word}', '', 'noun', '名詞',
${Math.floor(options.seenMs / 1000)}, ${Math.floor(options.seenMs / 1000)}, 1);
`);
db.exec(
options.legacyOccurrences
? 'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (1, 1, 1)'
: `INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (1, 1, 1, ${options.seenMs})`,
);
db.close();
return dbPath;
}
function mergedOccurrences(dbPath: string): Array<{ word: string; seenMs: number | null }> {
const db = new Database(dbPath);
try {
return db
.prepare(
`SELECT w.word AS word, o.seen_ms AS seenMs
FROM imm_word_line_occurrences o
JOIN imm_words w ON w.id = o.word_id
ORDER BY w.word`,
)
.all() as Array<{ word: string; seenMs: number | null }>;
} finally {
db.close();
}
}
for (const legacyOccurrences of [false, true]) {
const label = legacyOccurrences ? 'a peer predating the seen_ms column' : 'a current peer';
test(`sync merge carries occurrence timestamps in from ${label}`, () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-merge-occurrences-test-'));
try {
const local = buildDb(dir, 'local.sqlite', {
word: '猫',
seenMs: BASE_MS,
legacyOccurrences: false,
});
const remoteSeenMs = BASE_MS + 5 * DAY_MS;
const remote = buildDb(dir, 'remote.sqlite', {
word: '犬',
seenMs: remoteSeenMs,
legacyOccurrences,
});
mergeSnapshotIntoDb(local, remote);
const rows = mergedOccurrences(local);
assert.equal(rows.length, 2, 'local and remote occurrences both survive the merge');
for (const row of rows) {
assert.ok(row.seenMs, `${row.word} must carry a timestamp so aggregates stay index-only`);
}
assert.equal(
Number(rows.find((row) => row.word === '犬')?.seenMs),
remoteSeenMs,
"the remote line's own timestamp is preserved rather than stamped with merge time",
);
} finally {
fs.rmSync(dir, { recursive: true, force: true });
}
});
}
+13 -6
View File
@@ -215,12 +215,16 @@ function copySubtitleLines(
'SELECT kanji_id, occurrence_count FROM imm_kanji_line_occurrences WHERE line_id = ?',
);
const insertWordOccurrence = local.query(
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count) VALUES (?, ?, ?)
ON CONFLICT(line_id, word_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
ON CONFLICT(line_id, word_id) DO UPDATE SET
occurrence_count = occurrence_count + excluded.occurrence_count,
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
);
const insertKanjiOccurrence = local.query(
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count) VALUES (?, ?, ?)
ON CONFLICT(line_id, kanji_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
occurrence_count = occurrence_count + excluded.occurrence_count,
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
);
for (const row of rows) {
@@ -241,17 +245,20 @@ function copySubtitleLines(
],
);
summary.subtitleLinesAdded += 1;
// Taken from the line rather than the remote occurrence row so this also
// works when the remote database predates the seen_ms column.
const seenMs = row.CREATED_DATE ?? row.LAST_UPDATE_DATE ?? null;
for (const occurrence of wordOccurrences.all(row.line_id) as SqlRow[]) {
const localWordId = lexicon.resolveWord(Number(occurrence.word_id));
const count = Number(occurrence.occurrence_count);
insertWordOccurrence.run(localLineId, localWordId, count);
insertWordOccurrence.run(localLineId, localWordId, count, seenMs);
lexicon.addWordOccurrences(Number(occurrence.word_id), count);
}
for (const occurrence of kanjiOccurrences.all(row.line_id) as SqlRow[]) {
const localKanjiId = lexicon.resolveKanji(Number(occurrence.kanji_id));
const count = Number(occurrence.occurrence_count);
insertKanjiOccurrence.run(localLineId, localKanjiId, count);
insertKanjiOccurrence.run(localLineId, localKanjiId, count, seenMs);
lexicon.addKanjiOccurrences(Number(occurrence.kanji_id), count);
}
}
+2 -2
View File
@@ -6,7 +6,7 @@ import * as os from 'node:os';
import * as path from 'node:path';
import type { YoutubeMediaCacheMode } from '../../../types/integrations';
import { getYoutubeYtDlpCommand } from './ytdlp-command';
import { getYoutubeYtDlpCommand, YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
type MediaCacheSessionState = 'running' | 'ready' | 'failed';
@@ -88,7 +88,7 @@ function normalizeMaxHeight(maxHeight: number | undefined): number {
function createYtDlpArgs(url: string, outputTemplate: string, maxHeight?: number): string[] {
return [
'--no-playlist',
YTDLP_SINGLE_VIDEO_ARG,
'--no-warnings',
'--force-ipv4',
'--retries',
+14 -6
View File
@@ -1,6 +1,6 @@
import { spawn } from 'node:child_process';
import type { YoutubeVideoMetadata } from '../immersion-tracker/types';
import { getYoutubeYtDlpCommand } from './ytdlp-command';
import { getYoutubeYtDlpCommand, YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
const YOUTUBE_METADATA_PROBE_TIMEOUT_MS = 15_000;
@@ -85,15 +85,23 @@ function pickChannelThumbnail(thumbnails: YtDlpThumbnail[] | undefined): string
return null;
}
export async function probeYoutubeVideoMetadata(
targetUrl: string,
): Promise<YoutubeVideoMetadata | null> {
const { stdout } = await runCapture(getYoutubeYtDlpCommand(), [
export function buildYoutubeMetadataProbeArgs(targetUrl: string): string[] {
return [
YTDLP_SINGLE_VIDEO_ARG,
'--dump-single-json',
'--no-warnings',
'--skip-download',
targetUrl,
]);
];
}
export async function probeYoutubeVideoMetadata(
targetUrl: string,
): Promise<YoutubeVideoMetadata | null> {
const { stdout } = await runCapture(
getYoutubeYtDlpCommand(),
buildYoutubeMetadataProbeArgs(targetUrl),
);
let info: YtDlpYoutubeMetadata;
try {
info = JSON.parse(stdout) as YtDlpYoutubeMetadata;
@@ -1,5 +1,5 @@
import { spawn } from 'node:child_process';
import { getYoutubeYtDlpCommand } from './ytdlp-command';
import { getYoutubeYtDlpCommand, YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
const YOUTUBE_PLAYBACK_RESOLVE_TIMEOUT_MS = 15_000;
const DEFAULT_PLAYBACK_FORMAT = 'b';
@@ -85,17 +85,18 @@ function runCapture(
});
}
export function buildYoutubePlaybackResolveArgs(targetUrl: string, format: string): string[] {
return [YTDLP_SINGLE_VIDEO_ARG, '--get-url', '--no-warnings', '-f', format, targetUrl];
}
export async function resolveYoutubePlaybackUrl(
targetUrl: string,
format = DEFAULT_PLAYBACK_FORMAT,
): Promise<string> {
const { stdout } = await runCapture(getYoutubeYtDlpCommand(), [
'--get-url',
'--no-warnings',
'-f',
format,
targetUrl,
]);
const { stdout } = await runCapture(
getYoutubeYtDlpCommand(),
buildYoutubePlaybackResolveArgs(targetUrl, format),
);
const playbackUrl =
stdout
.split(/\r?\n/)
+3 -3
View File
@@ -2,7 +2,7 @@ import fs from 'node:fs';
import path from 'node:path';
import { spawn } from 'node:child_process';
import type { YoutubeTrackOption } from './track-probe';
import { getYoutubeYtDlpCommand } from './ytdlp-command';
import { getYoutubeYtDlpCommand, YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
import {
convertYoutubeTimedTextToVtt,
isYoutubeTimedTextExtension,
@@ -126,14 +126,14 @@ function pickLatestSubtitleFileForLanguage(
return candidates[0] ?? null;
}
function buildDownloadArgs(input: {
export function buildDownloadArgs(input: {
targetUrl: string;
outputTemplate: string;
sourceLanguages: string[];
includeAutoSubs: boolean;
includeManualSubs: boolean;
}): string[] {
const args = ['--skip-download', '--no-warnings'];
const args = [YTDLP_SINGLE_VIDEO_ARG, '--skip-download', '--no-warnings'];
if (input.includeAutoSubs) {
args.push('--write-auto-subs');
}
+9 -6
View File
@@ -1,7 +1,7 @@
import { spawn } from 'node:child_process';
import type { YoutubeTrackOption } from '../../../types';
import { formatYoutubeTrackLabel, normalizeYoutubeLangCode, type YoutubeTrackKind } from './labels';
import { getYoutubeYtDlpCommand } from './ytdlp-command';
import { getYoutubeYtDlpCommand, YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
const YOUTUBE_TRACK_PROBE_TIMEOUT_MS = 15_000;
@@ -111,12 +111,15 @@ function toTracks(entries: Record<string, YtDlpSubtitleEntry> | undefined, kind:
export type { YoutubeTrackOption };
export function buildYoutubeTrackProbeArgs(targetUrl: string): string[] {
return [YTDLP_SINGLE_VIDEO_ARG, '--dump-single-json', '--no-warnings', targetUrl];
}
export async function probeYoutubeTracks(targetUrl: string): Promise<YoutubeTrackProbeResult> {
const { stdout } = await runCapture(getYoutubeYtDlpCommand(), [
'--dump-single-json',
'--no-warnings',
targetUrl,
]);
const { stdout } = await runCapture(
getYoutubeYtDlpCommand(),
buildYoutubeTrackProbeArgs(targetUrl),
);
const trimmedStdout = stdout.trim();
if (!trimmedStdout) {
throw new Error('yt-dlp returned empty output while probing subtitle tracks');
@@ -4,6 +4,13 @@ import path from 'node:path';
const DEFAULT_YTDLP_COMMAND = 'yt-dlp';
const WINDOWS_YTDLP_COMMANDS = ['yt-dlp.cmd', 'yt-dlp.exe', 'yt-dlp'];
/**
* yt-dlp expands `list=`/`index=` URL params into the whole playlist unless told not to, which
* makes single-video extraction hang (e.g. a full Watch Later list) until our timeouts fire.
* Every yt-dlp invocation targeting one video must include this.
*/
export const YTDLP_SINGLE_VIDEO_ARG = '--no-playlist';
function resolveFromPath(commandName: string): string | null {
if (!process.env.PATH) {
return null;
@@ -0,0 +1,38 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { buildYoutubeMetadataProbeArgs } from './metadata-probe';
import { buildYoutubePlaybackResolveArgs } from './playback-resolve';
import { buildDownloadArgs } from './track-download';
import { buildYoutubeTrackProbeArgs } from './track-probe';
import { YTDLP_SINGLE_VIDEO_ARG } from './ytdlp-command';
// Regression guard for issue #179: a `list=`/`index=` URL made yt-dlp enumerate the whole
// playlist (e.g. Watch Later) and blow past our 15s timeouts on every single-video call.
const PLAYLIST_URL = 'https://www.youtube.com/watch?v=LKfWC6CgFng&list=WL&index=3';
const cases: Array<{ name: string; args: string[] }> = [
{ name: 'track probe', args: buildYoutubeTrackProbeArgs(PLAYLIST_URL) },
{ name: 'metadata probe', args: buildYoutubeMetadataProbeArgs(PLAYLIST_URL) },
{ name: 'playback resolve', args: buildYoutubePlaybackResolveArgs(PLAYLIST_URL, 'b') },
{
name: 'subtitle download',
args: buildDownloadArgs({
targetUrl: PLAYLIST_URL,
outputTemplate: '/tmp/out.%(ext)s',
sourceLanguages: ['ja'],
includeAutoSubs: true,
includeManualSubs: false,
}),
},
];
test('YTDLP_SINGLE_VIDEO_ARG is the yt-dlp flag that disables playlist expansion', () => {
assert.equal(YTDLP_SINGLE_VIDEO_ARG, '--no-playlist');
});
for (const { name, args } of cases) {
test(`${name} passes --no-playlist for playlist-scoped URLs`, () => {
assert.ok(args.includes('--no-playlist'), `${name} args: ${args.join(' ')}`);
assert.equal(args.at(-1), PLAYLIST_URL);
});
}
@@ -73,6 +73,7 @@ test('createConfigHotReloadAppliedHandler applies safe Anki, annotation, and log
config.ankiConnect.fields.miscInfo = 'MiscInfoCustom';
config.ankiConnect.isLapis.sentenceCardModel = 'Sentence Card Custom';
config.ankiConnect.isKiku.fieldGrouping = 'manual';
config.ankiConnect.lapisKiku.wordCardKind = 'click';
config.logging.level = 'debug';
config.logging.rotation = 14;
config.logging.files.mpv = true;
@@ -114,6 +115,7 @@ test('createConfigHotReloadAppliedHandler applies safe Anki, annotation, and log
'ankiConnect.fields.miscInfo',
'ankiConnect.isLapis.sentenceCardModel',
'ankiConnect.isKiku.fieldGrouping',
'ankiConnect.lapisKiku.wordCardKind',
'logging.level',
'logging.rotation',
'logging.files.mpv',
@@ -138,6 +140,7 @@ test('createConfigHotReloadAppliedHandler applies safe Anki, annotation, and log
},
isLapis: { sentenceCardModel: 'Sentence Card Custom' },
isKiku: { fieldGrouping: 'manual' },
lapisKiku: { wordCardKind: 'click' },
},
]);
assert.ok(calls.includes('invalidate:tokens'));
@@ -134,6 +134,9 @@ function buildAnkiRuntimeConfigPatch(
if (diff.hotReloadFields.includes('ankiConnect.isKiku.fieldGrouping')) {
patch.isKiku = { fieldGrouping: config.ankiConnect.isKiku.fieldGrouping };
}
if (diff.hotReloadFields.includes('ankiConnect.lapisKiku.wordCardKind')) {
patch.lapisKiku = { wordCardKind: config.ankiConnect.lapisKiku.wordCardKind };
}
return Object.keys(patch).length > 0 ? patch : null;
}
@@ -2,6 +2,7 @@ import assert from 'node:assert/strict';
import test from 'node:test';
import {
getPlaybackFeedbackNotificationOptions,
getSubsyncStatusNotificationOptions,
getYoutubeFlowStatusNotificationOptions,
notifyConfiguredStatus,
} from './configured-status-notification';
@@ -158,6 +159,64 @@ test('notifyConfiguredStatus can suppress desktop delivery for progress ticks',
assert.deepEqual(calls, ['overlay:subsync-status:Subsync:Subsync: syncing |:progress:pin']);
});
test('subsync progress keeps the osd spinner frame but strips it from the overlay card', () => {
const calls: string[] = [];
for (const frame of ['|', '/', '-', '\\']) {
const message = `Subsync: syncing ${frame}`;
notifyConfiguredStatus(
message,
{
getNotificationType: () => 'both',
showOsd: (osdMessage) => {
calls.push(`osd:${osdMessage}`);
},
showOverlayNotification: (payload) =>
calls.push(
`overlay:${payload.body}:${payload.variant}:${payload.persistent ? 'pin' : 'auto'}`,
),
showDesktopNotification: (title, options) =>
calls.push(`desktop:${title}:${options.body ?? ''}`),
},
getSubsyncStatusNotificationOptions(message),
);
}
assert.deepEqual(calls, [
'overlay:Subsync: syncing:progress:pin',
'overlay:Subsync: syncing:progress:pin',
'overlay:Subsync: syncing:progress:pin',
'overlay:Subsync: syncing:progress:pin',
]);
calls.length = 0;
notifyConfiguredStatus(
'Subsync: syncing /',
{
getNotificationType: () => 'osd',
showOsd: (osdMessage) => {
calls.push(`osd:${osdMessage}`);
},
showOverlayNotification: (payload) => calls.push(`overlay:${payload.body}`),
showDesktopNotification: (title, options) =>
calls.push(`desktop:${title}:${options.body ?? ''}`),
},
getSubsyncStatusNotificationOptions('Subsync: syncing /'),
);
assert.deepEqual(calls, ['osd:Subsync: syncing /']);
});
test('subsync result notifications keep their message intact', () => {
assert.equal(
getSubsyncStatusNotificationOptions('Subtitle synchronized with ffsubsync').overlayBody,
'Subtitle synchronized with ffsubsync',
);
const failure = getSubsyncStatusNotificationOptions('ffsubsync synchronization failed: boom');
assert.equal(failure.variant, 'error');
assert.equal(failure.overlayBody, 'ffsubsync synchronization failed: boom');
});
test('notifyConfiguredStatus routes feedback through overlay without desktop delivery', () => {
const calls: string[] = [];
@@ -12,6 +12,8 @@ export interface ConfiguredStatusNotificationDeps {
export interface ConfiguredStatusNotificationOptions {
id?: string;
/** Overrides the overlay card body (the OSD/desktop paths keep the raw message). */
overlayBody?: string;
title?: string;
variant?: OverlayNotificationPayload['variant'];
persistent?: boolean;
@@ -31,6 +33,23 @@ export function getPlaybackFeedbackNotificationOptions(
return {};
}
export function getSubsyncStatusNotificationOptions(
message: string,
): ConfiguredStatusNotificationOptions {
const syncing = message.startsWith('Subsync: syncing');
const failed = message.toLowerCase().includes('failed');
return {
id: 'subsync-status',
title: 'Subsync',
// The overlay card renders its own animated spinner, so drop the ASCII
// spinner frame that the OSD path still needs.
overlayBody: syncing ? message.replace(/\s+[|/\-\\]$/, '') : message,
variant: failed ? 'error' : syncing ? 'progress' : 'info',
persistent: syncing,
desktop: !syncing,
};
}
export function getYoutubeFlowStatusNotificationOptions(
message: string,
): ConfiguredStatusNotificationOptions {
@@ -74,7 +93,7 @@ export function notifyConfiguredStatus(
deps.showOverlayNotification({
id: options.id,
title: options.title ?? 'SubMiner',
body: message,
body: options.overlayBody ?? message,
variant: options.variant ?? 'info',
persistent: options.persistent ?? false,
});
@@ -336,8 +336,8 @@ test('tick only writes interaction state on change', () => {
state.active = active;
},
});
tickLinuxOverlayPointerInteraction(deps); // off→on
tickLinuxOverlayPointerInteraction(deps); // no change
tickLinuxOverlayPointerInteraction(deps, 'linux'); // off→on
tickLinuxOverlayPointerInteraction(deps, 'linux'); // no change
assert.deepEqual(calls, [true]);
});
@@ -352,7 +352,7 @@ test('tick reapplies an unchanged inactive state when the window passthrough sta
},
});
tickLinuxOverlayPointerInteraction(deps);
tickLinuxOverlayPointerInteraction(deps, 'linux');
assert.deepEqual(calls, [false]);
});
@@ -363,7 +363,7 @@ test('tick does not flip state when suspended (returns null)', () => {
shouldSuspend: () => true,
setInteractionActive: (active) => calls.push(active),
});
tickLinuxOverlayPointerInteraction(deps);
tickLinuxOverlayPointerInteraction(deps, 'linux');
assert.deepEqual(calls, []);
});
@@ -379,10 +379,26 @@ test('tick clears active hover while a separate SubMiner window suppresses overl
});
state.active = true;
tickLinuxOverlayPointerInteraction(deps);
tickLinuxOverlayPointerInteraction(deps, 'linux');
assert.deepEqual(calls, [false]);
});
test('tick never clears interaction state on macOS/Windows, where renderer hover owns it', () => {
for (const platform of ['darwin', 'win32'] as const) {
const calls: boolean[] = [];
// Pointer is off the measured subtitle rect (e.g. sitting on a Yomitan popup) while the
// renderer has marked the overlay interactive.
const { deps } = makeDeps({
getCursorScreenPoint: () => ({ x: 200, y: 200 }),
getInteractionActive: () => true,
setInteractionActive: (active) => calls.push(active),
});
tickLinuxOverlayPointerInteraction(deps, platform);
assert.deepEqual(calls, [], `expected no interaction writes on ${platform}`);
}
});
test('tick skips cursor-driven mouse-ignore toggles when Linux input shape owns hit rects', () => {
const calls: boolean[] = [];
const { deps } = makeDeps({
@@ -391,7 +407,7 @@ test('tick skips cursor-driven mouse-ignore toggles when Linux input shape owns
setInteractionActive: (active) => calls.push(active),
});
tickLinuxOverlayPointerInteraction(deps);
tickLinuxOverlayPointerInteraction(deps, 'linux');
assert.deepEqual(calls, []);
});
@@ -270,7 +270,16 @@ export function resolveDesiredOverlayInteractive(
);
}
export function tickLinuxOverlayPointerInteraction(deps: LinuxOverlayPointerInteractionDeps): void {
export function tickLinuxOverlayPointerInteraction(
deps: LinuxOverlayPointerInteractionDeps,
platform: NodeJS.Platform = process.platform,
): void {
// Linux-only. Windows/macOS drive interaction state from renderer hover (setIgnoreMouseEvents),
// which knows about Yomitan popups and modals that sit off the measured subtitle rects. This
// cursor poll only hit-tests those rects, so running it elsewhere would clear interaction state
// (and re-enable window passthrough) whenever a measurement lands while the pointer is on a
// popup, swallowing popup clicks and scroll.
if (platform !== 'linux') return;
if (deps.shouldUseInputShape?.()) return;
const desired = resolveDesiredOverlayInteractive(deps);
if (desired === null) return;
@@ -22,6 +22,7 @@ import { withConfiguredOverlayNotificationPosition } from './overlay-notificatio
import { createOverlayNotificationDelivery } from './overlay-notification-delivery';
import {
getPlaybackFeedbackNotificationOptions,
getSubsyncStatusNotificationOptions,
getYoutubeFlowStatusNotificationOptions,
notifyConfiguredStatus,
type ConfiguredStatusNotificationOptions,
@@ -195,15 +196,7 @@ export function createOverlayNotificationsRuntime(deps: OverlayNotificationsRunt
}
function showSubsyncStatusNotification(message: string): void {
const syncing = message.startsWith('Subsync: syncing');
const failed = message.toLowerCase().includes('failed');
showConfiguredStatusNotification(message, {
id: 'subsync-status',
title: 'Subsync',
variant: failed ? 'error' : syncing ? 'progress' : 'info',
persistent: syncing,
desktop: !syncing,
});
showConfiguredStatusNotification(message, getSubsyncStatusNotificationOptions(message));
}
function showYoutubeFlowStatusNotification(message: string): void {
@@ -827,6 +827,9 @@ export function createVisibleOverlayInteractionRuntime(deps: VisibleOverlayInter
};
function tickLinuxOverlayPointerInteractionNow(): void {
if (process.platform !== 'linux') {
return;
}
if (applyLinuxOverlayInputShapeFromLatestMeasurement()) {
return;
}
+12
View File
@@ -2,6 +2,15 @@ import type { AiFeatureConfig } from './integrations';
import type { NotificationType } from './notification';
import type { NPlusOneMatchMode } from './subtitle';
/**
* Card that a Kiku/Lapis note generates. The note types mark this with mutually
* exclusive `Is...Card` flag fields, so only one kind may be flagged per note.
*/
export type CardKind = 'sentence' | 'audio' | 'word-and-sentence' | 'click';
/** Card kind SubMiner flags on word cards; 'none' leaves the flag fields untouched. */
export type WordCardKind = CardKind | 'none';
export interface NotificationOptions {
body?: string;
icon?: string;
@@ -115,4 +124,7 @@ export interface AnkiConnectConfig {
fieldGrouping?: 'auto' | 'manual' | 'disabled';
deleteDuplicateInAuto?: boolean;
};
lapisKiku?: {
wordCardKind?: WordCardKind;
};
}
+4 -1
View File
@@ -1,4 +1,4 @@
import type { AnkiConnectConfig } from './anki';
import type { AnkiConnectConfig, WordCardKind } from './anki';
import type {
AiConfig,
AiFeatureConfig,
@@ -285,6 +285,9 @@ export interface ResolvedConfig {
fieldGrouping: 'auto' | 'manual' | 'disabled';
deleteDuplicateInAuto: boolean;
};
lapisKiku: {
wordCardKind: WordCardKind;
};
};
shortcuts: Required<ShortcutsConfig>;
secondarySub: Required<SecondarySubConfig>;
+2
View File
@@ -141,6 +141,7 @@ export interface StatsJsonResponseMap {
deleteSessions: StatsOkResponse;
deleteSession: StatsOkResponse;
deleteVideo: StatsOkResponse;
deleteAnime: StatsOkResponse;
anilistSearch: StatsAnilistSearchResult[];
knownWords: string[];
knownWordsSummary: StatsKnownWordsSummary;
@@ -219,6 +220,7 @@ export interface StatsHttpClient {
deleteSession: (sessionId: number) => Promise<void>;
deleteSessions: (sessionIds: number[]) => Promise<void>;
deleteVideo: (videoId: number) => Promise<void>;
deleteAnime: (animeId: number) => Promise<void>;
getKnownWords: () => Promise<string[]>;
getKnownWordsSummary: () => Promise<StatsKnownWordsSummary>;
getAnimeKnownWordsSummary: (animeId: number) => Promise<StatsKnownWordsSummary>;
+4 -1
View File
@@ -1,8 +1,10 @@
import { Suspense, lazy, useCallback, useState } from 'react';
import { DeleteConfirmDialog } from './components/layout/DeleteConfirmDialog';
import { DeleteProgressToast } from './components/common/DeleteProgressToast';
import { TabBar } from './components/layout/TabBar';
import { OverviewTab } from './components/overview/OverviewTab';
import { useExcludedWords } from './hooks/useExcludedWords';
import { assetUrl } from './lib/asset-url';
import type { TabId } from './components/layout/TabBar';
import {
closeMediaDetail,
@@ -140,7 +142,7 @@ export function App() {
onClick={() => handleTabChange('overview')}
className="flex items-center gap-2 mb-2 hover:opacity-80 transition-opacity"
>
<img src="/favicon.png" alt="" className="h-6 object-contain" />
<img src={assetUrl('favicon.png')} alt="" className="h-6 object-contain" />
<h1 className="text-lg font-semibold text-ctp-text">SubMiner Stats</h1>
</button>
<TabBar activeTab={activeTab} onTabChange={handleTabChange} />
@@ -293,6 +295,7 @@ export function App() {
</Suspense>
) : null}
<DeleteConfirmDialog />
<DeleteProgressToast />
</div>
);
}
+53 -1
View File
@@ -1,6 +1,7 @@
import { useState, useEffect } from 'react';
import { useState, useEffect, useRef } from 'react';
import { useAnimeDetail } from '../../hooks/useAnimeDetail';
import { getStatsClient } from '../../hooks/useStatsApi';
import { confirmAnimeDelete } from '../../lib/delete-confirm';
import { epochDayToDate } from '../../lib/formatters';
import { AnimeHeader } from './AnimeHeader';
import { EpisodeList } from './EpisodeList';
@@ -16,6 +17,14 @@ interface AnimeDetailViewProps {
onBack: () => void;
onNavigateToWord?: (wordId: number) => void;
onOpenEpisodeDetail?: (videoId: number) => void;
/** Called after the whole library entry is deleted, so the caller can refresh. */
onAnimeDeleted?: () => void;
/**
* Called after the AniList link changes. The library list caches the old
* anilistId (and with it the cover URL), so it has to refetch or the grid
* keeps showing the previous title's art.
*/
onAnilistRelinked?: () => void;
}
type Range = 14 | 30 | 90;
@@ -139,10 +148,15 @@ export function AnimeDetailView({
onBack,
onNavigateToWord,
onOpenEpisodeDetail,
onAnimeDeleted,
onAnilistRelinked,
}: AnimeDetailViewProps) {
const { data, loading, error, reload } = useAnimeDetail(animeId);
const [showAnilistSelector, setShowAnilistSelector] = useState(false);
const [coverRetryToken, setCoverRetryToken] = useState(0);
const [isDeletingAnime, setIsDeletingAnime] = useState(false);
const [deleteError, setDeleteError] = useState<string | null>(null);
const isDeletingAnimeRef = useRef(false);
const knownWordsSummary = useAnimeKnownWords(animeId);
useEffect(() => {
@@ -154,6 +168,40 @@ export function AnimeDetailView({
if (!data?.detail) return <div className="text-ctp-overlay2 p-4">Anime not found</div>;
const { detail, episodes, anilistEntries } = data;
const handleDeleteAnime = async () => {
if (isDeletingAnimeRef.current) return;
isDeletingAnimeRef.current = true;
// Cleared up front so cancelling a retry doesn't leave the previous
// attempt's failure on screen.
setDeleteError(null);
let confirmed = false;
try {
confirmed = await confirmAnimeDelete(detail.canonicalTitle, detail.episodeCount);
} catch (err) {
setDeleteError(err instanceof Error ? err.message : 'Failed to confirm delete.');
isDeletingAnimeRef.current = false;
return;
}
if (!confirmed) {
isDeletingAnimeRef.current = false;
return;
}
setDeleteError(null);
setIsDeletingAnime(true);
try {
await getStatsClient().deleteAnime(animeId);
onAnimeDeleted?.();
onBack();
} catch (err) {
setDeleteError(err instanceof Error ? err.message : 'Failed to delete this title.');
setIsDeletingAnime(false);
} finally {
isDeletingAnimeRef.current = false;
}
};
return (
<div className="space-y-4">
<button
@@ -168,7 +216,10 @@ export function AnimeDetailView({
anilistEntries={anilistEntries ?? []}
coverRetryToken={coverRetryToken}
onChangeAnilist={() => setShowAnilistSelector(true)}
onDeleteAnime={() => void handleDeleteAnime()}
isDeletingAnime={isDeletingAnime}
/>
{deleteError ? <div className="text-sm text-ctp-red">{deleteError}</div> : null}
<AnimeOverviewStats detail={detail} knownWordsSummary={knownWordsSummary} />
<EpisodeList
episodes={episodes}
@@ -185,6 +236,7 @@ export function AnimeDetailView({
setShowAnilistSelector(false);
setCoverRetryToken((value) => value + 1);
reload();
onAnilistRelinked?.();
}}
/>
)}
@@ -0,0 +1,71 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { renderToStaticMarkup } from 'react-dom/server';
import { AnimeHeader } from './AnimeHeader';
import { confirmAnimeDelete, setDeleteConfirmPresenter } from '../../lib/delete-confirm';
import type { AnimeDetailData } from '../../types/stats';
const DETAIL: AnimeDetailData['detail'] = {
animeId: 3,
canonicalTitle: 'Project Radio Noise Season 2',
anilistId: 20661,
titleRomaji: 'Toaru Kagaku no Railgun S',
titleEnglish: 'A Certain Scientific Railgun S',
titleNative: null,
description: null,
totalSessions: 1,
totalActiveMs: 960_000,
totalCards: 0,
totalTokensSeen: 1_655,
totalLinesSeen: 300,
totalLookupCount: 0,
totalLookupHits: 0,
totalYomitanLookupCount: 0,
episodeCount: 1,
lastWatchedMs: 1_700_000_000_000,
};
test('AnimeHeader offers a delete-entry action alongside the AniList actions', () => {
const markup = renderToStaticMarkup(
<AnimeHeader detail={DETAIL} anilistEntries={[]} onDeleteAnime={() => {}} />,
);
assert.match(markup, /Delete Entry/);
assert.match(markup, /Delete this title and every session and stat recorded for it/);
});
test('AnimeHeader hides the delete action when no handler is wired', () => {
const markup = renderToStaticMarkup(<AnimeHeader detail={DETAIL} anilistEntries={[]} />);
assert.doesNotMatch(markup, /Delete Entry/);
});
test('AnimeHeader shows a pending state while the entry is being deleted', () => {
const markup = renderToStaticMarkup(
<AnimeHeader detail={DETAIL} anilistEntries={[]} onDeleteAnime={() => {}} isDeletingAnime />,
);
assert.match(markup, /disabled=""/);
assert.match(markup, /animate-spin/);
assert.doesNotMatch(markup, /Delete Entry/);
});
test('confirmAnimeDelete spells out how much data the entry deletion removes', async () => {
const seen: string[] = [];
const restore = setDeleteConfirmPresenter((message) => {
seen.push(message);
return false;
});
try {
await confirmAnimeDelete('Railgun Season 2', 1);
await confirmAnimeDelete('Railgun Season 2', 3);
} finally {
restore();
}
assert.match(seen[0] ?? '', /"Railgun Season 2"/);
assert.match(seen[0] ?? '', /1 episode\b/);
assert.match(seen[1] ?? '', /3 episodes/);
assert.match(seen[0] ?? '', /every session and stat/);
});
@@ -6,6 +6,8 @@ interface AnimeHeaderProps {
anilistEntries: AnilistEntry[];
coverRetryToken?: number;
onChangeAnilist?: () => void;
onDeleteAnime?: () => void;
isDeletingAnime?: boolean;
}
function AnilistButton({ entry }: { entry: AnilistEntry }) {
@@ -32,6 +34,8 @@ export function AnimeHeader({
anilistEntries,
coverRetryToken = 0,
onChangeAnilist,
onDeleteAnime,
isDeletingAnime = false,
}: AnimeHeaderProps) {
const altTitles = [detail.titleRomaji, detail.titleEnglish, detail.titleNative].filter(
(t): t is string => t != null && t !== detail.canonicalTitle,
@@ -95,6 +99,25 @@ export function AnimeHeader({
: 'Link to AniList'}
</button>
)}
{onDeleteAnime && (
<button
type="button"
onClick={onDeleteAnime}
disabled={isDeletingAnime}
title="Delete this title and every session and stat recorded for it"
className="inline-flex items-center gap-1 px-2 py-1 text-xs rounded bg-ctp-surface1 text-ctp-red/80 hover:bg-ctp-red/15 hover:text-ctp-red transition-colors disabled:opacity-50 disabled:cursor-not-allowed"
>
{isDeletingAnime ? (
<span
aria-hidden="true"
className="h-3 w-3 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
/>
) : (
<span aria-hidden="true">{'\u2715'}</span>
)}
{isDeletingAnime ? 'Deleting\u2026' : 'Delete Entry'}
</button>
)}
</div>
{detail.description && (
<p className="text-xs text-ctp-subtext0 mt-3 line-clamp-3 leading-relaxed">
@@ -0,0 +1,179 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { Window } from 'happy-dom';
import { act } from 'react';
import { createRoot } from 'react-dom/client';
import { apiClient } from '../../lib/api-client';
import type { AnimeDetailData, AnimeLibraryItem } from '../../types/stats';
import { AnimeTab } from './AnimeTab';
interface TestWindow extends Window {
IS_REACT_ACT_ENVIRONMENT?: boolean;
}
function installDom(): () => void {
const previousWindow = globalThis.window;
const previousDocument = globalThis.document;
const previousHTMLElement = globalThis.HTMLElement;
const previousIsReactActEnvironment = (
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
).IS_REACT_ACT_ENVIRONMENT;
const window = new Window() as TestWindow;
Object.defineProperty(globalThis, 'window', { value: window, configurable: true });
Object.defineProperty(globalThis, 'document', { value: window.document, configurable: true });
Object.defineProperty(globalThis, 'HTMLElement', {
value: window.HTMLElement,
configurable: true,
});
(
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
).IS_REACT_ACT_ENVIRONMENT = true;
return () => {
Object.defineProperty(globalThis, 'window', { value: previousWindow, configurable: true });
Object.defineProperty(globalThis, 'document', { value: previousDocument, configurable: true });
Object.defineProperty(globalThis, 'HTMLElement', {
value: previousHTMLElement,
configurable: true,
});
(
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
).IS_REACT_ACT_ENVIRONMENT = previousIsReactActEnvironment;
};
}
function libraryItem(anilistId: number | null): AnimeLibraryItem {
return {
animeId: 7,
canonicalTitle: 'Test Anime Season 2',
anilistId,
totalSessions: 1,
totalActiveMs: 1000,
totalCards: 0,
totalTokensSeen: 0,
episodeCount: 1,
episodesTotal: 13,
lastWatchedMs: 1,
};
}
function detailData(anilistId: number | null): AnimeDetailData {
return {
detail: {
animeId: 7,
canonicalTitle: 'Test Anime Season 2',
anilistId,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
description: null,
totalSessions: 1,
totalActiveMs: 1000,
totalCards: 0,
totalTokensSeen: 0,
totalLinesSeen: 0,
totalLookupCount: 0,
totalLookupHits: 0,
totalYomitanLookupCount: 0,
episodeCount: 1,
lastWatchedMs: 1,
},
episodes: [],
anilistEntries: [],
};
}
function findButton(container: Element, label: string): HTMLElement {
const match = [...container.querySelectorAll('button')].find((button) =>
(button.textContent ?? '').includes(label),
);
assert.ok(match, `expected a "${label}" button`);
return match as unknown as HTMLElement;
}
test('AnimeTab refetches the library after the AniList entry is relinked', async () => {
const uninstallDom = installDom();
const original = {
getAnimeLibrary: apiClient.getAnimeLibrary,
getAnimeDetail: apiClient.getAnimeDetail,
getAnimeWords: apiClient.getAnimeWords,
getAnimeRollups: apiClient.getAnimeRollups,
getAnimeKnownWordsSummary: apiClient.getAnimeKnownWordsSummary,
searchAnilist: apiClient.searchAnilist,
reassignAnimeAnilist: apiClient.reassignAnimeAnilist,
};
// The library grid keys its cover URL off anilistId, so a stale list keeps
// pointing at the previous entry's art.
let anilistId: number | null = 14813;
let libraryFetches = 0;
apiClient.getAnimeLibrary = (async () => {
libraryFetches += 1;
return [libraryItem(anilistId)];
}) as typeof apiClient.getAnimeLibrary;
apiClient.getAnimeDetail = (async () => detailData(anilistId)) as typeof apiClient.getAnimeDetail;
apiClient.getAnimeWords = (async () => []) as typeof apiClient.getAnimeWords;
apiClient.getAnimeRollups = (async () => []) as typeof apiClient.getAnimeRollups;
apiClient.getAnimeKnownWordsSummary = (async () => ({
totalUniqueWords: 0,
knownWordCount: 0,
})) as typeof apiClient.getAnimeKnownWordsSummary;
apiClient.searchAnilist = (async () => [
{
id: 108489,
episodes: 12,
season: null,
seasonYear: null,
description: null,
coverImage: null,
title: { romaji: 'Test Anime Kan', english: null, native: null },
},
]) as typeof apiClient.searchAnilist;
apiClient.reassignAnimeAnilist = (async (_animeId: number, info: { anilistId: number }) => {
anilistId = info.anilistId;
}) as typeof apiClient.reassignAnimeAnilist;
try {
const container = document.createElement('div');
document.body.append(container);
const root = createRoot(container);
await act(async () => {
root.render(<AnimeTab />);
});
assert.equal(libraryFetches, 1);
assert.match(
container.querySelector('img')?.getAttribute('src') ?? '',
/\/api\/stats\/anime\/7\/cover\?coverRetry=14813/,
);
await act(async () => {
findButton(container, 'Test Anime Season 2').click();
});
await act(async () => {
findButton(container, 'Change AniList Entry').click();
});
await act(async () => {
findButton(container, 'Test Anime Kan').click();
});
await act(async () => {
findButton(container, 'Back to Library').click();
});
assert.equal(libraryFetches, 2);
assert.match(
container.querySelector('img')?.getAttribute('src') ?? '',
/\/api\/stats\/anime\/7\/cover\?coverRetry=108489/,
);
await act(async () => {
root.unmount();
});
} finally {
Object.assign(apiClient, original);
uninstallDom();
}
});
+3 -1
View File
@@ -53,7 +53,7 @@ export function AnimeTab({
onNavigateToWord,
onOpenEpisodeDetail,
}: AnimeTabProps) {
const { anime, loading, error } = useAnimeLibrary();
const { anime, loading, error, reload } = useAnimeLibrary();
const [search, setSearch] = useState('');
const [sortKey, setSortKey] = useState<SortKey>('lastWatched');
const [cardSize, setCardSize] = useState<LibraryCardSize>(() =>
@@ -98,6 +98,8 @@ export function AnimeTab({
? (videoId) => onOpenEpisodeDetail(selectedAnimeId, videoId)
: undefined
}
onAnimeDeleted={reload}
onAnilistRelinked={reload}
/>
);
}
@@ -1,32 +1,49 @@
interface DeleteProgressToastProps {
/** Number of sessions currently being deleted. The toast is hidden when 0. */
count: number;
}
import { useSyncExternalStore } from 'react';
import { getDeleteProgressSnapshot, subscribeDeleteProgress } from '../../lib/delete-progress';
/**
* Fixed-position toast shown while session deletions are in flight.
* Global "deletion in progress" indicator.
*
* The per-row delete buttons are only visible on hover, so once the confirm
* dialog closes the user has no signal that a (potentially slow) batch delete
* is still running. This stays on screen, independent of hover, until the work
* finishes.
* Mounted once at the app root, outside every tab panel, so it shows no matter
* which view started the delete — per-tab copies were hidden along with their
* `hidden` tab panel, and detail views had no indicator at all. State comes
* from the delete-progress store rather than props so it also survives the
* initiating component unmounting mid-request.
*
* Renders two signals: a sweeping bar pinned to the top edge (visible even when
* the eye is on the content being deleted) and a bottom-right toast naming the
* work.
*/
export function DeleteProgressToast({ count }: DeleteProgressToastProps) {
export function DeleteProgressToast() {
const { count, label } = useSyncExternalStore(
subscribeDeleteProgress,
getDeleteProgressSnapshot,
getDeleteProgressSnapshot,
);
if (count <= 0) return null;
const message = count > 1 ? `Deleting ${count} items` : (label ?? 'Deleting');
return (
<div
role="status"
aria-live="polite"
className="fixed bottom-4 right-4 z-50 flex items-center gap-3 rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-4 py-3 shadow-lg shadow-black/30"
>
<span
<>
<div
aria-hidden="true"
className="h-4 w-4 shrink-0 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
/>
<span className="text-sm text-ctp-text">
Deleting {count} session{count === 1 ? '' : 's'}&hellip;
</span>
</div>
className="fixed inset-x-0 top-0 z-[2147483646] h-0.5 overflow-hidden bg-ctp-surface1"
>
<div className="h-full w-1/3 animate-indeterminate rounded-full bg-gradient-to-r from-ctp-red via-ctp-peach to-ctp-red" />
</div>
<div
role="status"
aria-live="polite"
className="fixed bottom-4 right-4 z-[2147483646] flex items-center gap-3 rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-4 py-3 shadow-lg shadow-black/30"
>
<span
aria-hidden="true"
className="h-4 w-4 shrink-0 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
/>
<span className="text-sm text-ctp-text">{message}&hellip;</span>
</div>
</>
);
}
@@ -6,7 +6,6 @@ import { StreakCalendar } from './StreakCalendar';
import { RecentSessions } from './RecentSessions';
import { TrackingSnapshot } from './TrackingSnapshot';
import { TrendChart } from '../trends/TrendChart';
import { DeleteProgressToast } from '../common/DeleteProgressToast';
import { buildOverviewSummary, buildStreakCalendar } from '../../lib/dashboard-data';
import { apiClient } from '../../lib/api-client';
import { getStatsClient } from '../../hooks/useStatsApi';
@@ -160,8 +159,6 @@ export function OverviewTab({
deletingIds={deletingIds}
isActive={isActive}
/>
<DeleteProgressToast count={deletingIds.size} />
</div>
);
}
@@ -2,7 +2,6 @@ import { useEffect, useMemo, useState } from 'react';
import { useSessions } from '../../hooks/useSessions';
import { SessionRow } from './SessionRow';
import { SessionDetail } from './SessionDetail';
import { DeleteProgressToast } from '../common/DeleteProgressToast';
import { apiClient } from '../../lib/api-client';
import { confirmBucketDelete, confirmSessionDelete } from '../../lib/delete-confirm';
import { formatDuration, formatNumber, formatSessionDayLabel } from '../../lib/formatters';
@@ -344,8 +343,6 @@ export function SessionsTab({
{search.trim() ? 'No sessions matching your search.' : 'No sessions recorded yet.'}
</div>
)}
<DeleteProgressToast count={deletingSessionIds.size} />
</div>
);
}
@@ -1,6 +1,7 @@
import { useRef, useState, useEffect } from 'react';
import { useWordDetail } from '../../hooks/useWordDetail';
import { apiClient } from '../../lib/api-client';
import { assetUrl } from '../../lib/asset-url';
import { epochMsFromDbTimestamp, formatNumber, formatRelativeDate } from '../../lib/formatters';
import {
buildStatsMineCardParams,
@@ -164,7 +165,10 @@ export function WordDetailPanel({
? data!.detail.headword
: occ.text.slice(0, 30);
if (typeof Notification !== 'undefined' && Notification.permission === 'granted') {
new Notification('Anki Card Created', { body: `Mined: ${label}`, icon: '/favicon.png' });
new Notification('Anki Card Created', {
body: `Mined: ${label}`,
icon: assetUrl('favicon.png'),
});
} else if (typeof Notification !== 'undefined' && Notification.permission !== 'denied') {
Notification.requestPermission().then((p) => {
if (p === 'granted') new Notification('Anki Card Created', { body: `Mined: ${label}` });
+8 -3
View File
@@ -1,4 +1,4 @@
import { useState, useEffect } from 'react';
import { useCallback, useState, useEffect } from 'react';
import { getStatsClient } from './useStatsApi';
import type { AnimeLibraryItem } from '../types/stats';
@@ -6,6 +6,11 @@ export function useAnimeLibrary() {
const [anime, setAnime] = useState<AnimeLibraryItem[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [reloadToken, setReloadToken] = useState(0);
const reload = useCallback(() => {
setReloadToken((token) => token + 1);
}, []);
useEffect(() => {
let cancelled = false;
@@ -23,7 +28,7 @@ export function useAnimeLibrary() {
return () => {
cancelled = true;
};
}, []);
}, [reloadToken]);
return { anime, loading, error };
return { anime, loading, error, reload };
}
+20 -7
View File
@@ -15,6 +15,7 @@ import type {
} from '../types/stats';
import type { StatsMineCardParams, StatsMineCardResponse } from './mining';
import { appendCoverRetryToken } from './cover-retry';
import { trackDelete } from './delete-progress';
type StatsLocationLike = Pick<Location, 'protocol' | 'origin' | 'search'>;
@@ -169,17 +170,29 @@ export const apiClient = {
});
},
deleteSession: async (sessionId: number): Promise<void> => {
await fetchResponse(`/api/stats/sessions/${sessionId}`, { method: 'DELETE' });
await trackDelete('Deleting session', () =>
fetchResponse(`/api/stats/sessions/${sessionId}`, { method: 'DELETE' }),
);
},
deleteSessions: async (sessionIds: number[]): Promise<void> => {
await fetchResponse('/api/stats/sessions', {
method: 'DELETE',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ sessionIds } satisfies StatsDeleteSessionsRequest),
});
const label = `Deleting ${sessionIds.length} session${sessionIds.length === 1 ? '' : 's'}`;
await trackDelete(label, () =>
fetchResponse('/api/stats/sessions', {
method: 'DELETE',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ sessionIds } satisfies StatsDeleteSessionsRequest),
}),
);
},
deleteVideo: async (videoId: number): Promise<void> => {
await fetchResponse(`/api/stats/media/${videoId}`, { method: 'DELETE' });
await trackDelete('Deleting episode', () =>
fetchResponse(`/api/stats/media/${videoId}`, { method: 'DELETE' }),
);
},
deleteAnime: async (animeId: number): Promise<void> => {
await trackDelete('Deleting library entry', () =>
fetchResponse(`/api/stats/anime/${animeId}`, { method: 'DELETE' }),
);
},
getKnownWords: () => fetchJson('knownWords', '/api/stats/known-words'),
getKnownWordsSummary: () => fetchJson('knownWordsSummary', '/api/stats/known-words-summary'),
+39
View File
@@ -0,0 +1,39 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { assetUrl, resolveAssetUrl } from './asset-url';
// vite.config.ts sets `base: './'`, so this is what the built bundle sees.
const BUILT_BASE = './';
test('built asset URLs are never root-absolute', () => {
assert.equal(resolveAssetUrl('favicon.png', BUILT_BASE).startsWith('/'), false);
});
test('built asset URL resolves next to a file:// index.html', () => {
const resolved = new URL(
resolveAssetUrl('favicon.png', BUILT_BASE),
'file:///opt/SubMiner/stats/dist/index.html',
);
assert.equal(resolved.href, 'file:///opt/SubMiner/stats/dist/favicon.png');
});
test('built asset URL resolves against the server root when served over http', () => {
const resolved = new URL(resolveAssetUrl('favicon.png', BUILT_BASE), 'http://127.0.0.1:8770/');
assert.equal(resolved.href, 'http://127.0.0.1:8770/favicon.png');
});
test('dev server base stays root-absolute', () => {
assert.equal(resolveAssetUrl('favicon.png', '/'), '/favicon.png');
});
test('a base without a trailing slash still joins cleanly', () => {
assert.equal(resolveAssetUrl('favicon.png', '/stats'), '/stats/favicon.png');
});
test('a leading slash in the requested path is tolerated', () => {
assert.equal(resolveAssetUrl('/favicon.png', BUILT_BASE), './favicon.png');
});
test('assetUrl falls back to a relative base outside a Vite bundle', () => {
assert.equal(assetUrl('favicon.png').startsWith('/'), false);
});
+24
View File
@@ -0,0 +1,24 @@
/**
* Resolve a bundled public asset against Vite's base URL.
*
* The in-player stats window is loaded with `loadFile`, so the document lives on
* `file://`. A root-absolute path like `/favicon.png` resolves to the filesystem
* root there and 404s, while the HTTP-served web app resolves it fine. Vite
* rewrites asset refs in `index.html` but not string literals in JSX, so build
* the URL from the configured base instead of hardcoding a leading slash.
*/
export function resolveAssetUrl(path: string, base: string): string {
const normalizedBase = base.endsWith('/') ? base : `${base}/`;
return `${normalizedBase}${path.replace(/^\/+/, '')}`;
}
function currentBase(): string {
// Vite injects BASE_URL at build time ('./' per vite.config.ts) and serves '/'
// in dev. Outside a Vite bundle (tests) there is no env, so fall back to './'.
const env = (import.meta as { env?: Record<string, string | undefined> }).env;
return env?.BASE_URL || './';
}
export function assetUrl(path: string): string {
return resolveAssetUrl(path, currentBase());
}
+7
View File
@@ -60,6 +60,13 @@ export function confirmAnimeGroupDelete(title: string, count: number): Promise<b
);
}
export function confirmAnimeDelete(title: string, episodeCount: number): Promise<boolean> {
const episodes = `${episodeCount} episode${episodeCount === 1 ? '' : 's'}`;
return confirmWithStatsNativeDialogLayer(
`Delete "${title}" from your library? This removes ${episodes} plus every session and stat recorded for them.`,
);
}
export function confirmEpisodeDelete(title: string): Promise<boolean> {
return confirmWithStatsNativeDialogLayer(`Delete "${title}" and all its sessions?`);
}
Loaded 100 of 104 files, more files were not shown because too many files have changed in this diff. Show more