mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-07-30 07:21:32 -07:00
Compare commits
12 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
48c36aa195
|
|||
|
105bd86410
|
|||
|
75566a49bf
|
|||
|
38060e0ead
|
|||
|
77badd7a54
|
|||
|
33b9d97d63
|
|||
|
9eb7f93a6b
|
|||
|
88d0fe6b98
|
|||
|
ffae99caa8
|
|||
|
d140ae6e56
|
|||
|
e0dbe5c8cd
|
|||
|
bc8dce870b
|
@@ -1,41 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## v0.19.0 (2026-07-29)
|
||||
|
||||
### Added
|
||||
- Anki Maturity Highlighting: Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature), similar to asbplayer. Tier thresholds and colors are configurable, with a runtime toggle and an updated help legend.
|
||||
- Post-Playback Menu: After a watch-history episode ends, the fzf/rofi launcher returns to that series with options to play the previous or next episode, rewatch, pick another episode, or quit. The pre-playback series menu now offers the previous episode too.
|
||||
- Delete Library Entries: The stats Library detail view can now delete an entire title in one step (episodes, sessions, subtitle lines, rollups, cover art, and vocabulary counts). Delete progress is now shown app-wide via a progress bar and status toast instead of disappearing when you switch tabs.
|
||||
- Cross-Machine Sync: Added SSH-based syncing of stats and watch history between machines, available from the tray ("Sync Stats & History") or `subminer sync`, with saved devices, per-host sync direction, background auto-sync, connection testing, manual snapshots, and support for Windows remotes.
|
||||
- TsukiHime Subtitle Downloads: Added subtitle downloads for the current video via TsukiHime, loading Japanese as the primary track and your configured secondary language directly into mpv.
|
||||
|
||||
### Changed
|
||||
- Clipboard-Video Shortcut: The "append clipboard video to queue" shortcut is now configurable.
|
||||
|
||||
### Fixed
|
||||
- AniList Season Resolution: Season 2+ files now resolve to the correct AniList entry instead of silently falling back to season 1 (which mismatched character dictionaries and watch progress). Manual overrides now stay scoped per season, fix both the dictionary and progress tracking together, and also correct per-season cover art.
|
||||
- Subtitle Annotation Accuracy: Fixed several annotation edge cases, including inconsistent POS exclusions on merged quote-particle tokens, dropped annotations on supplementary-plane kanji, katakana punctuation wrongly treated as noise, and certain kanji vocabulary losing N+1 highlighting eligibility.
|
||||
- AnkiConnect Proxy Port Conflict: Video startup no longer crashes when another process already holds the configured AnkiConnect proxy port; a notification now explains how to resolve it.
|
||||
- AppImage Quit Crash: Fixed a "Service Crash" desktop notification appearing after closing a video when running the Linux AppImage.
|
||||
- Autoplay Pause Timing: Fixed playback resuming a few seconds before subtitle tokenization warmup finished, most noticeable when resuming mid-episode or when a cue starts within the first two seconds.
|
||||
- Stats Known-Word Count: Fixed stats reporting 0 known words for every session after the known-word cache format changed.
|
||||
- Stats Library Cover Art: Relinking a title to a different AniList entry now updates its cover in the Library grid immediately instead of leaving a stale, mismatched cover cached.
|
||||
- Rofi Prompt Spacing: Rofi menu prompts now keep a space before the input field instead of running into the placeholder text.
|
||||
- Stats Settings & Reliability: Hardened stats settings validation (nested/legacy AnkiConnect config now falls back safely instead of breaking) and stats routes against malformed requests and other edge cases.
|
||||
- Stats Delete Performance: Deleting sessions, episodes, and library entries is now dramatically faster and no longer stalls playback (e.g. a 12-episode title dropped from about a minute to under a second on a large library); the Vocabulary tab also loads much faster. The first launch after upgrading runs a one-time database migration.
|
||||
|
||||
<details>
|
||||
<summary>Internal changes</summary>
|
||||
|
||||
### Internal
|
||||
- Added a golden-file regression test corpus for the tokenizer/annotation pipeline, plus scripts to record new fixtures and diff against stock Yomitan.
|
||||
- Consolidated renderer modal state handling into a descriptor registry.
|
||||
- Consolidated CI quality checks (PR, stable, and prerelease) into one reusable workflow with mpv plugin tests and dependency audits.
|
||||
- Removed the unused stats IPC transport and unified stats dashboard HTTP types with the backend contract.
|
||||
- Added a script to verify known-word highlight tiers against live Anki data outside of playback.
|
||||
|
||||
</details>
|
||||
|
||||
## v0.18.0 (2026-07-10)
|
||||
|
||||
### Added
|
||||
|
||||
@@ -66,7 +66,7 @@ Local stats dashboard tracking watch time, vocabulary growth, mining throughput,
|
||||
Browse sibling episode files and the active mpv queue in one overlay modal. Open it with `Ctrl+Alt+P` to append episodes from the current directory, jump to queued items, remove entries, or reorder the playlist without leaving playback.
|
||||
|
||||
<div align="center">
|
||||
<img src="docs-site/public/screenshots/playlist-browser.png" width="800" alt="Playlist browser modal showing sibling episode files beside the active mpv queue">
|
||||
<img src="docs-site/public/screenshots/playlist-browser.png" width="800" alt="Stats dashboard showing watch time, cards mined, streaks, and tracking data">
|
||||
</div>
|
||||
|
||||
<br>
|
||||
@@ -90,14 +90,6 @@ Browse sibling episode files and the active mpv queue in one overlay modal. Open
|
||||
<td><b>Jimaku</b></td>
|
||||
<td>Search and download Japanese subtitles</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>TsukiHime</b></td>
|
||||
<td>Search and download subtitles extracted from anime releases, with Japanese and secondary-language tabs (<code>Ctrl+Shift+T</code>) — no API key, requires <code>xz</code> on your <code>PATH</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>AniSkip</b></td>
|
||||
<td>Automatic intro detection with chapter markers and a one-key skip (<code>TAB</code> by default)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>alass / ffsubsync</b></td>
|
||||
<td>Manual subtitle retiming — requires <code>alass</code> or <code>ffsubsync</code> on your <code>PATH</code> (optional; subtitle syncing is disabled without them)</td>
|
||||
@@ -118,19 +110,18 @@ Browse sibling episode files and the active mpv queue in one overlay modal. Open
|
||||
|
||||
## Requirements
|
||||
|
||||
Only **mpv** is required to run SubMiner. Anki + AnkiConnect are required to mine cards, which is the point of the app, but everything else is optional.
|
||||
Only **mpv** and Anki+AnkiConnect are required. Everything else is optional but enhances the experience.
|
||||
|
||||
| Dependency | Status | What it does |
|
||||
| -------------------- | ---------------- | -------------------------------------------------------- |
|
||||
| mpv | Required | The video player SubMiner overlays on |
|
||||
| Anki + AnkiConnect | Required to mine | Card creation from the Yomitan popup |
|
||||
| ffmpeg | Recommended | Audio clips & screenshots for Anki cards |
|
||||
| MeCab + mecab-ipadic | Recommended | More precise annotations and filtering |
|
||||
| yt-dlp | Optional | YouTube playback |
|
||||
| xz | Optional | TsukiHime subtitle downloads (not on Windows by default) |
|
||||
| alass / ffsubsync | Optional | Subtitle sync |
|
||||
| guessit | Optional | Better anime title and episode detection |
|
||||
| fzf / rofi | Optional | Video picker in the `subminer` launcher (Linux/macOS) |
|
||||
| Dependency | Status | What it does |
|
||||
| -------------------- | ----------- | ---------------------------------------- |
|
||||
| mpv | Required | The video player SubMiner overlays on |
|
||||
| Anki + AnkiConnect | Required | Card creation from the Yomitan popup |
|
||||
| ffmpeg | Recommended | Audio clips & screenshots for Anki cards |
|
||||
| MeCab + mecab-ipadic | Recommended | More precise annotations and filtering |
|
||||
| yt-dlp | Optional | YouTube playback |
|
||||
| fzf / rofi | Optional | Video picker in the launcher |
|
||||
| alass / ffsubsync | Optional | Subtitle sync |
|
||||
| guessit | Optional | Better anime title and episode detection |
|
||||
|
||||
<details>
|
||||
<summary><b>Platform-specific install commands</b></summary>
|
||||
@@ -147,23 +138,9 @@ sudo pacman -S --needed mpv ffmpeg mecab mecab-ipadic
|
||||
brew install mpv ffmpeg mecab mecab-ipadic
|
||||
```
|
||||
|
||||
**Windows:**
|
||||
**Windows:** Install [mpv](https://mpv.io/installation/) and [ffmpeg](https://ffmpeg.org/download.html) and ensure both are on `PATH`.
|
||||
|
||||
```powershell
|
||||
winget install shinchiro.mpv
|
||||
winget install Gyan.FFmpeg
|
||||
```
|
||||
|
||||
Then reopen your terminal and check `mpv --version` and `ffmpeg -version`. winget puts `ffmpeg` on `PATH` automatically; mpv uses a regular installer that may not, so if `mpv` is not found, either add its folder (usually `%LOCALAPPDATA%\Programs\mpv`) to `PATH` or set `mpv.executablePath` during first-run setup.
|
||||
|
||||
[Scoop](https://scoop.sh) is the alternative if you want one package manager for everything, since it is the only one that also carries `xz`:
|
||||
|
||||
```powershell
|
||||
scoop bucket add extras
|
||||
scoop install extras/mpv main/ffmpeg main/yt-dlp main/xz
|
||||
```
|
||||
|
||||
See the [full requirements list](https://docs.subminer.moe/installation#_1-install-requirements) for optional dependencies.
|
||||
See the [full requirements list](https://docs.subminer.moe/installation#1-install-requirements) for optional dependencies.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -189,11 +166,6 @@ paru -S subminer-bin
|
||||
mkdir -p ~/.local/bin
|
||||
wget https://github.com/ksyasuda/SubMiner/releases/latest/download/SubMiner.AppImage -O ~/.local/bin/SubMiner.AppImage \
|
||||
&& chmod +x ~/.local/bin/SubMiner.AppImage
|
||||
```
|
||||
|
||||
The AppImage is all you need. The optional `subminer` command-line launcher runs on [Bun](https://bun.sh), and first-run setup can install both for you. To grab it manually instead, install Bun first, then:
|
||||
|
||||
```bash
|
||||
wget https://github.com/ksyasuda/SubMiner/releases/latest/download/subminer -O ~/.local/bin/subminer \
|
||||
&& chmod +x ~/.local/bin/subminer
|
||||
```
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
type: added
|
||||
area: overlay
|
||||
|
||||
- Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature) like asbplayer. Enable with `ankiConnect.knownWords.maturityEnabled`; the mature interval threshold (`matureThresholdDays`, default 21) and the four tier colors (`subtitleStyle.knownWordMaturityColors`) are configurable, and a runtime option toggles it in-session. The session help color legend shows the four tier colors while maturity highlighting is on.
|
||||
- Tiers follow Anki's own card counts: the interval tiers exclude cards in the learning/relearning queue, so a lapsed card shows the learning color instead of young (its interval is reset to at least 1 day, which previously made the learning tier unreachable). A note with a mature card alongside a relearning card still shows mature.
|
||||
@@ -0,0 +1,5 @@
|
||||
type: fixed
|
||||
area: overlay
|
||||
|
||||
- Applied configured primary POS exclusions consistently to merged trailing quote-particle tokens, preserved annotations for supplementary-plane kanji, and stopped treating katakana punctuation as kana-only annotation noise.
|
||||
- Kept kanji vocabulary tagged `名詞/非自立` eligible for N+1 highlighting, consistent with frequency, JLPT, and vocabulary persistence.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: changed
|
||||
area: shortcuts
|
||||
|
||||
- Made the clipboard-video playlist shortcut configurable through `shortcuts.appendClipboardVideoToQueue`.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: anki
|
||||
|
||||
- Prevented video startup from crashing when another process already owns the configured AnkiConnect proxy port, and added a notification explaining how to resolve the conflict.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: app
|
||||
|
||||
- Fixed "Service Crash" desktop notifications (KDE DrKonqi) after closing a video when running the Linux AppImage: the short-lived background bootstrap spawned a Chromium GPU child that outlived it (surviving `app.exit`) and died with SIGBUS at session end when the bootstrap's FUSE mount was finally released. The bootstrap now runs with the GPU in-process so it leaves no children behind, and the detached app's mount remains supervised until its Chromium children finish. Set `SUBMINER_NO_APPIMAGE_MOUNT_KEEPALIVE=1` to disable the detached-app mount supervisor.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: character dictionary
|
||||
|
||||
- Kept manual AniList overrides active across episodes in the same season directory when filename guesses differ.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: overlay
|
||||
|
||||
- Fixed `mpv.pauseUntilOverlayReady` releasing playback seconds before tokenization warmup finished: startup subtitle priming emits the current cue untokenized so the overlay can paint early, and that emission was treated as the autoplay-readiness signal as soon as the overlay window loaded. The autoplay gate now ignores untokenized subtitle payloads while tokenization warmup is pending, so playback resumes only after the first tokenized delivery (or the post-warmup release). Most visible when resuming mid-episode or when a subtitle cue starts within the first two seconds.
|
||||
@@ -0,0 +1,5 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Stats reported 0 known words for every session after the known-word cache gained maturity tiers. The stats server carried its own copy of the cache parser that only recognized versions up to 3, so the new v4 file was read as "no cache" rather than as a format it should understand.
|
||||
- The cache format, its parser, and the derived known-word set now live in one module that both the cache manager and the stats server read, and the version dispatch ends in an exhaustive check so a future format bump fails the build instead of silently reporting zero. A cache that exists but does not parse now logs a warning rather than passing for an empty one.
|
||||
@@ -0,0 +1,7 @@
|
||||
type: internal
|
||||
area: tokenizer
|
||||
|
||||
- Added a golden-file regression corpus for the tokenizer/annotation pipeline: recorded Yomitan backend responses and MeCab tokens replay through the real tokenizeSubtitle pipeline in bun tests without Electron or dictionaries.
|
||||
- Added `record-tokenizer-fixture:electron` script to capture new fixtures from a live Yomitan/MeCab session, with flags for known words, JLPT levels, and annotation toggles.
|
||||
- Seeded eleven fixtures covering the #147–#156 regression classes (grammar-helper suppression, lexical くれる, kanji non-independent nouns, N+1 targeting, reading collisions, unparsed runs, ordinal/honorific prefixes).
|
||||
- Added `compare-yomitan-api:electron` script that diffs SubMiner tokenization against a stock Yomitan instance via the yomitan-api bridge (segmentation, readings, headword forms).
|
||||
@@ -0,0 +1,5 @@
|
||||
type: added
|
||||
area: launcher
|
||||
|
||||
- After a watch-history episode ends or mpv closes, the fzf or rofi launcher returns to that series with options to play the previous episode, rewatch, play the next episode, select another episode, or quit SubMiner. Previous and Next continue across season directories.
|
||||
- The action menu shown right after picking a series from `subminer -H` now also offers the previous episode, matching the menu shown after playback.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: internal
|
||||
area: overlay
|
||||
|
||||
- Consolidated renderer modal state handling into a descriptor registry.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: internal
|
||||
area: release
|
||||
|
||||
- Consolidated pull request, stable release, and prerelease quality checks in one reusable workflow, with Lua mpv plugin tests and blocking high-severity dependency audits running in every gate.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: launcher
|
||||
|
||||
- Rofi menu prompts now keep a space between the prompt text and the input field instead of running into the search placeholder.
|
||||
@@ -0,0 +1,5 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Validated nested and legacy AnkiConnect settings after splitting the resolver, preserving valid modern overrides while warning and falling back for invalid primitive values.
|
||||
- Hardened stats routes against malformed IDs and static paths, stalled AniList searches, word-mining media collisions, missing Yomitan bridges, and throwing timing observers.
|
||||
@@ -0,0 +1,8 @@
|
||||
type: added
|
||||
area: sync
|
||||
|
||||
- Added cross-machine immersion sync for stats and watch history over SSH, available as a window (**Sync Stats & History** in the tray menu, or `subminer sync --ui`) and as a command (`subminer sync <host>`, with `--push` / `--pull` for one-way insert-only transfers). The window keeps saved devices with per-host direction, one-click sync with live stage-by-stage progress and separate merge summaries for each machine, connection testing for first-time setup, cancellable runs while the app/stats server/playback is active, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Hosts with auto-sync enabled sync in the background on a configurable interval, including during playback, with results reported as overlay notifications; hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and appear in the window automatically.
|
||||
- Merges are an insert-only union keyed on session UUIDs / video keys / series title keys, so re-syncing is idempotent and nothing is double-counted: each side snapshots its database (`VACUUM INTO`) from a consistent WAL point, snapshots are exchanged with `scp`, and each machine merges the other's data transactionally. Lifetime totals and daily/monthly rollups are updated incrementally (history older than the session retention window is preserved), unfinished sessions are excluded until a later sync sees them finalized, and remote-only historical rollups are copied only when they do not conflict with retained local session history. Sync aborts on stats schema version mismatches and refuses to run while the stats daemon or a live mpv session is active (`--force` overrides).
|
||||
- The sync engine runs only inside the app: the sync window and the `subminer sync` command both delegate to `SubMiner --sync-cli` (headless, works over SSH with no display), so neither machine needs bun or the command-line launcher. A remote machine only needs SubMiner itself, found automatically as the app binary or via the launcher proxy.
|
||||
- Windows remotes are supported: sync detects the remote shell (POSIX, cmd, or PowerShell) and manages remote temp files through SubMiner itself (`sync --make-temp` / `--remove-temp`) instead of `mktemp` / `rm`, so a Windows machine with the built-in OpenSSH Server works as a sync remote, found in its default Windows install location automatically.
|
||||
- Added supporting flags: `subminer sync <host> --check` tests the SSH connection and remote launcher availability without syncing, `subminer sync --snapshot <file>` and `--merge <file>` expose the underlying steps for manual transfers, and `subminer sync --json` emits machine-readable NDJSON progress (the protocol the sync window consumes).
|
||||
@@ -0,0 +1,4 @@
|
||||
type: added
|
||||
area: overlay
|
||||
|
||||
- Added TsukiHime subtitle downloads for the current video, with Japanese primary and configured secondary-language tracks loaded directly into mpv.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: internal
|
||||
area: stats
|
||||
|
||||
- Removed the unused stats IPC data transport and unified the stats dashboard's HTTP wire types with the backend contract.
|
||||
@@ -0,0 +1,7 @@
|
||||
type: internal
|
||||
area: tokenizer
|
||||
|
||||
- Added `verify-known-word-highlights:electron` script: tokenizes a real subtitle file through the app's Yomitan/MeCab pipeline with the live known-word cache, prints each line in the configured tier colors, and summarizes the tier counts so highlighting can be checked outside of playback.
|
||||
- Added `--audit`, which re-derives every highlighted tier from live Anki card data (`notesInfo` + `cardsInfo` intervals) and reports each token whose rendered tier disagrees, catching both stale cache entries and tier-classification bugs.
|
||||
- Added `--profile-copy` so the check can run while SubMiner is open (Electron locks the Yomitan userData dir), plus `--refresh`, `--limit`, `--json`, and `--quiet`.
|
||||
- Added `KnownWordCacheManager.getKnownWordMatchNoteIds`, exposing the note ids behind a known-word match so an audit can trace a rendered tier back to the exact Anki notes.
|
||||
@@ -1,6 +0,0 @@
|
||||
type: added
|
||||
area: anki
|
||||
|
||||
- Added `ankiConnect.lapisKiku.wordCardKind` (Settings > Mining/Anki > Kiku/Lapis Features > "Word Card Type") to choose which card-type flag SubMiner marks on Kiku/Lapis word cards: `word-and-sentence` (default, unchanged behavior), `click`, `sentence`, `audio`, or `none` to leave the flags alone. Requested by Kiku users who only want `IsClickCard` set.
|
||||
- Word cards can now be flagged as Kiku click cards (`IsClickCard`), which SubMiner previously never set.
|
||||
- Setting a card-type flag now clears `IsClickCard` alongside the other card-type flags, so a note can no longer claim two card types at once.
|
||||
@@ -523,7 +523,7 @@
|
||||
// ==========================================
|
||||
// AnkiConnect Integration
|
||||
// Automatic Anki updates and media generation options.
|
||||
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.
|
||||
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.
|
||||
// Shared AI provider transport settings are read from top-level ai and typically require restart.
|
||||
// Most other AnkiConnect settings still require restart.
|
||||
// ==========================================
|
||||
@@ -605,10 +605,7 @@
|
||||
"enabled": false, // Enable Kiku-specific mining behaviors (duplicate handling, field grouping). Values: true | false
|
||||
"fieldGrouping": "disabled", // Kiku duplicate-card field grouping mode. Values: auto | manual | disabled
|
||||
"deleteDuplicateInAuto": true // When Kiku field grouping is "auto", delete the duplicate source card after grouping completes. Values: true | false
|
||||
}, // Is kiku setting.
|
||||
"lapisKiku": {
|
||||
"wordCardKind": "word-and-sentence" // Card-type flag SubMiner marks on Kiku/Lapis word cards: word-and-sentence (IsWordAndSentenceCard), click (IsClickCard), sentence (IsSentenceCard), audio (IsAudioCard), or none to leave the flags untouched. Only applies when isKiku.enabled or isLapis.enabled; the other card-type flags are cleared so a note never claims two kinds. Cards mined with the dedicated sentence/audio actions keep their own flag. Values: word-and-sentence | click | sentence | audio | none
|
||||
} // Lapis kiku setting.
|
||||
} // Is kiku setting.
|
||||
}, // Automatic Anki updates and media generation options.
|
||||
|
||||
// ==========================================
|
||||
@@ -694,7 +691,7 @@
|
||||
"executablePath": "", // Optional absolute path to mpv.exe for Windows launch flows. Leave empty to auto-discover from SUBMINER_MPV_PATH or PATH.
|
||||
"launchMode": "normal", // Default window state for SubMiner-managed mpv launches. Values: normal | maximized | fullscreen
|
||||
"profile": "", // Optional mpv profile name passed to SubMiner-managed mpv launches. Leave empty to pass no profile.
|
||||
"socketPath": "/tmp/subminer-socket", // mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin.
|
||||
"socketPath": "\\\\.\\pipe\\subminer-socket", // mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin.
|
||||
"backend": "auto", // Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform. Values: auto | hyprland | sway | x11 | macos | windows
|
||||
"autoStartSubMiner": true, // Start SubMiner in the background when SubMiner-managed mpv loads a file. Values: true | false
|
||||
"pauseUntilOverlayReady": true, // Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness. Values: true | false
|
||||
|
||||
@@ -28,7 +28,7 @@ const outDir = optionalEnv(process.env.SUBMINER_DOCS_OUT_DIR);
|
||||
const docsSourceDir = optionalEnv(process.env.SUBMINER_DOCS_SOURCE_DIR) ?? process.cwd();
|
||||
const channel = normalizeChannel(optionalEnv(process.env.SUBMINER_DOCS_CHANNEL));
|
||||
const docsVersion = optionalEnv(process.env.SUBMINER_DOCS_VERSION);
|
||||
const latestStable = optionalEnv(process.env.SUBMINER_DOCS_LATEST_STABLE) ?? 'v0.18.0';
|
||||
const latestStable = optionalEnv(process.env.SUBMINER_DOCS_LATEST_STABLE) ?? 'v0.14.0';
|
||||
const versionManifest = parseVersionManifest(process.env.SUBMINER_DOCS_VERSION_MANIFEST);
|
||||
const versionLinkOrigin =
|
||||
optionalEnv(process.env.SUBMINER_DOCS_VERSION_LINK_ORIGIN) ?? 'production';
|
||||
@@ -306,7 +306,6 @@ const sidebar: DefaultTheme.SidebarItem[] = [
|
||||
{ text: 'Usage', link: '/usage' },
|
||||
{ text: 'Mining Workflow', link: '/mining-workflow' },
|
||||
{ text: 'Launcher Script', link: '/launcher-script' },
|
||||
{ text: 'Feature Demos', link: '/demos' },
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -39,7 +39,7 @@ SubMiner monitors playback and triggers an AniList progress update when an episo
|
||||
The update flow:
|
||||
|
||||
1. **Title detection** -- SubMiner extracts the anime title, season, and episode number from the media filename and path. Season folders such as `Season 2` are treated as a strong season signal. SubMiner tries [`guessit`](https://github.com/guessit-io/guessit) first for accurate parsing, then falls back to an internal filename parser if guessit is unavailable.
|
||||
2. **AniList search** -- The base title (with any `Season N` / `SN` marker stripped) is searched against the AniList GraphQL API, and SubMiner picks the best match by comparing titles (romaji, English, native, synonyms) and filtering by episode count. AniList has no notion of numbered seasons -- sequels are separate entries with their own titles (`Zoku`, `Kan`, `2nd Season`), so searching `<title> Season 3` finds nothing. For season 2 and later, SubMiner instead walks `SEQUEL` relations from the season 1 entry, preferring the TV line, and falls back to ordering the franchise's TV entries by air date when the relation chain is incomplete. If neither locates the season, SubMiner **skips the update** rather than writing progress to the season 1 entry, and tells you to pin the right entry with a [character dictionary override](/character-dictionary#correcting-anilist-matches).
|
||||
2. **AniList search** -- The detected title is searched against the AniList GraphQL API. For season 2 and later files, SubMiner searches the season-specific title first, then falls back to the base title. SubMiner picks the best match by comparing titles (romaji, English, native) and filtering by episode count.
|
||||
3. **Progress check** -- SubMiner fetches your current list entry for the matched media. The media must already be in Planning or Watching; otherwise SubMiner shows an MPV message explaining that the update is not possible. If your recorded progress already meets or exceeds the detected episode, the update is skipped.
|
||||
4. **Mutation** -- A `SaveMediaListEntry` mutation sets the new progress and marks the entry as `CURRENT`, or `COMPLETED` when the watched episode is the final episode of the season (the "already at this progress" skip is bypassed for the final episode so completion still lands).
|
||||
|
||||
@@ -69,8 +69,6 @@ SubMiner fetches cover art from AniList for display in the stats dashboard. When
|
||||
|
||||
A no-match result is cached for 5 minutes before SubMiner retries, preventing repeated API calls for unrecognized media.
|
||||
|
||||
If the automatic match is wrong, use **Change AniList Entry** on a title in the stats Library. Relinking rewrites the cached art for every episode of that title, and both the detail view and the Library grid pick up the new cover right away: the grid refetches after a relink, and cover responses carry an ETag and are revalidated on each request instead of being cached for a day.
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
All AniList API calls go through a shared rate limiter that enforces a sliding window of 20 requests per minute. The limiter also reads AniList's `X-RateLimit-Remaining` and `Retry-After` response headers and pauses requests when the server signals throttling. This applies to both episode tracking and cover art fetching.
|
||||
@@ -95,15 +93,15 @@ All AniList API calls go through a shared rate limiter that enforces a sliding w
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
|
||||
| `accessToken` | string | Explicit AniList access token override; when blank, SubMiner uses the stored encrypted token (default: `""`) |
|
||||
| `characterDictionary.maxLoaded` | number | Number of recent media snapshots kept in the merged dictionary (default: `3`) |
|
||||
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
|
||||
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
|
||||
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary to all Yomitan profiles or only the active one |
|
||||
| `characterDictionary.collapsibleSections.*` | `true`, `false` | Control which dictionary entry sections start expanded |
|
||||
| Option | Values | Description |
|
||||
| ------------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
|
||||
| `accessToken` | string | Explicit AniList access token override; when blank, SubMiner uses the stored encrypted token (default: `""`) |
|
||||
| `characterDictionary.maxLoaded` | number | Number of recent media snapshots kept in the merged dictionary (default: `3`) |
|
||||
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
|
||||
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
|
||||
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary to all Yomitan profiles or only the active one |
|
||||
| `characterDictionary.collapsibleSections.*` | `true`, `false` | Control which dictionary entry sections start expanded |
|
||||
|
||||
There is no `characterDictionary.enabled` key: character dictionary sync is enabled by `subtitleStyle.nameMatchEnabled`. See the [Character Dictionary](/character-dictionary) page for full details on the character dictionary feature, including name generation, matching, auto-sync lifecycle, and dictionary entry format.
|
||||
|
||||
|
||||
@@ -289,21 +289,6 @@ Trigger with the mine sentence shortcut (`Ctrl/Cmd+S` by default). The card is c
|
||||
|
||||
To mine multiple subtitle lines as one sentence card, use `Ctrl/Cmd+Shift+S` followed by a digit (1–9) to select how many recent lines to combine.
|
||||
|
||||
## Word Card Type (Kiku/Lapis)
|
||||
|
||||
Word cards get a card-type flag when SubMiner fills their sentence, whether that comes from Yomitan auto-enrichment, a manual clipboard update, or stats-dashboard word mining. By default the flag is `IsWordAndSentenceCard`; pick a different one with `ankiConnect.lapisKiku.wordCardKind`.
|
||||
|
||||
```jsonc
|
||||
"ankiConnect": {
|
||||
"isKiku": { "enabled": true },
|
||||
"lapisKiku": {
|
||||
"wordCardKind": "click" // word-and-sentence (default), click, sentence, audio, none
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`click` marks `IsClickCard`, `sentence` marks `IsSentenceCard`, `audio` marks `IsAudioCard`, and `none` leaves the flags untouched for templates that manage them elsewhere. Whichever flag is chosen, the other card-type flags are cleared so the note never claims two card types. The setting is only read when `isKiku` or `isLapis` is enabled, and cards mined with Mine Sentence or Mine Audio keep their own flag.
|
||||
|
||||
## Field Grouping (Kiku)
|
||||
|
||||
When you mine the same word multiple times, SubMiner can merge the cards instead of creating duplicates. This is designed for note types like [Kiku](https://github.com/youyoumu/kiku) that support grouped sentence/audio/image fields.
|
||||
@@ -328,11 +313,11 @@ When you mine the same word multiple times, SubMiner can merge the cards instead
|
||||
|
||||
### What Gets Merged
|
||||
|
||||
| Field | Merge behavior |
|
||||
| -------- | --------------------------------------------- |
|
||||
| Field | Merge behavior |
|
||||
| -------- | ---------------------------------------- |
|
||||
| Sentence | Both cards' sentences kept as grouped entries |
|
||||
| Audio | Both cards' `[sound:...]` entries kept |
|
||||
| Image | Both cards' images kept |
|
||||
| Audio | Both cards' `[sound:...]` entries kept |
|
||||
| Image | Both cards' images kept |
|
||||
|
||||
Identical values from both cards are kept as separate grouped entries; the merge does not deduplicate.
|
||||
|
||||
|
||||
+2
-43
@@ -1,47 +1,6 @@
|
||||
# Changelog
|
||||
|
||||
## v0.19.0 (2026-07-29)
|
||||
|
||||
**Added**
|
||||
- Anki Maturity Highlighting: Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature), similar to asbplayer. Tier thresholds and colors are configurable, with a runtime toggle and an updated help legend.
|
||||
- Post-Playback Menu: After a watch-history episode ends, the fzf/rofi launcher returns to that series with options to play the previous or next episode, rewatch, pick another episode, or quit. The pre-playback series menu now offers the previous episode too.
|
||||
- Delete Library Entries: The stats Library detail view can now delete an entire title in one step (episodes, sessions, subtitle lines, rollups, cover art, and vocabulary counts). Delete progress is now shown app-wide via a progress bar and status toast instead of disappearing when you switch tabs.
|
||||
- Cross-Machine Sync: Added SSH-based syncing of stats and watch history between machines, available from the tray ("Sync Stats & History") or `subminer sync`, with saved devices, per-host sync direction, background auto-sync, connection testing, manual snapshots, and support for Windows remotes.
|
||||
- TsukiHime Subtitle Downloads: Added subtitle downloads for the current video via TsukiHime, loading Japanese as the primary track and your configured secondary language directly into mpv.
|
||||
|
||||
**Changed**
|
||||
- Clipboard-Video Shortcut: The "append clipboard video to queue" shortcut is now configurable.
|
||||
|
||||
**Fixed**
|
||||
- AniList Season Resolution: Season 2+ files now resolve to the correct AniList entry instead of silently falling back to season 1 (which mismatched character dictionaries and watch progress). Manual overrides now stay scoped per season, fix both the dictionary and progress tracking together, and also correct per-season cover art.
|
||||
- Subtitle Annotation Accuracy: Fixed several annotation edge cases, including inconsistent POS exclusions on merged quote-particle tokens, dropped annotations on supplementary-plane kanji, katakana punctuation wrongly treated as noise, and certain kanji vocabulary losing N+1 highlighting eligibility.
|
||||
- AnkiConnect Proxy Port Conflict: Video startup no longer crashes when another process already holds the configured AnkiConnect proxy port; a notification now explains how to resolve it.
|
||||
- AppImage Quit Crash: Fixed a "Service Crash" desktop notification appearing after closing a video when running the Linux AppImage.
|
||||
- Autoplay Pause Timing: Fixed playback resuming a few seconds before subtitle tokenization warmup finished, most noticeable when resuming mid-episode or when a cue starts within the first two seconds.
|
||||
- Stats Known-Word Count: Fixed stats reporting 0 known words for every session after the known-word cache format changed.
|
||||
- Stats Library Cover Art: Relinking a title to a different AniList entry now updates its cover in the Library grid immediately instead of leaving a stale, mismatched cover cached.
|
||||
- Rofi Prompt Spacing: Rofi menu prompts now keep a space before the input field instead of running into the placeholder text.
|
||||
- Stats Settings & Reliability: Hardened stats settings validation (nested/legacy AnkiConnect config now falls back safely instead of breaking) and stats routes against malformed requests and other edge cases.
|
||||
- Stats Delete Performance: Deleting sessions, episodes, and library entries is now dramatically faster and no longer stalls playback (e.g. a 12-episode title dropped from about a minute to under a second on a large library); the Vocabulary tab also loads much faster. The first launch after upgrading runs a one-time database migration.
|
||||
|
||||
<details>
|
||||
<summary>Internal changes</summary>
|
||||
|
||||
**Internal**
|
||||
- Added a golden-file regression test corpus for the tokenizer/annotation pipeline, plus scripts to record new fixtures and diff against stock Yomitan.
|
||||
- Consolidated renderer modal state handling into a descriptor registry.
|
||||
- Consolidated CI quality checks (PR, stable, and prerelease) into one reusable workflow with mpv plugin tests and dependency audits.
|
||||
- Removed the unused stats IPC transport and unified stats dashboard HTTP types with the backend contract.
|
||||
- Added a script to verify known-word highlight tiers against live Anki data outside of playback.
|
||||
|
||||
</details>
|
||||
|
||||
## Previous Versions
|
||||
|
||||
<details>
|
||||
<summary>v0.18.x</summary>
|
||||
|
||||
<h2>v0.18.0 (2026-07-10)</h2>
|
||||
## v0.18.0 (2026-07-10)
|
||||
|
||||
**Added**
|
||||
- Sentence Audio Normalization: Generated sentence audio is now normalized to -23 LUFS by default, and clips mined from playback mirror mpv's software volume curve with a limiter to prevent clipping. Both behaviors are configurable independently.
|
||||
@@ -72,7 +31,7 @@
|
||||
|
||||
</details>
|
||||
|
||||
</details>
|
||||
## Previous Versions
|
||||
|
||||
<details>
|
||||
<summary>v0.17.x</summary>
|
||||
|
||||
@@ -223,9 +223,7 @@ SubMiner.AppImage --dictionary-select --dictionary-anilist-id 21355 --dictionary
|
||||
subminer app --session-action '{"actionId":"openCharacterDictionaryManager"}'
|
||||
```
|
||||
|
||||
SubMiner stores manual selections in `character-dictionaries/anilist-overrides.json`. The episode's parent directory **and detected season** define the override scope, so later episodes in the same season keep the selected AniList ID even if their filename guesses differ, while a different season never inherits the override -- including when every season sits in one flat folder. When you replace a wrong match, SubMiner removes that stale media ID from the merged dictionary's active set and rebuilds/imports the merged character dictionary.
|
||||
|
||||
An override also pins the entry used for [AniList watch progress](/anilist-integration), so correcting a wrong match once fixes both the character dictionary and progress tracking.
|
||||
SubMiner stores manual selections in `character-dictionaries/anilist-overrides.json`. The episode's parent directory defines the override scope, so later episodes in the same season directory keep the selected AniList ID even if their filename guesses differ. Separate season directories can keep separate overrides and character dictionaries. When you replace a wrong match, SubMiner removes that stale media ID from the merged dictionary's active set and rebuilds/imports the merged character dictionary.
|
||||
|
||||
## Managing Loaded Entries
|
||||
|
||||
|
||||
+54
-109
@@ -71,10 +71,6 @@ When both files exist, SubMiner prefers `config.jsonc` over `config.json`.
|
||||
|
||||
See [config.example.jsonc](/config.example.jsonc) for a comprehensive example with all available options, default values, and detailed comments. Only include the options you want to customize in your config file.
|
||||
|
||||
::: warning One value in that file is platform-specific
|
||||
The example is generated with a fixed Linux/macOS socket path so it stays reproducible, so it shows `"socketPath": "/tmp/subminer-socket"`. On Windows the real default is `\\\\.\\pipe\\subminer-socket`. Leave `mpv.socketPath` out of your config entirely unless you need a custom path, and SubMiner picks the right one for your platform.
|
||||
:::
|
||||
|
||||
Generate a fresh default config from the centralized config registry:
|
||||
|
||||
```bash
|
||||
@@ -149,13 +145,12 @@ The configuration file includes several main sections:
|
||||
- [**Shared AI Provider**](#shared-ai-provider) - Canonical OpenAI-compatible provider config shared by Anki and YouTube subtitle fixing
|
||||
- [**AnkiConnect**](#ankiconnect) - Automatic Anki card creation with media
|
||||
- [**Kiku/Lapis Integration**](#kiku-lapis-integration) - Sentence cards and duplicate handling for Kiku/Lapis note types
|
||||
- [**N+1 Word Highlighting**](#n-1-word-highlighting) - Known-word cache and single-target highlighting
|
||||
- [**N+1 Word Highlighting**](#n1-word-highlighting) - Known-word cache and single-target highlighting
|
||||
- [**Field Grouping Modes**](#field-grouping-modes) - Kiku/Lapis duplicate card merging
|
||||
|
||||
**External Integrations**
|
||||
|
||||
- [**Jimaku**](#jimaku) - Jimaku API configuration and defaults
|
||||
- [**TsukiHime**](#tsukihime) - Multi-language subtitle search and download
|
||||
- [**Subtitle Sync**](#subtitle-sync) - Sync current subtitle with `alass`/`ffsubsync`
|
||||
- [**AniList**](#anilist) - Optional post-watch progress updates
|
||||
- [**Yomitan**](#yomitan) - Reuse an external read-only Yomitan profile
|
||||
@@ -398,30 +393,30 @@ See `config.example.jsonc` for detailed configuration options.
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ---------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `primaryDefaultMode` | string | Default primary subtitle bar visibility mode: `"hidden"`, `"visible"`, or `"hover"` (default: `"visible"`) |
|
||||
| `subtitleStyle.css` | object | CSS declaration object applied to primary subtitles after normal style defaults. Use CSS property names such as `font-size`. |
|
||||
| `secondary.css` | object | CSS declaration object applied to secondary subtitles after normal secondary style defaults. |
|
||||
| `enableJlpt` | boolean | Enable JLPT level underline styling (`false` by default) |
|
||||
| `preserveLineBreaks` | boolean | Preserve line breaks in visible overlay subtitle rendering (`false` by default). Enable to mirror mpv line layout. |
|
||||
| `autoPauseVideoOnHover` | boolean | Pause playback while mouse hovers subtitle text, then resume on leave (`true` by default). |
|
||||
| `autoPauseVideoOnYomitanPopup` | boolean | Pause playback while the Yomitan popup is open, then resume when the popup closes (`true` by default). |
|
||||
| `primaryVisibleOnYomitanPopup` | boolean | Keep hover-mode primary subtitles visible while the Yomitan popup is open (`true` by default). |
|
||||
| `nameMatchEnabled` | boolean | Enable character dictionary sync and subtitle token coloring for character-name matches (`false` by default) |
|
||||
| `nameMatchImagesEnabled` | boolean | Show small cached AniList character portraits beside matched character-name tokens (`false` by default) |
|
||||
| `nameMatchColor` | string | Hex color used for subtitle tokens matched from the SubMiner character dictionary (default: `#f5bde6`) |
|
||||
| `knownWordColor` | string | Hex color used for known-word subtitle highlights (default: `#a6da95`) |
|
||||
| Option | Values | Description |
|
||||
| ---------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `primaryDefaultMode` | string | Default primary subtitle bar visibility mode: `"hidden"`, `"visible"`, or `"hover"` (default: `"visible"`) |
|
||||
| `subtitleStyle.css` | object | CSS declaration object applied to primary subtitles after normal style defaults. Use CSS property names such as `font-size`. |
|
||||
| `secondary.css` | object | CSS declaration object applied to secondary subtitles after normal secondary style defaults. |
|
||||
| `enableJlpt` | boolean | Enable JLPT level underline styling (`false` by default) |
|
||||
| `preserveLineBreaks` | boolean | Preserve line breaks in visible overlay subtitle rendering (`false` by default). Enable to mirror mpv line layout. |
|
||||
| `autoPauseVideoOnHover` | boolean | Pause playback while mouse hovers subtitle text, then resume on leave (`true` by default). |
|
||||
| `autoPauseVideoOnYomitanPopup` | boolean | Pause playback while the Yomitan popup is open, then resume when the popup closes (`true` by default). |
|
||||
| `primaryVisibleOnYomitanPopup` | boolean | Keep hover-mode primary subtitles visible while the Yomitan popup is open (`true` by default). |
|
||||
| `nameMatchEnabled` | boolean | Enable character dictionary sync and subtitle token coloring for character-name matches (`false` by default) |
|
||||
| `nameMatchImagesEnabled` | boolean | Show small cached AniList character portraits beside matched character-name tokens (`false` by default) |
|
||||
| `nameMatchColor` | string | Hex color used for subtitle tokens matched from the SubMiner character dictionary (default: `#f5bde6`) |
|
||||
| `knownWordColor` | string | Hex color used for known-word subtitle highlights (default: `#a6da95`) |
|
||||
| `knownWordMaturityColors` | object | Per-tier known-word colors used when `ankiConnect.knownWords.maturityEnabled` is on: `new` (`#ee99a0`), `learning` (`#b7bdf8`), `young` (`#91d7e3`), `mature` (`#a6da95`) |
|
||||
| `nPlusOneColor` | string | Hex color used for the single N+1 target subtitle highlight (default: `#c6a0f6`) |
|
||||
| `frequencyDictionary.enabled` | boolean | Enable frequency highlighting from dictionary lookups (`false` by default) |
|
||||
| `frequencyDictionary.sourcePath` | string | Path to a local frequency dictionary root. Leave empty or omit to use installed/default frequency-dictionary search paths. |
|
||||
| `frequencyDictionary.topX` | number | Only color tokens whose frequency rank is `<= topX` (`10000` by default) |
|
||||
| `frequencyDictionary.mode` | string | `"single"` or `"banded"` (`"single"` by default) |
|
||||
| `frequencyDictionary.matchMode` | string | `"headword"` or `"surface"` (`"headword"` by default) |
|
||||
| `frequencyDictionary.singleColor` | string | Color used for all highlighted tokens in single mode |
|
||||
| `frequencyDictionary.bandedColors` | string[] | Array of five hex colors used for ranked bands in banded mode |
|
||||
| `jlptColors` | object | JLPT level underline colors object (`N1`..`N5`) |
|
||||
| `nPlusOneColor` | string | Hex color used for the single N+1 target subtitle highlight (default: `#c6a0f6`) |
|
||||
| `frequencyDictionary.enabled` | boolean | Enable frequency highlighting from dictionary lookups (`false` by default) |
|
||||
| `frequencyDictionary.sourcePath` | string | Path to a local frequency dictionary root. Leave empty or omit to use installed/default frequency-dictionary search paths. |
|
||||
| `frequencyDictionary.topX` | number | Only color tokens whose frequency rank is `<= topX` (`10000` by default) |
|
||||
| `frequencyDictionary.mode` | string | `"single"` or `"banded"` (`"single"` by default) |
|
||||
| `frequencyDictionary.matchMode` | string | `"headword"` or `"surface"` (`"headword"` by default) |
|
||||
| `frequencyDictionary.singleColor` | string | Color used for all highlighted tokens in single mode |
|
||||
| `frequencyDictionary.bandedColors` | string[] | Array of five hex colors used for ranked bands in banded mode |
|
||||
| `jlptColors` | object | JLPT level underline colors object (`N1`..`N5`) |
|
||||
|
||||
Subtitle CSS custom properties:
|
||||
|
||||
@@ -543,8 +538,6 @@ Display a second subtitle track (e.g., English alongside Japanese) in the overla
|
||||
|
||||
See `config.example.jsonc` for detailed configuration options.
|
||||
|
||||
Secondary subtitles do **not** auto-load by default. To turn them on for local and Jellyfin playback, set `autoLoadSecondarySub` to `true` and list the language codes you want:
|
||||
|
||||
```json
|
||||
{
|
||||
"secondarySub": {
|
||||
@@ -555,15 +548,11 @@ Secondary subtitles do **not** auto-load by default. To turn them on for local a
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ----------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `secondarySubLanguages` | string[] | Language codes to auto-load (e.g., `["eng", "en"]`); non-Signs/Songs tracks are preferred when several tracks match. Default is empty (`[]`). |
|
||||
| `autoLoadSecondarySub` | `true`, `false` | Auto-detect and load a matching secondary subtitle track for local/Jellyfin sidecar files (default: `false`) |
|
||||
| `defaultMode` | `"hidden"`, `"visible"`, `"hover"` | Initial display mode (default: `"hover"`) |
|
||||
|
||||
These two settings apply to local and Jellyfin playback only. YouTube secondary selection is fixed to English and ignores them; see [YouTube Integration](/youtube-integration#secondary-subtitle-languages). `defaultMode` still controls how the loaded secondary bar is displayed in every case.
|
||||
|
||||
Because the mined-card translation field is filled from the secondary subtitle when one is present, leaving `autoLoadSecondarySub` off means local-file cards fall back to AI translation (when configured) or the original sentence text.
|
||||
| Option | Values | Description |
|
||||
| ----------------------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| `secondarySubLanguages` | string[] | Language codes to auto-load (e.g., `["eng", "en"]`); non-Signs/Songs tracks are preferred when several tracks match |
|
||||
| `autoLoadSecondarySub` | `true`, `false` | Auto-detect and load matching secondary subtitle track |
|
||||
| `defaultMode` | `"hidden"`, `"visible"`, `"hover"` | Initial display mode (default: `"hover"`) |
|
||||
|
||||
The secondary-subtitle language list also acts as the fallback secondary-language priority for managed startup subtitle selection on local playback and YouTube playback.
|
||||
|
||||
@@ -874,8 +863,8 @@ When config hot-reload updates shortcut/keybinding/style values, close and reope
|
||||
Use the runtime options palette to toggle settings live while SubMiner is running. These changes are session-only and reset on restart.
|
||||
|
||||
Current runtime options cover automatic card updates, known-word highlighting,
|
||||
known-word maturity coloring, N+1 annotation, JLPT underlines, frequency
|
||||
highlighting, known-word match mode, and Kiku field grouping mode.
|
||||
N+1 annotation, JLPT underlines, frequency highlighting, known-word match mode,
|
||||
and Kiku field grouping mode.
|
||||
|
||||
Annotation toggles only apply to new subtitle lines after the toggle. The currently displayed line is not re-tokenized in place.
|
||||
|
||||
@@ -1042,8 +1031,6 @@ This example is intentionally compact. The option table below documents availabl
|
||||
| `ankiConnect.knownWords.matchMode` | `"headword"`, `"surface"` | Matching strategy for known-word highlighting (default: `"headword"`). `headword` uses token headwords; `surface` uses visible subtitle text. |
|
||||
| `ankiConnect.knownWords.refreshMinutes` | number | Minutes between known-word cache refreshes (default: `1440`) |
|
||||
| `ankiConnect.knownWords.decks` | object | Deck→fields mapping used for known-word cache query scope (e.g. `{ "Kaishi 1.5k": ["Word"] }`). |
|
||||
| `ankiConnect.knownWords.maturityEnabled` | `true`, `false` | Color known words by Anki card maturity (new/learning/young/mature) instead of one color. Requires `knownWords.highlightEnabled` (default: `false`). Tier colors come from `subtitleStyle.knownWordMaturityColors`. |
|
||||
| `ankiConnect.knownWords.matureThresholdDays` | number | Card interval in days at which a known word counts as mature (default: `21`, matching Anki's own convention) |
|
||||
| `ankiConnect.nPlusOne.enabled` | `true`, `false` | Enable N+1 subtitle highlighting (highlights the one unknown word in a sentence). Independent from `knownWords.highlightEnabled`. Requires known-word cache data (default: `false`). |
|
||||
| `ankiConnect.nPlusOne.minSentenceWords` | number | Minimum number of words required in a sentence before single unknown-word N+1 highlighting can trigger (default: `3`). |
|
||||
| `behavior.notificationType` | `"overlay"`, `"system"`, `"both"`, `"none"` | Notification type on card update (default: `"overlay"`). `"both"` means overlay + system. `osd` and `osd-system` are legacy config-file-only values; use `"osd-system"` to keep the old OSD + system behavior. |
|
||||
@@ -1069,9 +1056,6 @@ SubMiner is intentionally built for [Kiku](https://kiku.youyoumu.my.id/) and [La
|
||||
"enabled": true,
|
||||
"fieldGrouping": "manual",
|
||||
"deleteDuplicateInAuto": true
|
||||
},
|
||||
"lapisKiku": {
|
||||
"wordCardKind": "word-and-sentence"
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -1080,21 +1064,6 @@ SubMiner is intentionally built for [Kiku](https://kiku.youyoumu.my.id/) and [La
|
||||
- Enable `isKiku` to turn on duplicate merge behavior for mined Word/Expression hits.
|
||||
- When both are enabled, Kiku behavior is applied for grouping while sentence-card model settings are still read from `isLapis`.
|
||||
- `isKiku.fieldGrouping` supports `disabled`, `auto`, and `manual` merge modes; see [Field Grouping Modes](#field-grouping-modes).
|
||||
- `lapisKiku.wordCardKind` picks the card-type flag set on word cards; see [Word Card Type](#word-card-type). It is read only while `isLapis` or `isKiku` is enabled.
|
||||
|
||||
### Word Card Type
|
||||
|
||||
When SubMiner fills the sentence on a mined word card - from Yomitan auto-enrichment, a manual clipboard update, or stats-dashboard word mining - it marks which card that note should generate. `ankiConnect.lapisKiku.wordCardKind` chooses the flag:
|
||||
|
||||
| Value | Flag set |
|
||||
| ----------------------------- | ----------------------- |
|
||||
| `word-and-sentence` (default) | `IsWordAndSentenceCard` |
|
||||
| `click` | `IsClickCard` |
|
||||
| `sentence` | `IsSentenceCard` |
|
||||
| `audio` | `IsAudioCard` |
|
||||
| `none` | none; flags left as-is |
|
||||
|
||||
The other card-type flags are cleared so a note never claims two card types at once. Notes are skipped when the note type has no field for the chosen flag, and when the note was already mined as a sentence or audio card. Cards created by Mine Sentence and Mine Audio keep their own flag regardless of this setting.
|
||||
|
||||
### N+1 Word Highlighting
|
||||
|
||||
@@ -1107,7 +1076,6 @@ Known-word cache policy:
|
||||
- `subtitleStyle.nPlusOneColor` sets the color for the single target token when exactly one eligible unknown word exists.
|
||||
- The N+1 minimum sentence-word setting controls the token count required before N+1 highlighting can trigger.
|
||||
- `subtitleStyle.knownWordColor` sets the known-word highlight color for tokens already in Anki.
|
||||
- Set `ankiConnect.knownWords.maturityEnabled` to `true` to color known words by Anki card maturity instead, using the four `subtitleStyle.knownWordMaturityColors` tiers. See [Known-Word Maturity Highlighting](/subtitle-annotations#known-word-maturity-highlighting) for how tiers are derived. Changing it or `matureThresholdDays` forces a full cache refresh.
|
||||
- The known-word deck map accepts an object keyed by deck name.
|
||||
- Prefer expression/word fields such as `Expression` or `Word`. Avoid reading-only fields unless you intentionally want homophone readings to count as known words.
|
||||
- Cache state is persisted to `known-words-cache.json` under the app `userData` directory.
|
||||
@@ -1146,7 +1114,6 @@ When the manual merge popup opens, SubMiner pauses playback and closes any open
|
||||
|
||||
<video controls playsinline preload="metadata" :poster="withBase('/assets/kiku-integration-poster.jpg')" style="width: 100%; max-width: 960px;">
|
||||
<source :src="withBase('/assets/kiku-integration.webm')" type="video/webm" />
|
||||
<source :src="withBase('/assets/kiku-integration.mp4')" type="video/mp4" />
|
||||
Your browser does not support the video tag.
|
||||
</video>
|
||||
|
||||
@@ -1172,28 +1139,6 @@ Configure Jimaku API access and defaults:
|
||||
|
||||
Jimaku is rate limited; if you hit a limit, SubMiner will surface the retry delay from the API response.
|
||||
|
||||
### TsukiHime
|
||||
|
||||
TsukiHime subtitle search works out of the box and needs no account or API key. It does require the `xz` binary on your `PATH`, because TsukiHime serves extracted subtitles xz-compressed.
|
||||
|
||||
```json
|
||||
{
|
||||
"tsukihime": {
|
||||
"apiBaseUrl": "https://api.tsukihime.org/v1",
|
||||
"maxSearchResults": 10
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ---------------------------- | ------------ | ----------------------------------------------------------------------------------------------------- |
|
||||
| `tsukihime.apiBaseUrl` | string (URL) | Base URL of the TsukiHime API (default: `https://api.tsukihime.org/v1`). Only change it for a mirror. |
|
||||
| `tsukihime.maxSearchResults` | number | Maximum releases returned per search (default: `10`; the API caps this at 100) |
|
||||
|
||||
The keyboard shortcut lives under `shortcuts.openTsukihime` (default `Ctrl+Shift+T`; set to `null` to disable). The older `animetosho` section and `shortcuts.openAnimetosho` are still accepted as deprecated aliases, with the current names taking precedence when both are set.
|
||||
|
||||
See [TsukiHime Integration](/tsukihime-integration) for the modal workflow, language tabs, and troubleshooting.
|
||||
|
||||
### Subtitle Sync
|
||||
|
||||
Sync the active subtitle track from the overlay picker using `alass` or `ffsubsync`. Both are **optional external tools** that must be installed separately and available on your `PATH` (or configured via the path options below).
|
||||
@@ -1246,17 +1191,17 @@ AniList integration is opt-in and disabled by default. Enable it to allow SubMin
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| -------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `anilist.enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
|
||||
| `accessToken` | string | Optional explicit AniList access token override (default: empty string) |
|
||||
| `characterDictionary.maxLoaded` | number | Maximum number of most-recently-used AniList media snapshots included in the merged dictionary (default: `3`) |
|
||||
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
|
||||
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
|
||||
| `characterDictionary.collapsibleSections.description` | `true`, `false` | Open the Description section by default in generated dictionary entries |
|
||||
| `characterDictionary.collapsibleSections.characterInformation` | `true`, `false` | Open the Character Information section by default in generated dictionary entries |
|
||||
| `characterDictionary.collapsibleSections.voicedBy` | `true`, `false` | Open the Voiced by section by default in generated dictionary entries |
|
||||
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary settings updates to all Yomitan profiles or only active profile |
|
||||
| Option | Values | Description |
|
||||
| -------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| `anilist.enabled` | `true`, `false` | Enable AniList post-watch progress updates (default: `false`) |
|
||||
| `accessToken` | string | Optional explicit AniList access token override (default: empty string) |
|
||||
| `characterDictionary.maxLoaded` | number | Maximum number of most-recently-used AniList media snapshots included in the merged dictionary (default: `3`) |
|
||||
| `characterDictionary.refreshTtlHours` | number | Hours before a cached media snapshot is refreshed (default: `168`, clamped to 1–8760) |
|
||||
| `characterDictionary.evictionPolicy` | `"delete"`, `"disable"` | What happens to snapshots evicted beyond `maxLoaded` (default: `"delete"`) |
|
||||
| `characterDictionary.collapsibleSections.description` | `true`, `false` | Open the Description section by default in generated dictionary entries |
|
||||
| `characterDictionary.collapsibleSections.characterInformation` | `true`, `false` | Open the Character Information section by default in generated dictionary entries |
|
||||
| `characterDictionary.collapsibleSections.voicedBy` | `true`, `false` | Open the Voiced by section by default in generated dictionary entries |
|
||||
| `characterDictionary.profileScope` | `"all"`, `"active"` | Apply dictionary settings updates to all Yomitan profiles or only active profile |
|
||||
|
||||
When `enabled` is `true` and `accessToken` is empty, SubMiner opens an AniList setup helper window. Keep `enabled` as `false` to disable all AniList setup/update behavior.
|
||||
|
||||
@@ -1546,7 +1491,7 @@ Configure the mpv executable, profile, and window state for SubMiner-managed mpv
|
||||
"executablePath": "",
|
||||
"launchMode": "normal",
|
||||
"profile": "",
|
||||
"socketPath": "/tmp/subminer-socket",
|
||||
"socketPath": "\\\\.\\pipe\\subminer-socket",
|
||||
"backend": "auto",
|
||||
"autoStartSubMiner": true,
|
||||
"pauseUntilOverlayReady": true,
|
||||
@@ -1557,18 +1502,18 @@ Configure the mpv executable, profile, and window state for SubMiner-managed mpv
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ------------------------ | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `executablePath` | string | Absolute path to `mpv.exe` for Windows launch flows. Leave empty to auto-discover from `SUBMINER_MPV_PATH` or `PATH` (default `""`) |
|
||||
| `profile` | string | mpv profile name passed as `--profile=<name>`. Leave empty to pass no profile (default `""`) |
|
||||
| `launchMode` | `"normal"` \| `"maximized"` \| `"fullscreen"` | Window state when SubMiner spawns mpv (default `"normal"`) |
|
||||
| Option | Values | Description |
|
||||
| ------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `executablePath` | string | Absolute path to `mpv.exe` for Windows launch flows. Leave empty to auto-discover from `SUBMINER_MPV_PATH` or `PATH` (default `""`) |
|
||||
| `profile` | string | mpv profile name passed as `--profile=<name>`. Leave empty to pass no profile (default `""`) |
|
||||
| `launchMode` | `"normal"` \| `"maximized"` \| `"fullscreen"` | Window state when SubMiner spawns mpv (default `"normal"`) |
|
||||
| `socketPath` | string | mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin (platform-dependent default: `/tmp/subminer-socket`, or `\\\\.\\pipe\\subminer-socket` on Windows) |
|
||||
| `backend` | `"auto"` \| `"hyprland"` \| `"sway"` \| `"x11"` \| `"macos"` \| `"windows"` | Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform (default: `"auto"`) |
|
||||
| `autoStartSubMiner` | `true`, `false` | Start SubMiner in the background when SubMiner-managed mpv loads a file (default: `true`) |
|
||||
| `pauseUntilOverlayReady` | `true`, `false` | Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness, with a 30-second fallback (default: `true`) |
|
||||
| `subminerBinaryPath` | string | SubMiner app binary path passed to the bundled mpv plugin. Leave empty to use the launcher-detected app path (default: `""`) |
|
||||
| `aniskipEnabled` | `true`, `false` | Enable AniSkip intro detection, chapter markers, and the skip-intro key (default: `true`) |
|
||||
| `aniskipButtonKey` | string | mpv key used to skip the detected intro while the skip prompt is visible (default: `"TAB"`) |
|
||||
| `backend` | `"auto"` \| `"hyprland"` \| `"sway"` \| `"x11"` \| `"macos"` \| `"windows"` | Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform (default: `"auto"`) |
|
||||
| `autoStartSubMiner` | `true`, `false` | Start SubMiner in the background when SubMiner-managed mpv loads a file (default: `true`) |
|
||||
| `pauseUntilOverlayReady` | `true`, `false` | Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness, with a 30-second fallback (default: `true`) |
|
||||
| `subminerBinaryPath` | string | SubMiner app binary path passed to the bundled mpv plugin. Leave empty to use the launcher-detected app path (default: `""`) |
|
||||
| `aniskipEnabled` | `true`, `false` | Enable AniSkip intro detection, chapter markers, and the skip-intro key (default: `true`) |
|
||||
| `aniskipButtonKey` | string | mpv key used to skip the detected intro while the skip prompt is visible (default: `"TAB"`) |
|
||||
|
||||
If `mpv.profile` is configured and the launcher also receives `--profile`, SubMiner passes both as a comma-separated mpv profile list.
|
||||
|
||||
|
||||
@@ -20,6 +20,9 @@ Mine vocabulary cards from Yomitan or directly from subtitle lines. SubMiner aut
|
||||
</a>
|
||||
</video>
|
||||
|
||||
::: info VIDEO COMING SOON
|
||||
:::
|
||||
|
||||
## Subtitle Download & Sync
|
||||
|
||||
Search and download subtitles from Jimaku, then retime them with alass or ffsubsync - all from within SubMiner.
|
||||
|
||||
@@ -14,10 +14,6 @@ const ankiIntegrationContents = readFileSync(
|
||||
'utf8',
|
||||
);
|
||||
const configurationContents = readFileSync(new URL('./configuration.md', import.meta.url), 'utf8');
|
||||
const troubleshootingContents = readFileSync(
|
||||
new URL('./troubleshooting.md', import.meta.url),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
function extractReleaseHeadings(content: string, count: number): string[] {
|
||||
return Array.from(content.matchAll(/^## v[^\n]+$/gm))
|
||||
@@ -62,33 +58,6 @@ test('docs reflect current launcher and release surfaces', () => {
|
||||
expect(changelogContents).toContain('v0.5.1 (2026-03-09)');
|
||||
});
|
||||
|
||||
test('docs document config surfaces that are easy to miss when they ship', () => {
|
||||
// Anki maturity-based known-word highlighting (#172) landed in
|
||||
// subtitle-annotations.md but was missing from the config reference.
|
||||
expect(configurationContents).toContain('ankiConnect.knownWords.maturityEnabled');
|
||||
expect(configurationContents).toContain('ankiConnect.knownWords.matureThresholdDays');
|
||||
|
||||
// Every top-level config block should be reachable from the config reference.
|
||||
expect(configurationContents).toContain('### TsukiHime');
|
||||
expect(configurationContents).toContain('tsukihime.maxSearchResults');
|
||||
|
||||
// xz is a hard runtime dependency of the TsukiHime download path.
|
||||
expect(installationContents).toContain('xz');
|
||||
});
|
||||
|
||||
test('docs state the real secondary-subtitle and Anki field-matching behavior', () => {
|
||||
// secondarySub auto-load is off by default; the config example previously
|
||||
// implied otherwise while youtube-integration.md documented it correctly.
|
||||
expect(configurationContents).toContain('Secondary subtitles do **not** auto-load by default');
|
||||
expect(configurationContents).toContain('default: `false`');
|
||||
|
||||
// Anki field names are matched case-insensitively (src/anki-integration.ts
|
||||
// resolveFieldName: exact match first, then a lowercase comparison).
|
||||
expect(usageContents).not.toContain('exactly (case-sensitive)');
|
||||
expect(troubleshootingContents).not.toContain('exactly (case-sensitive)');
|
||||
expect(ankiIntegrationContents).toContain('case-insensitively');
|
||||
});
|
||||
|
||||
test('docs dev server links version navigation to local dev routes', () => {
|
||||
expect(docsPackageContents).toContain('scripts/build-versioned-docs.ts');
|
||||
expect(docsPackageContents).toContain(
|
||||
|
||||
@@ -57,8 +57,6 @@ Jellyfin stream URLs are normalized to stable item links before stats titles are
|
||||
|
||||
When YouTube channel metadata is available, the Library tab groups videos by creator/channel and treats each tracked video as an episode-like entry inside that channel section.
|
||||
|
||||
Open a title and use **Delete Entry** in its header to remove a mistakenly tracked show outright. This deletes every episode of that title along with their sessions, subtitle lines, rollups and cover art, drops the words and kanji that were only seen there, and removes the card from the Library grid. Individual episodes and sessions can still be deleted on their own from the episode list and session rows. Entry deletion is refused while that title is the one currently playing.
|
||||
|
||||

|
||||
|
||||
#### Trends
|
||||
|
||||
+17
-109
@@ -12,23 +12,20 @@ Three steps to get started:
|
||||
|
||||
Only **mpv** is strictly required to run SubMiner. Everything else enhances the experience but is optional.
|
||||
|
||||
Several entries below exist only for the `subminer` command-line launcher, which is Linux and macOS only. On Windows you launch playback with the **SubMiner mpv** shortcut instead, so you can ignore those rows.
|
||||
|
||||
| Dependency | Status | Platforms | What it does |
|
||||
| -------------------- | ----------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| mpv | Required | All | The video player SubMiner overlays on. Must support `--input-ipc-server`. |
|
||||
| ffmpeg | Recommended | All | Audio extraction and screenshots for Anki cards. Without it SubMiner still runs, but media fields will be empty. |
|
||||
| MeCab + mecab-ipadic | Recommended | All | Part-of-speech filtering for more precise N+1, JLPT, and frequency annotations. Without it annotations still render, but POS-based filtering is less accurate. |
|
||||
| yt-dlp | Optional | All | YouTube playback and subtitle extraction. |
|
||||
| xz | Optional | All | Required for TsukiHime subtitle downloads (subtitles are served xz-compressed). Preinstalled on most Linux distros; not present on Windows by default. |
|
||||
| guessit | Optional | All | Better AniSkip title/season/episode parsing. |
|
||||
| alass | Optional | All | Subtitle sync engine (preferred). Disabled without alass or ffsubsync. |
|
||||
| ffsubsync | Optional | All | Audio-based subtitle sync engine. Disabled without alass or ffsubsync. |
|
||||
| fzf | Optional | Linux, macOS | Terminal-based video picker in the `subminer` launcher. |
|
||||
| rofi | Optional | Linux | GUI-based video picker in the `subminer` launcher. |
|
||||
| chafa | Optional | Linux, macOS | Thumbnail previews in the fzf picker. |
|
||||
| ffmpegthumbnailer | Optional | Linux, macOS | Video thumbnail generation for the pickers. |
|
||||
| fuse2 | Required | Linux | Needed to run the AppImage. |
|
||||
| Dependency | Status | What it does |
|
||||
| -------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| mpv | Required | The video player SubMiner overlays on. Must support `--input-ipc-server`. |
|
||||
| ffmpeg | Recommended | Audio extraction and screenshots for Anki cards. Without it SubMiner still runs, but media fields will be empty. |
|
||||
| MeCab + mecab-ipadic | Recommended | Part-of-speech filtering for more precise N+1, JLPT, and frequency annotations. Without it annotations still render, but POS-based filtering is less accurate. |
|
||||
| yt-dlp | Optional | YouTube playback and subtitle extraction. |
|
||||
| fzf | Optional | Terminal-based video picker in the launcher. |
|
||||
| rofi | Optional | GUI-based video picker (Linux). |
|
||||
| chafa | Optional | Thumbnail previews in fzf. |
|
||||
| ffmpegthumbnailer | Optional | Video thumbnail generation for the picker. |
|
||||
| guessit | Optional | Better AniSkip title/season/episode parsing. |
|
||||
| alass | Optional | Subtitle sync engine (preferred). Disabled without alass or ffsubsync. |
|
||||
| ffsubsync | Optional | Audio-based subtitle sync engine. Disabled without alass or ffsubsync. |
|
||||
| fuse2 | Linux only | Required to run the AppImage. |
|
||||
|
||||
### Linux
|
||||
|
||||
@@ -112,98 +109,9 @@ pip install ffsubsync
|
||||
|
||||
### Windows
|
||||
|
||||
Windows 10 or later. No compositor tools or window helpers are needed - native window tracking is built in.
|
||||
Windows 10 or later. Install [`mpv`](https://mpv.io/installation/) and [`ffmpeg`](https://ffmpeg.org/download.html) and ensure both are on `PATH`. Optionally install [MeCab for Windows](https://taku910.github.io/mecab/#download) with the UTF-8 dictionary.
|
||||
|
||||
You need **mpv** (required) and **ffmpeg** (strongly recommended, for card audio and screenshots), and both must be on your `PATH`.
|
||||
|
||||
::: tip What is PATH?
|
||||
`PATH` is the list of folders Windows searches when a program asks to run another program by name. SubMiner runs `mpv` and `ffmpeg` by name, so if their folders are not on `PATH`, SubMiner cannot find them even though they are installed. The routes below mostly handle `PATH` for you; the manual route explains how to add a folder yourself.
|
||||
:::
|
||||
|
||||
You can install these with a package manager or by hand. Coverage differs, so pick based on what you need:
|
||||
|
||||
| Dependency | winget | Scoop |
|
||||
| ---------------- | --------------- | ------------- |
|
||||
| mpv (required) | `shinchiro.mpv` | `extras/mpv` |
|
||||
| ffmpeg | `Gyan.FFmpeg` | `main/ffmpeg` |
|
||||
| yt-dlp (YouTube) | `yt-dlp.yt-dlp` | `main/yt-dlp` |
|
||||
| xz (TsukiHime) | not packaged | `main/xz` |
|
||||
|
||||
Use **winget** if you want Microsoft's first-party tool and don't need TsukiHime subtitle downloads. Use **Scoop** if you want one package manager to cover everything, since it is the only one that also packages `xz`.
|
||||
|
||||
#### Recommended: winget
|
||||
|
||||
[winget](https://learn.microsoft.com/windows/package-manager/winget/) is Microsoft's own package manager and ships with Windows 11 and current Windows 10 (it comes with **App Installer** from the Microsoft Store). In **PowerShell** or **Command Prompt**:
|
||||
|
||||
```powershell
|
||||
winget install shinchiro.mpv
|
||||
winget install Gyan.FFmpeg
|
||||
```
|
||||
|
||||
Close and reopen your terminal, then check that both are found:
|
||||
|
||||
```powershell
|
||||
mpv --version
|
||||
ffmpeg -version
|
||||
```
|
||||
|
||||
`ffmpeg` is installed as a portable package, so winget links it into a folder that is already on your `PATH` and it should work right away.
|
||||
|
||||
`mpv` uses a regular installer, and depending on the version it may **not** add itself to `PATH`. If `mpv --version` says `not recognized`, you have two easy options:
|
||||
|
||||
- Note where it installed (usually `%LOCALAPPDATA%\Programs\mpv`) and add that folder to `PATH` using the manual steps below, or
|
||||
- Skip `PATH` entirely and set `mpv.executablePath` to the full path of `mpv.exe` during first-run setup.
|
||||
|
||||
Once `mpv --version` works, or you have the full path to `mpv.exe` ready, continue to [step 2](#_2-install-subminer).
|
||||
|
||||
<details>
|
||||
<summary><b>Alternative: Scoop (covers every dependency, no admin rights)</b></summary>
|
||||
|
||||
[Scoop](https://scoop.sh) installs into your user profile, needs no administrator prompt, and always puts commands on `PATH`. It is the only Windows package manager that carries all of SubMiner's optional dependencies, including `xz`, so it is the best choice if you want a single tool to manage everything.
|
||||
|
||||
```powershell
|
||||
# One-time Scoop setup (skip if you already have it)
|
||||
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
|
||||
Invoke-RestMethod -Uri https://get.scoop.sh | Invoke-Expression
|
||||
|
||||
# mpv lives in the "extras" bucket; everything else is in "main"
|
||||
scoop bucket add extras
|
||||
scoop install extras/mpv main/ffmpeg
|
||||
|
||||
# Optional: yt-dlp for YouTube playback, xz for TsukiHime subtitle downloads
|
||||
scoop install main/yt-dlp main/xz
|
||||
```
|
||||
|
||||
Close and reopen your terminal, then verify with `mpv --version` and `ffmpeg -version`.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Manual install (download the zips yourself)</b></summary>
|
||||
|
||||
1. Download mpv from [mpv.io/installation](https://mpv.io/installation/) (the Windows builds link) and ffmpeg from [ffmpeg.org/download.html](https://ffmpeg.org/download.html).
|
||||
2. Unzip each one somewhere permanent, for example `C:\Tools\mpv` and `C:\Tools\ffmpeg`. Note the folder that actually contains `mpv.exe` and the one containing `ffmpeg.exe` (for ffmpeg this is usually a `bin` subfolder).
|
||||
3. Press `Win`, type **Edit the system environment variables**, and open it. Click **Environment Variables…**, select **Path** under **User variables**, click **Edit…**, then use **New** to add each of those two folders. Confirm with **OK** on every dialog. Microsoft documents this in more detail under [environment variables](https://learn.microsoft.com/windows/deployment/usmt/usmt-recognized-environment-variables).
|
||||
4. Close and reopen your terminal, since `PATH` changes only apply to newly opened windows. Then check:
|
||||
|
||||
```powershell
|
||||
mpv --version
|
||||
ffmpeg -version
|
||||
```
|
||||
|
||||
If you see `not recognized as the name of a cmdlet`, the folder you added is not the one holding the `.exe`. Reopen the Path editor and double-check.
|
||||
|
||||
::: tip mpv can skip PATH, ffmpeg cannot
|
||||
If you would rather not edit `PATH` for mpv, set `mpv.executablePath` to the full path of `mpv.exe` during first-run setup instead.
|
||||
|
||||
There is no equivalent setting for ffmpeg: SubMiner invokes it by bare name when generating card audio and screenshots, so ffmpeg has to be on `PATH`. Without it, cards are still created but their audio and image fields come out empty. (`subsync.ffmpeg_path` only affects subtitle sync, not card media.)
|
||||
:::
|
||||
|
||||
</details>
|
||||
|
||||
**Optional extras:** [MeCab for Windows](https://taku910.github.io/mecab/#download) with the UTF-8 dictionary improves annotation accuracy; it is not in any package manager, so install it from that page. `xz` is needed only for [TsukiHime](/tsukihime-integration) subtitle downloads and is not packaged by winget or Chocolatey, so use `scoop install main/xz` or download [XZ Utils](https://tukaani.org/xz/) and add its folder to `PATH`.
|
||||
|
||||
The `subminer` command-line launcher and its picker tools (`fzf`, `rofi`, `chafa`, `ffmpegthumbnailer`) are Linux/macOS only; on Windows you use the **SubMiner mpv** shortcut instead.
|
||||
No compositor tools or window helpers are needed - native window tracking is built in.
|
||||
|
||||
## 2. Install SubMiner
|
||||
|
||||
@@ -370,7 +278,7 @@ Run the built-in diagnostic to confirm everything is working:
|
||||
subminer doctor
|
||||
```
|
||||
|
||||
This checks for the app binary, mpv, ffmpeg, yt-dlp, fzf, rofi, your config file, and the mpv socket path. Only the app binary and mpv are hard failures; the rest are reported as optional. Fix any hard failures before continuing.
|
||||
This checks for the app binary, mpv, ffmpeg, config file, and socket path. Fix any failures before continuing.
|
||||
|
||||
## Anki Setup (Recommended)
|
||||
|
||||
|
||||
@@ -112,5 +112,5 @@ Verify mpv is running and connected via IPC. SubMiner loads the subtitle by issu
|
||||
## Related
|
||||
|
||||
- [Configuration Reference](/configuration#jimaku) - full config options
|
||||
- [Mining Workflow](/mining-workflow#related-features) - how Jimaku fits into the sentence mining loop
|
||||
- [Mining Workflow](/mining-workflow#jimaku-subtitle-search) - how Jimaku fits into the sentence mining loop
|
||||
- [Troubleshooting](/troubleshooting#jimaku) - additional error guidance
|
||||
|
||||
@@ -176,22 +176,22 @@ Use `subminer <subcommand> -h` for command-specific help.
|
||||
|
||||
## Options
|
||||
|
||||
| Flag | Description |
|
||||
| --------------------- | ---------------------------------------------------------------------------- |
|
||||
| `-d, --directory` | Video search directory (default: cwd) |
|
||||
| `-r, --recursive` | Search directories recursively |
|
||||
| `-R, --rofi` | Use rofi instead of fzf |
|
||||
| `-H, --history` | Browse local watch history (see [Watch History](#watch-history)) |
|
||||
| `-v, --version` | Print the launcher's own version (can differ from the installed app binary) |
|
||||
| `-u, --update` | Check for SubMiner updates and update the app/launcher when possible |
|
||||
| `--start` | Explicitly start overlay after mpv launches |
|
||||
| `-S, --start-overlay` | Force the visible overlay on start |
|
||||
| `-T, --no-texthooker` | Disable texthooker server |
|
||||
| `-p, --profile` | mpv profile name (no default; omitted unless set) |
|
||||
| `-a, --args` | Pass additional mpv arguments as a quoted string |
|
||||
| `-b, --backend` | Force window backend (`auto`, `hyprland`, `sway`, `x11`, `macos`, `windows`) |
|
||||
| `--settings` | Open the SubMiner settings window |
|
||||
| `--log-level` | Logger verbosity (`debug`, `info`, `warn`, `error`) |
|
||||
| Flag | Description |
|
||||
| --------------------- | --------------------------------------------------------------------------- |
|
||||
| `-d, --directory` | Video search directory (default: cwd) |
|
||||
| `-r, --recursive` | Search directories recursively |
|
||||
| `-R, --rofi` | Use rofi instead of fzf |
|
||||
| `-H, --history` | Browse local watch history (see [Watch History](#watch-history)) |
|
||||
| `-v, --version` | Print the launcher's own version (can differ from the installed app binary) |
|
||||
| `-u, --update` | Check for SubMiner updates and update the app/launcher when possible |
|
||||
| `--start` | Explicitly start overlay after mpv launches |
|
||||
| `-S, --start-overlay` | Force the visible overlay on start |
|
||||
| `-T, --no-texthooker` | Disable texthooker server |
|
||||
| `-p, --profile` | mpv profile name (no default; omitted unless set) |
|
||||
| `-a, --args` | Pass additional mpv arguments as a quoted string |
|
||||
| `-b, --backend` | Force window backend (`hyprland`, `sway`, `x11`, `macos`, `windows`) |
|
||||
| `--settings` | Open the SubMiner settings window |
|
||||
| `--log-level` | Logger verbosity (`debug`, `info`, `warn`, `error`) |
|
||||
|
||||
App-binary flags such as `--setup`, `--dev`, and `--debug` are not launcher flags - pass them through with `subminer app`, for example `subminer app --setup`.
|
||||
|
||||
|
||||
@@ -1,132 +0,0 @@
|
||||
import { expect, test } from 'bun:test';
|
||||
import { readdirSync, readFileSync } from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const docsSiteDir = fileURLToPath(new URL('.', import.meta.url));
|
||||
|
||||
// Mirrors VitePress' heading slugifier (vitepress/dist/node, `rControl` + `rSpecial`).
|
||||
// Note that a *run* of special characters collapses to a single `-`, so
|
||||
// "KDE Plasma & other" becomes "kde-plasma-other", not "kde-plasma--other".
|
||||
const rControl = new RegExp('[\\u0000-\\u001f]', 'g');
|
||||
const rSpecial = /[\s~`!@#$%^&*()\-_+=[\]{}|\\;:"'“”‘’<>,.?/]+/g;
|
||||
|
||||
function slugify(heading: string): string {
|
||||
return heading
|
||||
.replace(rControl, '')
|
||||
.replace(rSpecial, '-')
|
||||
.replace(/-{2,}/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
.replace(/^(\d)/, '_$1')
|
||||
.toLowerCase();
|
||||
}
|
||||
|
||||
const EXCLUDED_PAGES = new Set(['README.md']);
|
||||
const PUBLIC_PREFIXES = ['/assets/', '/screenshots/', '/config.example.jsonc', '/favicon'];
|
||||
|
||||
function loadPages(): Map<string, string> {
|
||||
const pages = new Map<string, string>();
|
||||
for (const file of readdirSync(docsSiteDir)) {
|
||||
if (!file.endsWith('.md') || EXCLUDED_PAGES.has(file)) continue;
|
||||
const route = `/${file.replace(/\.md$/, '')}`;
|
||||
pages.set(route, readFileSync(`${docsSiteDir}${file}`, 'utf8'));
|
||||
}
|
||||
return pages;
|
||||
}
|
||||
|
||||
function anchorsFor(contents: string): Set<string> {
|
||||
const anchors = new Set<string>();
|
||||
for (const match of contents.matchAll(/^#{1,6}\s+(.+?)\s*$/gm)) {
|
||||
let heading = match[1]!;
|
||||
const explicitId = heading.match(/\{#([^}]+)\}\s*$/);
|
||||
if (explicitId) {
|
||||
anchors.add(explicitId[1]!);
|
||||
heading = heading.replace(/\{#[^}]+\}\s*$/, '');
|
||||
}
|
||||
anchors.add(slugify(heading.replace(/`/g, '')));
|
||||
}
|
||||
return anchors;
|
||||
}
|
||||
|
||||
function resolveRoute(target: string, fromRoute: string): string {
|
||||
if (target === '') return fromRoute;
|
||||
if (target.startsWith('./')) return `/${target.slice(2).replace(/\.md$/, '')}`;
|
||||
const normalized = target.replace(/\.md$/, '').replace(/\/$/, '');
|
||||
return normalized === '' ? '/index' : normalized;
|
||||
}
|
||||
|
||||
const pages = loadPages();
|
||||
const anchors = new Map([...pages].map(([route, body]) => [route, anchorsFor(body)]));
|
||||
|
||||
test('every internal docs link resolves to an existing page', () => {
|
||||
const broken: string[] = [];
|
||||
|
||||
for (const [route, body] of pages) {
|
||||
for (const match of body.matchAll(/\]\((\/[^)\s]*|\.\/[^)\s]*)\)/g)) {
|
||||
const link = match[1]!;
|
||||
const target = link.split('#')[0]!;
|
||||
if (PUBLIC_PREFIXES.some((prefix) => target.startsWith(prefix))) continue;
|
||||
|
||||
const resolved = resolveRoute(target, route);
|
||||
if (resolved !== '/index' && !pages.has(resolved)) {
|
||||
broken.push(`${route.slice(1)}.md -> ${link}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
expect(broken).toEqual([]);
|
||||
});
|
||||
|
||||
test('every internal docs anchor matches a real heading slug', () => {
|
||||
const broken: string[] = [];
|
||||
|
||||
for (const [route, body] of pages) {
|
||||
for (const match of body.matchAll(/\]\((\/[^)\s]*|\.\/[^)\s]*|#[^)\s]*)\)/g)) {
|
||||
const link = match[1]!;
|
||||
const hashIndex = link.indexOf('#');
|
||||
if (hashIndex < 0) continue;
|
||||
|
||||
const target = link.slice(0, hashIndex);
|
||||
const anchor = link.slice(hashIndex + 1);
|
||||
if (PUBLIC_PREFIXES.some((prefix) => target.startsWith(prefix))) continue;
|
||||
|
||||
const resolved = resolveRoute(target, route);
|
||||
const pageAnchors = anchors.get(resolved);
|
||||
if (!pageAnchors || pageAnchors.has(anchor)) continue;
|
||||
|
||||
broken.push(`${route.slice(1)}.md -> ${link}`);
|
||||
}
|
||||
}
|
||||
|
||||
expect(broken).toEqual([]);
|
||||
});
|
||||
|
||||
test('slugify matches the VitePress cases these docs actually rely on', () => {
|
||||
// Regression guards for the anchors that were previously wrong.
|
||||
expect(slugify('N+1 Word Highlighting')).toBe('n-1-word-highlighting');
|
||||
expect(slugify('KDE Plasma & other Wayland compositors')).toBe(
|
||||
'kde-plasma-other-wayland-compositors',
|
||||
);
|
||||
expect(slugify('Proxy Mode Setup (Yomitan / Texthooker)')).toBe(
|
||||
'proxy-mode-setup-yomitan-texthooker',
|
||||
);
|
||||
expect(slugify('Kiku/Lapis Integration')).toBe('kiku-lapis-integration');
|
||||
expect(slugify('Secondary Subtitles')).toBe('secondary-subtitles');
|
||||
expect(slugify('2. Install SubMiner')).toBe('_2-install-subminer');
|
||||
});
|
||||
|
||||
test('every docs page is reachable from the sidebar', async () => {
|
||||
const { default: config } = await import('./.vitepress/config');
|
||||
const sidebar = config.themeConfig?.sidebar as Array<{
|
||||
items?: Array<{ text: string; link?: string }>;
|
||||
}>;
|
||||
|
||||
const linked = new Set<string>();
|
||||
for (const group of sidebar) {
|
||||
for (const item of group.items ?? []) {
|
||||
if (item.link) linked.add(item.link === '/' ? '/index' : item.link);
|
||||
}
|
||||
}
|
||||
|
||||
const orphans = [...pages.keys()].filter((route) => !linked.has(route));
|
||||
expect(orphans).toEqual([]);
|
||||
});
|
||||
@@ -183,7 +183,7 @@ If you want to build your own browser client, websocket consumer, or automation
|
||||
These features support the mining loop but have their own dedicated pages:
|
||||
|
||||
- **[Jimaku subtitle search](/jimaku-integration)** - search and download anime subtitle files directly from the overlay (`Ctrl+Shift+J` by default), then load them into mpv.
|
||||
- **[N+1 word highlighting](/subtitle-annotations#n-1-word-highlighting)** - cross-reference your Anki decks to highlight known words, making true N+1 sentences (exactly one unknown word) easy to spot during immersion.
|
||||
- **[N+1 word highlighting](/subtitle-annotations#n1-word-highlighting)** - cross-reference your Anki decks to highlight known words, making true N+1 sentences (exactly one unknown word) easy to spot during immersion.
|
||||
- **[Immersion tracking](/immersion-tracking)** - log watching and mining activity to a local database and view session times, words seen, and cards mined in the built-in stats dashboard.
|
||||
|
||||
Next: [Anki Integration](/anki-integration) - field mapping, media generation, and card enrichment configuration.
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
"docs:dev": "SUBMINER_DOCS_VERSION_LINK_ORIGIN=local bun run ../scripts/build-versioned-docs.ts && SUBMINER_DOCS_VERSION_LINK_ORIGIN=local SUBMINER_DOCS_VERSION_MANIFEST=\"$(bun run ../scripts/print-docs-version-manifest.ts)\" VITE_EXTRA_EXTENSIONS=jsonc vitepress dev --host 0.0.0.0 --port 5173 --strictPort",
|
||||
"docs:build": "VITE_EXTRA_EXTENSIONS=jsonc vitepress build",
|
||||
"docs:preview": "VITE_EXTRA_EXTENSIONS=jsonc vitepress preview --host 0.0.0.0 --port 4173 --strictPort",
|
||||
"test": "bun test plausible.test.ts index.assets.test.ts docs-sync.test.ts links.test.ts seo.test.ts .vitepress/theme/status-line.test.ts ../scripts/docs-versioning.test.ts"
|
||||
"test": "bun test plausible.test.ts index.assets.test.ts docs-sync.test.ts seo.test.ts .vitepress/theme/status-line.test.ts ../scripts/docs-versioning.test.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@catppuccin/vitepress": "^0.1.2",
|
||||
|
||||
@@ -523,7 +523,7 @@
|
||||
// ==========================================
|
||||
// AnkiConnect Integration
|
||||
// Automatic Anki updates and media generation options.
|
||||
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.
|
||||
// Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.
|
||||
// Shared AI provider transport settings are read from top-level ai and typically require restart.
|
||||
// Most other AnkiConnect settings still require restart.
|
||||
// ==========================================
|
||||
@@ -605,10 +605,7 @@
|
||||
"enabled": false, // Enable Kiku-specific mining behaviors (duplicate handling, field grouping). Values: true | false
|
||||
"fieldGrouping": "disabled", // Kiku duplicate-card field grouping mode. Values: auto | manual | disabled
|
||||
"deleteDuplicateInAuto": true // When Kiku field grouping is "auto", delete the duplicate source card after grouping completes. Values: true | false
|
||||
}, // Is kiku setting.
|
||||
"lapisKiku": {
|
||||
"wordCardKind": "word-and-sentence" // Card-type flag SubMiner marks on Kiku/Lapis word cards: word-and-sentence (IsWordAndSentenceCard), click (IsClickCard), sentence (IsSentenceCard), audio (IsAudioCard), or none to leave the flags untouched. Only applies when isKiku.enabled or isLapis.enabled; the other card-type flags are cleared so a note never claims two kinds. Cards mined with the dedicated sentence/audio actions keep their own flag. Values: word-and-sentence | click | sentence | audio | none
|
||||
} // Lapis kiku setting.
|
||||
} // Is kiku setting.
|
||||
}, // Automatic Anki updates and media generation options.
|
||||
|
||||
// ==========================================
|
||||
@@ -694,7 +691,7 @@
|
||||
"executablePath": "", // Optional absolute path to mpv.exe for Windows launch flows. Leave empty to auto-discover from SUBMINER_MPV_PATH or PATH.
|
||||
"launchMode": "normal", // Default window state for SubMiner-managed mpv launches. Values: normal | maximized | fullscreen
|
||||
"profile": "", // Optional mpv profile name passed to SubMiner-managed mpv launches. Leave empty to pass no profile.
|
||||
"socketPath": "/tmp/subminer-socket", // mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin.
|
||||
"socketPath": "\\\\.\\pipe\\subminer-socket", // mpv IPC socket path used by SubMiner-managed playback and the bundled mpv plugin.
|
||||
"backend": "auto", // Window tracking backend passed to the bundled mpv plugin. Auto detects the current platform. Values: auto | hyprland | sway | x11 | macos | windows
|
||||
"autoStartSubMiner": true, // Start SubMiner in the background when SubMiner-managed mpv loads a file. Values: true | false
|
||||
"pauseUntilOverlayReady": true, // Pause mpv on visible-overlay auto-start until SubMiner signals subtitle tokenization readiness. Values: true | false
|
||||
|
||||
@@ -251,9 +251,7 @@ test('dev docs version links use local targets for version route testing', async
|
||||
delete process.env.SUBMINER_DOCS_CHANNEL;
|
||||
delete process.env.SUBMINER_DOCS_BASE;
|
||||
delete process.env.SUBMINER_DOCS_VERSION;
|
||||
// Set explicitly (like the sibling version-nav tests) so this assertion stays
|
||||
// pinned to the manifest under test instead of the config's fallback constant.
|
||||
process.env.SUBMINER_DOCS_LATEST_STABLE = 'v0.14.0';
|
||||
delete process.env.SUBMINER_DOCS_LATEST_STABLE;
|
||||
process.env.SUBMINER_DOCS_VERSION_LINK_ORIGIN = 'local';
|
||||
process.env.SUBMINER_DOCS_VERSION_MANIFEST = JSON.stringify({
|
||||
latestStable: 'v0.14.0',
|
||||
|
||||
+87
-104
@@ -16,6 +16,90 @@ SubMiner retries the connection automatically with increasing delays (200 ms, 50
|
||||
|
||||
If the overlay never appears at all, see [Playback Startup Flow](./architecture#playback-startup-flow) for how a managed launch starts mpv and brings up the overlay.
|
||||
|
||||
## Logging and App Mode
|
||||
|
||||
- Default log output is `warn`.
|
||||
- Use `--log-level` for more/less output.
|
||||
- Use `--dev`/`--debug` only to force app/dev mode (for example to get dev behavior from the overlay/app); they do not change log verbosity.
|
||||
- You can combine both, for example `SubMiner.AppImage --start --dev --log-level debug`, when you need maximum diagnostics.
|
||||
|
||||
## Performance and Resource Impact
|
||||
|
||||
### At a glance
|
||||
|
||||
- Baseline: `SubMiner --start` is usually lightweight for normal playback.
|
||||
- Common spikes come from:
|
||||
- first subtitle parse/tokenization bursts
|
||||
- media generation (`ffmpeg` audio/image and AVIF paths)
|
||||
- media sync and subtitle tooling (`alass`, `ffsubsync`)
|
||||
- `ankiConnect` enrichment (plus polling overhead when proxy mode is disabled)
|
||||
|
||||
### If playback feels sluggish
|
||||
|
||||
1. Reduce overlay workload:
|
||||
|
||||
- set secondary subtitles hidden:
|
||||
- `secondarySub.defaultMode: "hidden"`
|
||||
- disable optional enrichment:
|
||||
- `subtitleStyle.enableJlpt: false`
|
||||
- `subtitleStyle.frequencyDictionary.enabled: false`
|
||||
|
||||
2. Reduce rendering pressure:
|
||||
|
||||
- lower `subtitleStyle.css["font-size"]`
|
||||
- keep overlay complexity minimal during heavy CPU periods
|
||||
|
||||
3. Reduce media overhead:
|
||||
|
||||
- keep `ankiConnect.media.imageType` set to `static` (avoid animated AVIF unless needed)
|
||||
- lower `ankiConnect.media.imageQuality`
|
||||
- reduce `ankiConnect.media.maxMediaDuration`
|
||||
|
||||
4. Lower integration cost:
|
||||
|
||||
- disable AI translation when not needed (`ankiConnect.ai.enabled: false`)
|
||||
- if needed, run immersion telemetry with lower duration expectations (`immersionTracking.enabled: false` for constrained sessions)
|
||||
- favor the default lightweight YouTube subtitle startup settings on low-resource systems
|
||||
|
||||
### Practical low-impact profile
|
||||
|
||||
```json
|
||||
{
|
||||
"subtitleStyle": {
|
||||
"css": {
|
||||
"font-size": "30px"
|
||||
},
|
||||
"enableJlpt": false,
|
||||
"frequencyDictionary": {
|
||||
"enabled": false
|
||||
}
|
||||
},
|
||||
"secondarySub": {
|
||||
"defaultMode": "hidden"
|
||||
},
|
||||
"ankiConnect": {
|
||||
"media": {
|
||||
"imageType": "static",
|
||||
"imageQuality": 80,
|
||||
"maxMediaDuration": 12
|
||||
},
|
||||
"ai": {
|
||||
"enabled": false
|
||||
}
|
||||
},
|
||||
"immersionTracking": {
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### If usage is still high
|
||||
|
||||
- Confirm only one SubMiner instance is running.
|
||||
- Check whether bottlenecks are `ffmpeg`, `yt-dlp`, or sync tooling in system monitor.
|
||||
- Keep the default `warn` level for normal use; raise to `info` or `debug` only for targeted diagnosis.
|
||||
- Reproduce once with `SubMiner.AppImage --start --log-level debug` and open DevTools (`y` then `d`) if freezes recur.
|
||||
|
||||
**"Failed to parse MPV message"**
|
||||
|
||||
Logged when a malformed JSON line arrives from the mpv socket. Usually harmless - SubMiner skips the bad line and continues. If it happens constantly, check that nothing else is writing to the same socket path.
|
||||
@@ -64,7 +148,7 @@ SubMiner retries with exponential backoff (up to 5 s) and suppresses repeated er
|
||||
|
||||
**Cards are created but fields are empty**
|
||||
|
||||
Field names in your config must name a field that exists on your Anki note type. Matching is case-insensitive (`sentenceaudio` finds `SentenceAudio`), but the spelling must otherwise match, and unknown fields are skipped silently. Check `ankiConnect.fields` - for example, if your note type uses `SentenceAudio` but your config says `Audio`, the field will not be populated.
|
||||
Field names in your config must match your Anki note type exactly (case-sensitive). Check `ankiConnect.fields` - for example, if your note type uses `SentenceAudio` but your config says `Audio`, the field will not be populated.
|
||||
|
||||
See [Anki Integration](/anki-integration) for the full field mapping reference.
|
||||
|
||||
@@ -232,115 +316,17 @@ If subtitle sync fails (the error message is prefixed with the engine name):
|
||||
- Try running the sync tool manually to see detailed error output.
|
||||
- ffsubsync requires local files and cannot handle remote media streams (e.g., streaming URLs).
|
||||
|
||||
## TsukiHime
|
||||
|
||||
**"xz binary not found"**
|
||||
|
||||
TsukiHime serves extracted subtitles xz-compressed, so SubMiner shells out to `xz` to decompress them. Install it:
|
||||
|
||||
- **Arch Linux**: `sudo pacman -S xz`
|
||||
- **Ubuntu/Debian**: `sudo apt install xz-utils`
|
||||
- **Fedora**: `sudo dnf install xz`
|
||||
- **macOS**: `brew install xz`
|
||||
- **Windows**: neither winget nor Chocolatey packages `xz`. Use `scoop install main/xz`, or download XZ Utils from [tukaani.org/xz](https://tukaani.org/xz/) and add the folder containing `xz.exe` to your `PATH`. Restart SubMiner afterwards.
|
||||
|
||||
Most Linux distributions ship it already. See [TsukiHime Integration](/tsukihime-integration#troubleshooting) for the other TsukiHime error messages.
|
||||
|
||||
## Jimaku
|
||||
|
||||
**"Jimaku request failed" or HTTP 429**
|
||||
|
||||
The Jimaku API has rate limits. If you see 429 errors, wait for the retry duration shown in the OSD message and try again. If you have a Jimaku API key, set it in `jimaku.apiKey` or `jimaku.apiKeyCommand` to get higher rate limits.
|
||||
|
||||
## Logging and App Mode
|
||||
|
||||
- Default log output is `warn`.
|
||||
- Use `--log-level` for more/less output.
|
||||
- Use `--dev`/`--debug` only to force app/dev mode (for example to get dev behavior from the overlay/app); they do not change log verbosity.
|
||||
- You can combine both, for example `SubMiner.AppImage --start --dev --log-level debug`, when you need maximum diagnostics.
|
||||
|
||||
## Performance and Resource Impact
|
||||
|
||||
### At a glance
|
||||
|
||||
- Baseline: `SubMiner --start` is usually lightweight for normal playback.
|
||||
- Common spikes come from:
|
||||
- first subtitle parse/tokenization bursts
|
||||
- media generation (`ffmpeg` audio/image and AVIF paths)
|
||||
- media sync and subtitle tooling (`alass`, `ffsubsync`)
|
||||
- `ankiConnect` enrichment (plus polling overhead when proxy mode is disabled)
|
||||
|
||||
### If playback feels sluggish
|
||||
|
||||
1. Reduce overlay workload:
|
||||
|
||||
- set secondary subtitles hidden:
|
||||
- `secondarySub.defaultMode: "hidden"`
|
||||
- disable optional enrichment:
|
||||
- `subtitleStyle.enableJlpt: false`
|
||||
- `subtitleStyle.frequencyDictionary.enabled: false`
|
||||
|
||||
2. Reduce rendering pressure:
|
||||
|
||||
- lower `subtitleStyle.css["font-size"]`
|
||||
- keep overlay complexity minimal during heavy CPU periods
|
||||
|
||||
3. Reduce media overhead:
|
||||
|
||||
- keep `ankiConnect.media.imageType` set to `static` (avoid animated AVIF unless needed)
|
||||
- lower `ankiConnect.media.imageQuality`
|
||||
- reduce `ankiConnect.media.maxMediaDuration`
|
||||
|
||||
4. Lower integration cost:
|
||||
|
||||
- disable AI translation when not needed (`ankiConnect.ai.enabled: false`)
|
||||
- if needed, run immersion telemetry with lower duration expectations (`immersionTracking.enabled: false` for constrained sessions)
|
||||
- favor the default lightweight YouTube subtitle startup settings on low-resource systems
|
||||
|
||||
### Practical low-impact profile
|
||||
|
||||
```json
|
||||
{
|
||||
"subtitleStyle": {
|
||||
"css": {
|
||||
"font-size": "30px"
|
||||
},
|
||||
"enableJlpt": false,
|
||||
"frequencyDictionary": {
|
||||
"enabled": false
|
||||
}
|
||||
},
|
||||
"secondarySub": {
|
||||
"defaultMode": "hidden"
|
||||
},
|
||||
"ankiConnect": {
|
||||
"media": {
|
||||
"imageType": "static",
|
||||
"imageQuality": 80,
|
||||
"maxMediaDuration": 12
|
||||
},
|
||||
"ai": {
|
||||
"enabled": false
|
||||
}
|
||||
},
|
||||
"immersionTracking": {
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### If usage is still high
|
||||
|
||||
- Confirm only one SubMiner instance is running.
|
||||
- Check whether bottlenecks are `ffmpeg`, `yt-dlp`, or sync tooling in system monitor.
|
||||
- Keep the default `warn` level for normal use; raise to `info` or `debug` only for targeted diagnosis.
|
||||
- Reproduce once with `SubMiner.AppImage --start --log-level debug` and open DevTools (`y` then `d`) if freezes recur.
|
||||
|
||||
## Platform-Specific
|
||||
|
||||
### Linux
|
||||
|
||||
- **Wayland (Hyprland/Sway only)**: Native Wayland support is limited to Hyprland and Sway. Window tracking uses compositor-specific commands (`hyprctl` / `swaymsg`). If these are not on `PATH`, tracking will fail silently. Other Wayland compositors (KDE Plasma, GNOME, …) are not supported natively - both mpv and SubMiner must run under X11 or Xwayland instead. On those sessions SubMiner forces XWayland automatically for itself and for every mpv it launches (see [KDE Plasma & other Wayland compositors](#kde-plasma-other-wayland-compositors)).
|
||||
- **Wayland (Hyprland/Sway only)**: Native Wayland support is limited to Hyprland and Sway. Window tracking uses compositor-specific commands (`hyprctl` / `swaymsg`). If these are not on `PATH`, tracking will fail silently. Other Wayland compositors (KDE Plasma, GNOME, …) are not supported natively - both mpv and SubMiner must run under X11 or Xwayland instead. On those sessions SubMiner forces XWayland automatically for itself and for every mpv it launches (see [KDE Plasma & other Wayland compositors](#kde-plasma--other-wayland-compositors)).
|
||||
- **X11 / Xwayland**: Requires `xdotool`, `xprop`, and `xwininfo`. If missing, the overlay cannot track the mpv window position. This is the required backend for any Wayland compositor other than Hyprland or Sway - both mpv and SubMiner must be running under X11/Xwayland for window tracking _and_ for the overlay to stay above mpv (Wayland forbids clients from controlling window stacking). SubMiner uses a managed X11 overlay while mpv is windowed, switches to an override-redirect X11 overlay while tracked mpv is fullscreen, and hides/releases that overlay when another X11/Xwayland app takes focus. The visible overlay stays hidden until SubMiner has tracked mpv geometry, so startup should not create a display-sized fallback overlay while tokenization warms up.
|
||||
- **Tray icon missing**: SubMiner creates an Electron tray icon in `--background` mode, but Linux trays require a StatusNotifier/AppIndicator host. Hyprland does not provide one by itself; enable a tray in Waybar, Hyprpanel, or another panel. If Electron cannot register the tray, SubMiner logs a warning that mentions the missing tray host.
|
||||
- **Mouse passthrough**: On Linux X11/Xwayland, SubMiner uses `xdotool` to poll the cursor and only enables overlay input while the cursor is over subtitle or popup regions. Outside those regions, pointer input passes through to mpv. Native Wayland compositors other than Hyprland/Sway cannot provide the stacking control SubMiner needs.
|
||||
@@ -442,11 +428,8 @@ Feature-specific issues are covered in each feature's own page:
|
||||
- [Character Dictionary](/character-dictionary) - AniList character name matching and inline portraits
|
||||
- [Jellyfin Integration](/jellyfin-integration) - remote playback and library connection
|
||||
- [Jimaku Integration](/jimaku-integration) - subtitle fetching and API rate limits
|
||||
- [TsukiHime Integration](/tsukihime-integration) - multi-language subtitle download and `xz` decompression
|
||||
- [YouTube Integration](/youtube-integration) - subtitle generation and playback
|
||||
- [Immersion Tracking](/immersion-tracking) - telemetry, session logging, and the stats dashboard
|
||||
- [Launcher Script](/launcher-script) - `subminer` commands, pickers, watch history, and cross-machine sync
|
||||
- [MPV Plugin](/mpv-plugin) - in-player chords, script-opts, and binary auto-detection
|
||||
- [Immersion Tracking](/immersion-tracking) - telemetry and session logging
|
||||
- [WebSocket / Texthooker API](/websocket-texthooker-api) - external texthooker clients
|
||||
- [Subtitle Annotations](/subtitle-annotations) - N+1, frequency, JLPT, and name-match layers
|
||||
- [Subtitle Sidebar](/subtitle-sidebar) - sidebar navigation and behavior
|
||||
|
||||
+117
-96
@@ -33,7 +33,7 @@ If you want sentence, audio, and screenshot fields on your Anki cards, add this
|
||||
}
|
||||
```
|
||||
|
||||
Field names must match a field on your Anki note type. Matching is case-insensitive (an exact match wins, then a lowercase comparison), but the spelling must otherwise match. See [Anki Integration](/anki-integration) for the full reference.
|
||||
Field names must match your Anki note type exactly (case-sensitive). See [Anki Integration](/anki-integration) for the full reference.
|
||||
:::
|
||||
|
||||
## How It Works
|
||||
@@ -56,90 +56,126 @@ From there, subtitles render as interactive, hoverable word spans and you mine c
|
||||
|
||||
The mpv plugin is always available - it's bundled with SubMiner and injected at runtime. On Linux, normal `subminer` playback auto-installs the launcher-managed runtime plugin copy from the bundled app if that managed copy is missing, so no separate plugin install is needed for standard launcher usage. If you launch mpv yourself (without the launcher), pass `--input-ipc-server=/tmp/subminer-socket` in your mpv config for the overlay to connect.
|
||||
|
||||
## Live Config Reload
|
||||
|
||||
While SubMiner is running, it watches your active config file and applies safe updates automatically.
|
||||
|
||||
Live-updated settings include:
|
||||
|
||||
- `subtitleStyle`
|
||||
- `keybindings`
|
||||
- `shortcuts`
|
||||
- `secondarySub.defaultMode`
|
||||
- `subtitleSidebar`
|
||||
- `notifications`
|
||||
- `logging`
|
||||
- `jimaku`, `subsync`
|
||||
- `mpv.aniskipEnabled`, `mpv.aniskipButtonKey`
|
||||
- `stats.toggleKey`, `stats.markWatchedKey`
|
||||
- `youtube.primarySubLanguages`
|
||||
- most `ankiConnect.*` settings (including `ankiConnect.ai`)
|
||||
|
||||
Invalid config edits are rejected; SubMiner keeps the previous valid runtime config and shows an error notification.
|
||||
For restart-required sections, SubMiner shows a restart-needed notification.
|
||||
|
||||
## Commands
|
||||
|
||||
These are the commands you will actually use day to day. The full inventory of subcommands and flags lives in [Launcher Script](/launcher-script#subcommands).
|
||||
On Windows, replace `SubMiner.AppImage` with `SubMiner.exe` in the direct packaged-app examples below.
|
||||
|
||||
```bash
|
||||
subminer video.mkv # Play a specific file
|
||||
subminer # Browse the current directory (fzf picker)
|
||||
subminer -R # Browse with the rofi picker instead
|
||||
subminer -d ~/Anime -r # Browse a specific directory, recursively
|
||||
subminer -H # Browse watch history, then replay/next/previous
|
||||
# Browse and play videos
|
||||
subminer # Current directory (uses fzf)
|
||||
subminer -R # Use rofi instead of fzf
|
||||
subminer -d ~/Videos # Specific directory
|
||||
subminer -r -d ~/Anime # Recursive search
|
||||
subminer video.mkv # Play specific file (overlay auto-starts)
|
||||
subminer --start video.mkv # Explicit overlay start (use when mpv.autoStartSubMiner is false in config)
|
||||
subminer -S video.mkv # Also force the visible overlay on start (--start-overlay)
|
||||
subminer https://youtu.be/... # Play a YouTube URL
|
||||
subminer stats # Open the immersion stats dashboard
|
||||
subminer doctor # Check dependencies, config, and the mpv socket
|
||||
subminer settings # Open the SubMiner settings window
|
||||
subminer app --setup # Re-open first-run setup
|
||||
subminer -u # Check for updates
|
||||
```
|
||||
subminer ytsearch:"jp news" # Play first YouTube search result
|
||||
subminer -H # Browse history, then choose previous/replay/next after playback
|
||||
subminer app --setup # Open first-run setup popup
|
||||
subminer --version # Print the launcher's version
|
||||
subminer -v # Same as above
|
||||
subminer --log-level debug video.mkv # Enable verbose logs for launch/debugging
|
||||
subminer --log-level warn video.mkv # Set logging level explicitly
|
||||
subminer --args '--fs=opengl-hq --ytdl-format=bestvideo*+bestaudio/best' video.mkv # Pass extra mpv args
|
||||
|
||||
On **Windows** there is no `subminer` launcher. Use the **SubMiner mpv** shortcut for playback (see [Windows mpv Shortcut](#windows-mpv-shortcut)), and run `SubMiner.exe` directly for everything else.
|
||||
|
||||
Two flags are worth knowing early:
|
||||
|
||||
- `-a/--args` passes extra arguments straight to mpv, for example `subminer --args "--ao=alsa --volume=80" video.mkv`.
|
||||
- `--log-level debug` turns on verbose logging when something is not working.
|
||||
|
||||
<details>
|
||||
<summary><b>Less common launcher commands</b></summary>
|
||||
|
||||
```bash
|
||||
subminer --start video.mkv # Explicit overlay start (when mpv.autoStartSubMiner is false)
|
||||
subminer -S video.mkv # Also force the visible overlay on start
|
||||
subminer -T video.mkv # Disable the texthooker server
|
||||
subminer -b x11 video.mkv # Force a window backend
|
||||
# Options
|
||||
subminer -T video.mkv # Disable texthooker server
|
||||
subminer -b x11 video.mkv # Force X11 backend
|
||||
subminer video.mkv # No mpv profile passed by default
|
||||
subminer -p gpu-hq video.mkv # Use a specific mpv profile
|
||||
subminer ytsearch:"jp news" # Play the first YouTube search result
|
||||
subminer texthooker # Texthooker-only mode (-o also opens the browser)
|
||||
subminer jellyfin # Open Jellyfin setup window (subcommand form)
|
||||
subminer jellyfin -l --server http://127.0.0.1:8096 --username me --password 'secret'
|
||||
subminer jellyfin --logout # Clear stored Jellyfin token/session data
|
||||
subminer jellyfin -p # Interactive Jellyfin library/item picker + playback
|
||||
subminer jellyfin -d # Jellyfin cast-discovery mode (background tray app)
|
||||
subminer app --stop # Stop background app (including Jellyfin cast broadcast)
|
||||
subminer doctor # Dependency + config + socket diagnostics
|
||||
subminer logs -e # Export a sanitized log ZIP and print its path
|
||||
subminer config path # Print active config path
|
||||
subminer config show # Print active config contents
|
||||
subminer mpv socket # Print active mpv socket path
|
||||
subminer mpv status # Exit 0 if socket is ready, else exit 1
|
||||
subminer mpv idle # Launch detached idle mpv with SubMiner defaults
|
||||
subminer sync media-box # Sync stats/watch history with an SSH host
|
||||
subminer sync media-box --push # Merge this machine's stats into the host only
|
||||
subminer sync media-box --pull # Merge the host's stats into this machine only
|
||||
subminer sync media-box --check # Verify SSH and remote SubMiner without syncing
|
||||
subminer sync media-box --json # Emit machine-readable NDJSON progress
|
||||
subminer sync --ui # Open the Sync Stats & History window
|
||||
subminer sync --snapshot ~/subminer-snapshot.sqlite # Write a local DB snapshot
|
||||
subminer sync --merge ~/subminer-snapshot.sqlite # Merge a snapshot into the local DB
|
||||
subminer sync --make-temp # Create an internal sync temp directory
|
||||
subminer sync --remove-temp /tmp/subminer-sync-123 # Remove an internal sync temp directory
|
||||
subminer dictionary /path/to/file-or-directory # Generate character dictionary ZIP from target (manual Yomitan import)
|
||||
subminer dictionary --candidates /path/to/file.mkv
|
||||
subminer dictionary --select 21355 /path/to/file.mkv
|
||||
subminer texthooker # Launch texthooker-only mode
|
||||
subminer texthooker -o # Launch texthooker and open it in your browser
|
||||
subminer stats # Start the local stats server (see Immersion Tracking)
|
||||
subminer stats -b # Start/reuse the background stats daemon
|
||||
subminer stats -s # Stop the background stats daemon
|
||||
subminer stats cleanup # Backfill vocabulary metadata, prune stale rows
|
||||
subminer stats rebuild # Rebuild rollup data
|
||||
subminer doctor --refresh-known-words # Refresh the known-word cache
|
||||
subminer logs -e # Export a sanitized log ZIP and print its path
|
||||
subminer config path # Print the active config path
|
||||
subminer config show # Print the active config contents
|
||||
subminer mpv socket # Print the active mpv socket path
|
||||
subminer mpv status # Exit 0 if the socket is ready, else exit 1
|
||||
subminer mpv idle # Launch a detached idle mpv with SubMiner defaults
|
||||
subminer app --stop # Stop the background app
|
||||
subminer --version # Print the launcher's version
|
||||
```
|
||||
subminer app --anilist-setup # Pass args directly to SubMiner binary (example: AniList login flow)
|
||||
|
||||
Jellyfin, cross-machine sync, and character-dictionary commands have their own sections: [Jellyfin](/jellyfin-integration), [Sync Between Machines](/launcher-script#sync-between-machines), and [Character Dictionary](/character-dictionary).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Direct packaged-app flags (advanced)</b></summary>
|
||||
|
||||
These call the app binary directly rather than going through the launcher. On Windows, replace `SubMiner.AppImage` with `SubMiner.exe`.
|
||||
|
||||
```bash
|
||||
SubMiner.AppImage --background # Start in background (tray + IPC wait, minimal logs)
|
||||
SubMiner.AppImage --start --texthooker # Start overlay with texthooker
|
||||
SubMiner.AppImage --texthooker # Texthooker only (no overlay window)
|
||||
SubMiner.AppImage --setup # Open first-run setup
|
||||
# Direct packaged app control
|
||||
SubMiner.AppImage --background # Start in background (tray + IPC wait, minimal logs)
|
||||
SubMiner.AppImage --start --texthooker # Start overlay with texthooker
|
||||
SubMiner.AppImage --texthooker # Launch texthooker only (no overlay window)
|
||||
SubMiner.AppImage --texthooker --open-browser # Launch texthooker and open browser
|
||||
SubMiner.AppImage --setup # Open first-run setup popup
|
||||
SubMiner.AppImage --stop # Stop overlay
|
||||
SubMiner.AppImage --start --toggle # Start mpv IPC + toggle visibility
|
||||
SubMiner.AppImage --show-visible-overlay # Force show the visible overlay
|
||||
SubMiner.AppImage --hide-visible-overlay # Force hide the visible overlay
|
||||
SubMiner.AppImage --toggle-primary-subtitle-bar # Toggle the primary subtitle bar
|
||||
SubMiner.AppImage --toggle-subtitle-sidebar # Toggle the subtitle sidebar
|
||||
SubMiner.AppImage --open-tsukihime # Open TsukiHime subtitle search
|
||||
SubMiner.AppImage --start --toggle # Start MPV IPC + toggle visibility
|
||||
SubMiner.AppImage --show-visible-overlay # Force show visible overlay
|
||||
SubMiner.AppImage --hide-visible-overlay # Force hide visible overlay
|
||||
SubMiner.AppImage --toggle-primary-subtitle-bar # Toggle primary subtitle bar visibility
|
||||
SubMiner.AppImage --toggle-subtitle-sidebar # Toggle the subtitle sidebar
|
||||
SubMiner.AppImage --open-tsukihime # Open TsukiHime subtitle search
|
||||
SubMiner.AppImage --start --dev # Enable app/dev mode only
|
||||
SubMiner.AppImage --start --debug # Alias for --dev
|
||||
SubMiner.AppImage --start --log-level debug # Force verbose logging without app/dev mode
|
||||
SubMiner.AppImage --playback-feedback "your feedback" # Route playback feedback through the configured feedback surface
|
||||
SubMiner.AppImage --yomitan # Open Yomitan settings
|
||||
SubMiner.AppImage --settings # Open the SubMiner settings window
|
||||
SubMiner.AppImage --jellyfin # Open the Jellyfin setup window
|
||||
SubMiner.AppImage --dictionary # Generate a character dictionary ZIP
|
||||
SubMiner.AppImage --start --dev # Enable app/dev mode
|
||||
SubMiner.AppImage --start --log-level debug # Verbose logging without dev mode
|
||||
SubMiner.AppImage --settings # Open SubMiner settings window
|
||||
SubMiner.AppImage --jellyfin # Open Jellyfin setup window
|
||||
SubMiner.AppImage --jellyfin-login --jellyfin-server http://127.0.0.1:8096 --jellyfin-username me --jellyfin-password 'secret'
|
||||
SubMiner.AppImage --jellyfin-logout # Clear stored Jellyfin token/session data
|
||||
SubMiner.AppImage --jellyfin-libraries
|
||||
SubMiner.AppImage --jellyfin-items --jellyfin-library-id LIBRARY_ID --jellyfin-search anime --jellyfin-limit 20
|
||||
SubMiner.AppImage --jellyfin-play --jellyfin-item-id ITEM_ID --jellyfin-audio-stream-index 1 --jellyfin-subtitle-stream-index 2 # Requires connected mpv IPC (--start)
|
||||
SubMiner.AppImage --jellyfin-remote-announce # Force cast-target capability announce + visibility check
|
||||
SubMiner.AppImage --sync-cli --help # Show the packaged app's headless sync help
|
||||
SubMiner.AppImage --sync-cli sync media-box # Run the sync engine directly in headless mode
|
||||
SubMiner.AppImage --dictionary # Generate character dictionary ZIP for current anime
|
||||
SubMiner.AppImage --dictionary-candidates # List AniList candidates for current character dictionary series
|
||||
SubMiner.AppImage --dictionary-select --dictionary-anilist-id 21355 # Pin correct AniList media for series
|
||||
SubMiner.AppImage --help # Show all options
|
||||
```
|
||||
|
||||
The remaining flags are internal or scripting-only surfaces: the `--jellyfin-*` family (login, library listing, item playback, cast announce), `--sync-cli` (the app's headless sync entrypoint that `subminer sync` proxies to), `--dictionary-candidates` / `--dictionary-select`, and `--playback-feedback <text>`. Run `SubMiner.AppImage --help` for the complete list. The previous `--open-animetosho` flag is still accepted as a deprecated alias for `--open-tsukihime`.
|
||||
`--check` performs connection and version checks without changing data. `--json` emits the NDJSON event protocol used by the sync window. `--ui` opens that window in a detached app process and returns the shell immediately; closing a standalone-launched Sync window exits that app instance. `--make-temp` and `--remove-temp` are internal remote-transfer helpers and should normally be left to SubMiner. The packaged app's `--sync-cli` flag selects its headless sync-compatible entrypoint; the `subminer sync` launcher command proxies to it automatically.
|
||||
|
||||
</details>
|
||||
The previous `--open-animetosho` flag remains accepted as a deprecated alias for `--open-tsukihime`.
|
||||
|
||||
The tray menu includes `Export Logs`, which creates the same sanitized local-date log ZIP as `subminer logs -e` and shows the archive path when complete. Export sanitization masks common PII and secrets, including home-directory usernames, IP addresses, emails, auth/cookie headers, yt-dlp cookie arguments, URL credentials, token/key/password fields, and signed YouTube media URL query strings. The exported copy is sanitized; source log files remain unredacted on disk.
|
||||
|
||||
@@ -181,11 +217,18 @@ This flow requires `mpv.exe` to be discoverable. Leave `mpv.executablePath` blan
|
||||
|
||||
### Launcher Subcommands
|
||||
|
||||
The launcher groups related work under subcommands: `jellyfin` (aliased `jf`), `stats`, `sync`, `dictionary` (aliased `dict`), `texthooker`, `doctor`, `settings`, `config`, `mpv`, `logs`, and `app` (aliased `bin`) for passing arguments straight to the SubMiner binary.
|
||||
|
||||
Every subcommand has its own help page, for example `subminer jellyfin -h`. See [Launcher Script - Subcommands](/launcher-script#subcommands) for the full table, and [Sync Between Machines](/launcher-script#sync-between-machines) for the SSH stats/history sync.
|
||||
|
||||
A _texthooker_ is a web page that displays the current subtitle line as selectable text, so browser-based dictionary extensions and other tools can read along with playback.
|
||||
- `subminer jellyfin` / `subminer jf`: Jellyfin-focused workflow aliases.
|
||||
- `subminer doctor`: health checks for core dependencies and runtime paths.
|
||||
- `subminer settings`: open the SubMiner settings window (also `subminer --settings`).
|
||||
- `subminer logs -e`: export a sanitized ZIP of today's local-date logs, or the most recent logs when no current-day log exists. The exported copy masks common PII and secrets; on-disk logs are unchanged.
|
||||
- `subminer config`: config file helpers (`path`, `show`).
|
||||
- `subminer mpv`: mpv helpers (`status`, `socket`, `idle`).
|
||||
- `subminer sync <host>`: sync immersion stats and watch history with another machine over SSH. The host is the SSH destination (`user@host` or an SSH config alias). Use `--push` to merge only this machine's data into the host, or `--pull` to merge only the host's data into this machine; both remain insert-only and do not make either database an exact mirror. Remote launcher checks include standard SubMiner and Bun paths even when SSH omits them from `PATH`. Use `--snapshot <file>` to write a consistent local stats DB snapshot, `--merge <file>` to merge a snapshot into the local stats DB, and `--force` to skip the running stats/mpv safety check. Advanced options: `--db <file>` overrides the local stats DB path, and `--remote-cmd <cmd>` overrides the `subminer` command used on the remote host.
|
||||
- `subminer dictionary <path>`: generates a Yomitan-importable character dictionary ZIP from a file/directory target.
|
||||
- Use `subminer dictionary --candidates <path>` and `subminer dictionary --select <id> <path>` to correct AniList character-dictionary matches for a whole series.
|
||||
- `subminer texthooker`: texthooker-only shortcut (same behavior as `--texthooker`). A _texthooker_ is a web page that displays the current subtitle line as selectable text, so browser-based dictionary extensions and other tools can read along with playback.
|
||||
- `subminer app` / `subminer bin`: direct passthrough to the SubMiner binary/AppImage.
|
||||
- Subcommand help pages are available (for example `subminer jellyfin -h`).
|
||||
|
||||
### First-Run Setup
|
||||
|
||||
@@ -282,28 +325,6 @@ Notes:
|
||||
|
||||
For local video files, SubMiner uses the same config-driven language priorities to auto-select the primary and secondary subtitle tracks from internal and external subtitle sources.
|
||||
|
||||
## Live Config Reload
|
||||
|
||||
While SubMiner is running, it watches your active config file and applies safe updates automatically.
|
||||
|
||||
Live-updated settings include:
|
||||
|
||||
- `subtitleStyle`
|
||||
- `keybindings`
|
||||
- `shortcuts`
|
||||
- `secondarySub.defaultMode`
|
||||
- `subtitleSidebar`
|
||||
- `notifications`
|
||||
- `logging`
|
||||
- `jimaku`, `subsync`
|
||||
- `mpv.aniskipEnabled`, `mpv.aniskipButtonKey`
|
||||
- `stats.toggleKey`, `stats.markWatchedKey`
|
||||
- `youtube.primarySubLanguages`
|
||||
- most `ankiConnect.*` settings (including `ankiConnect.ai`)
|
||||
|
||||
Invalid config edits are rejected; SubMiner keeps the previous valid runtime config and shows an error notification.
|
||||
For restart-required sections, SubMiner shows a restart-needed notification.
|
||||
|
||||
## Controller Support
|
||||
|
||||
SubMiner supports gamepad/controller input for couch-friendly usage via the Chrome Gamepad API. Controller input drives the overlay while keyboard-only mode is enabled.
|
||||
|
||||
@@ -377,4 +377,4 @@ ws.on('message', async (raw) => {
|
||||
- [Mining Workflow - Texthooker](/mining-workflow#texthooker)
|
||||
- [MPV Plugin](/mpv-plugin)
|
||||
- [Launcher Script](/launcher-script)
|
||||
- [Anki Integration](/anki-integration#proxy-mode-setup-yomitan-texthooker)
|
||||
- [Anki Integration](/anki-integration#proxy-mode-setup-yomitan--texthooker)
|
||||
|
||||
@@ -153,6 +153,6 @@ These settings come from `config.jsonc` (or built-in defaults); there are no CLI
|
||||
|
||||
- [Usage --- YouTube Playback](/usage#youtube-playback)
|
||||
- [Configuration --- YouTube Playback Settings](/configuration#youtube-playback-settings)
|
||||
- [Configuration --- Secondary Subtitles](/configuration#secondary-subtitles)
|
||||
- [Configuration --- Secondary Subtitle](/configuration#secondary-subtitle)
|
||||
- [Keyboard Shortcuts](/shortcuts)
|
||||
- [Jellyfin Integration](/jellyfin-integration)
|
||||
|
||||
@@ -185,7 +185,6 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -194,7 +193,6 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -315,8 +313,8 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
CREATE INDEX idx_subtitle_lines_session_line ON imm_subtitle_lines(session_id, line_index);
|
||||
CREATE INDEX idx_subtitle_lines_video_line ON imm_subtitle_lines(video_id, line_index);
|
||||
CREATE INDEX idx_subtitle_lines_anime_line ON imm_subtitle_lines(anime_id, line_index);
|
||||
CREATE INDEX idx_word_line_occurrences_word_seen ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id);
|
||||
CREATE INDEX idx_kanji_line_occurrences_kanji_seen ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id);
|
||||
CREATE INDEX idx_word_line_occurrences_word ON imm_word_line_occurrences(word_id, line_id);
|
||||
CREATE INDEX idx_kanji_line_occurrences_kanji ON imm_kanji_line_occurrences(kanji_id, line_id);
|
||||
CREATE INDEX idx_media_art_cover_blob_hash ON imm_media_art(cover_blob_hash);
|
||||
CREATE INDEX idx_media_art_anilist_id ON imm_media_art(anilist_id);
|
||||
CREATE INDEX idx_media_art_cover_url ON imm_media_art(cover_url);
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
"name": "subminer",
|
||||
"productName": "SubMiner",
|
||||
"desktopName": "SubMiner.desktop",
|
||||
"version": "0.19.0",
|
||||
"version": "0.19.0-beta.3",
|
||||
"description": "All-in-one sentence mining overlay with AnkiConnect and dictionary integration",
|
||||
"packageManager": "bun@1.3.5",
|
||||
"main": "dist/main-entry.js",
|
||||
|
||||
@@ -17,17 +17,6 @@
|
||||
- **Post-Playback History Menu**
|
||||
- After a watch-history episode ends or mpv closes, the fzf/rofi launcher returns to that series with options to play the previous or next episode, rewatch, pick another episode, or quit SubMiner.
|
||||
- Previous/Next continue across season directories, so you can binge a show without manually browsing folders.
|
||||
- The menu shown right after picking a series from `subminer -H` now offers the previous episode too, matching the post-playback menu.
|
||||
|
||||
- **Known-Word Highlighting by Anki Maturity**
|
||||
- Subtitle highlights for known words can now be colored by Anki card maturity (new, learning, young, mature), similar to asbplayer. Enable it with `ankiConnect.knownWords.maturityEnabled`, or toggle it live during a session.
|
||||
- The mature-interval threshold and the four tier colors are configurable, and the in-session help legend shows the active tier colors while maturity highlighting is on.
|
||||
- Tiers follow Anki's own card state: a lapsed card correctly shows as learning rather than young, and a note is treated as mature if any of its cards are mature. Stats and other known-word tools stay accurate with this new data.
|
||||
|
||||
- **Stats Library Entry Deletion**
|
||||
- Added a "Delete Entry" action in the stats Library detail view that removes an entire title in one step: every episode, session, subtitle line, rollup, cover, and vocabulary count derived from it. Previously a mistaken entry had to be cleared episode by episode and still lingered in the Library.
|
||||
- Delete progress (session, session group, episode, or full entry) now shows app-wide as a progress bar plus a status toast, staying visible across tabs and windows instead of disappearing when you switch away.
|
||||
- Deletes are dramatically faster on large libraries, and opening the Vocabulary tab no longer stalls; the first launch after upgrading migrates the stats database in place to support this.
|
||||
|
||||
### Changed
|
||||
|
||||
@@ -39,10 +28,6 @@
|
||||
- **Word Highlighting Accuracy**
|
||||
- Fixed several incorrect word highlighting and annotation cases: inconsistent part-of-speech exclusions on merged quote-particle tokens, missing annotations for rare kanji, katakana punctuation wrongly treated as non-kana noise, and certain kanji vocabulary skipped for next-level ("N+1") highlighting.
|
||||
|
||||
- **AniList Season Resolution**
|
||||
- Season 2 and later episodes now resolve to the correct AniList entry by walking sequel relations instead of guessing from the title, so watch progress, the character dictionary, and cover art for later seasons no longer silently fall back to season 1.
|
||||
- Manual AniList overrides now stay in effect for every episode in the same season (by folder and detected season), and setting an override now fixes both the character dictionary and AniList watch progress together instead of needing separate corrections.
|
||||
|
||||
- **Startup Playback Pausing Too Early**
|
||||
- Fixed playback resuming before subtitle processing finished warming up, which could briefly show untranslated subtitles right after opening a video.
|
||||
- Most noticeable when resuming mid-episode or when a subtitle cue starts within the first couple of seconds.
|
||||
@@ -55,16 +40,9 @@
|
||||
- Fixed video playback failing to start when another process already held the configured AnkiConnect proxy port; SubMiner now shows a notification explaining how to resolve the conflict instead of crashing.
|
||||
|
||||
- **Stats & Settings Reliability**
|
||||
- Fixed session stats reporting zero known words after the known-word cache gained maturity tiers.
|
||||
- Hardened the stats server against malformed requests, stalled AniList lookups, media mismatches during word mining, and missing Yomitan connections.
|
||||
- AnkiConnect settings validation now preserves valid custom configurations while safely falling back on invalid values instead of failing.
|
||||
|
||||
- **Stats Library Cover After Relink**
|
||||
- Relinking a title to a different AniList entry now updates its cover art in the stats Library grid, not just the detail view, so unrelated titles no longer end up sharing the wrong cover.
|
||||
|
||||
- **Rofi Menu Prompt Spacing**
|
||||
- Rofi menu prompts now keep a space between the prompt label and the input field instead of crowding the search placeholder text.
|
||||
|
||||
## What's Changed
|
||||
|
||||
- feat(shortcuts): make clipboard-video-append shortcut configurable by @ksyasuda in #158
|
||||
@@ -72,9 +50,6 @@
|
||||
- refactor(tsukihime): swap Animetosho backend for TsukiHime API by @ksyasuda in #165
|
||||
- refactor: split anki-connect and stats-server resolvers into modules by @ksyasuda in #169
|
||||
- feat(launcher): add post-playback history menu with previous episode by @ksyasuda in #170
|
||||
- Anki maturity-based known-word highlighting by @ksyasuda in #172
|
||||
- fix(anilist): resolve later seasons via sequel relations, not title guessing by @ksyasuda in #173
|
||||
- feat(stats): add library entry deletion and app-wide delete progress by @ksyasuda in #174
|
||||
|
||||
## Installation
|
||||
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
## Highlights
|
||||
### Added
|
||||
- **Anki Maturity Known-Word Highlighting**
|
||||
- Subtitle words you already know can now be color-coded by their Anki card maturity (new, learning, young, mature), like asbplayer's known-word coloring.
|
||||
- Enable it with `ankiConnect.knownWords.maturityEnabled` (or toggle it mid-session); tier colors and the "mature" day threshold are configurable, and the in-session help legend shows the active colors.
|
||||
- **Cross-Machine Sync for Stats & Watch History**
|
||||
- Sync immersion stats and watch history between machines over SSH from a new Sync window (tray menu → Sync Stats & History, or `subminer sync --ui`) or the CLI (`subminer sync <host>`).
|
||||
- Save multiple devices with per-host sync direction, run one-click syncs with live progress, and take manual database snapshots for backup or transfer.
|
||||
- Windows remotes are supported over OpenSSH, and hosts can auto-sync in the background on a schedule, even during playback.
|
||||
- **History Menu After Playback**
|
||||
- After a watch-history episode ends (or mpv closes), the fzf/rofi launcher now offers to play the previous or next episode, rewatch, pick another episode, or quit, right from where you left off. Previous/Next continue across season folders.
|
||||
- **Delete Entire Library Titles from Stats**
|
||||
- The stats Library detail view now has a "Delete Entry" action that removes a whole title in one step, episodes, sessions, subtitle lines, rollups, cover art, and vocabulary counts, instead of clearing it episode by episode.
|
||||
- Delete progress (sessions, episodes, or whole titles) now shows app-wide with a progress bar and status toast visible from any tab or window.
|
||||
- **TsukiHime English Subtitle Downloads**
|
||||
- Download subtitles for the currently playing video directly from TsukiHime, with Japanese loaded as the primary track and your configured secondary language alongside it.
|
||||
|
||||
### Changed
|
||||
- **Configurable Clipboard-Video Shortcut**
|
||||
- The "append clipboard video to queue" shortcut is now configurable via `shortcuts.appendClipboardVideoToQueue` instead of fixed.
|
||||
|
||||
### Fixed
|
||||
- **AniList Season Matching**
|
||||
- Season 2+ episodes now resolve to the correct AniList entry instead of silently falling back to season 1. SubMiner follows AniList's sequel relations to find the right season, and cover art and watch progress now use the same season-aware match.
|
||||
- If a season still can't be found, SubMiner no longer force-writes progress or a cover to the season 1 entry, it skips the update and points you to a manual AniList override, which now fixes the character dictionary and watch progress together.
|
||||
- Manual overrides now stay applied consistently across every episode in a season folder, even when filenames guess differently episode to episode.
|
||||
- **Subtitle Highlighting Accuracy**
|
||||
- Fixed several known-word/annotation edge cases: part-of-speech exclusions now apply consistently to merged quote-particle tokens, annotations for rarer kanji are preserved, katakana punctuation is no longer mistaken for plain kana, and a specific noun-tagging case no longer loses its known+1 highlight.
|
||||
- **AnkiConnect Proxy Port Conflicts**
|
||||
- Fixed a crash on video startup when another process already held the configured AnkiConnect proxy port; you'll now get a notification explaining how to resolve it instead.
|
||||
- **AppImage Crash Notification on Quit**
|
||||
- Fixed a spurious "Service Crash" desktop notification appearing after closing a video when running the Linux AppImage.
|
||||
- **Startup Playback Pause Timing**
|
||||
- Fixed playback occasionally resuming a couple seconds before subtitle tokenization actually finished warming up, most noticeable when resuming mid-episode or when a subtitle appears in the first two seconds.
|
||||
- **Stats Library Cover After Relinking**
|
||||
- Fixed the stats Library grid showing a stale cover image after relinking a title to a different AniList entry.
|
||||
- **Faster Stats Deletes and Vocabulary Tab**
|
||||
- Deleting sessions, episodes, and titles from stats is now dramatically faster and no longer stalls playback while it runs; the Vocabulary tab also loads much faster.
|
||||
- The first launch after updating runs a one-time database migration (a few seconds, database grows about 20%); no action needed.
|
||||
- **Settings Validation and Stats Server Hardening**
|
||||
- Invalid AnkiConnect settings now fall back safely with a warning instead of silently breaking, and the stats server is hardened against malformed requests, stalled AniList searches, and other edge cases that could previously crash it.
|
||||
- **Rofi Prompt Spacing**
|
||||
- Fixed rofi menu prompts running into the search placeholder text with no space between them.
|
||||
|
||||
## What's Changed
|
||||
|
||||
- feat(shortcuts): make clipboard-video-append shortcut configurable by @ksyasuda in #158
|
||||
- refactor(tokenizer): extract subtitle annotation filter into rule table by @ksyasuda in #162
|
||||
- refactor(tsukihime): swap Animetosho backend for TsukiHime API by @ksyasuda in #165
|
||||
- refactor: split anki-connect and stats-server resolvers into modules by @ksyasuda in #169
|
||||
- feat(launcher): add post-playback history menu with previous episode by @ksyasuda in #170
|
||||
- Anki maturity-based known-word highlighting by @ksyasuda in #172
|
||||
- fix(anilist): resolve later seasons via sequel relations, not title guessing by @ksyasuda in #173
|
||||
- feat(stats): add library entry deletion and app-wide delete progress by @ksyasuda in #174
|
||||
|
||||
## Installation
|
||||
|
||||
See the README and docs/installation guide for full setup steps.
|
||||
|
||||
## Assets
|
||||
|
||||
- Linux: `SubMiner.AppImage`
|
||||
- macOS: `SubMiner-*.dmg` and `SubMiner-*.zip`
|
||||
- Windows: `SubMiner-*.exe` and `SubMiner-*-win.zip`
|
||||
- Optional extras: `subminer-assets.tar.gz` and the `subminer` launcher
|
||||
|
||||
Note: the `subminer` wrapper script uses Bun (`#!/usr/bin/env bun`), so `bun` must be installed and on `PATH`.
|
||||
@@ -224,19 +224,13 @@ function readPersistedCacheScope(cachePath: string): string | null {
|
||||
}
|
||||
}
|
||||
|
||||
const ANKI_REQUEST_TIMEOUT_MS = 30_000;
|
||||
|
||||
function createAnkiClient(url: string) {
|
||||
const request = async (action: string, params: unknown): Promise<unknown> => {
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ action, version: 6, params }),
|
||||
signal: AbortSignal.timeout(ANKI_REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`AnkiConnect ${action}: HTTP ${response.status} ${response.statusText}`);
|
||||
}
|
||||
const payload = (await response.json()) as { result: unknown; error: string | null };
|
||||
if (payload.error) {
|
||||
throw new Error(`AnkiConnect ${action}: ${payload.error}`);
|
||||
|
||||
@@ -152,6 +152,5 @@ export async function createYomitanRuntimeStateWithSearch(
|
||||
}
|
||||
}
|
||||
|
||||
// No usable manifest at the resolved path, so let the loader search on its own.
|
||||
return createYomitanRuntimeState(userDataPath);
|
||||
return createYomitanRuntimeState(userDataPath, resolvedExtensionPath ?? undefined);
|
||||
}
|
||||
|
||||
+74
-7
@@ -22,12 +22,10 @@ import { MediaGenerator } from './media-generator';
|
||||
import path from 'path';
|
||||
import {
|
||||
AnkiConnectConfig,
|
||||
type CardKind,
|
||||
KikuDuplicateCardInfo,
|
||||
KikuFieldGroupingChoice,
|
||||
KikuMergePreviewResponse,
|
||||
NotificationOptions,
|
||||
type WordCardKind,
|
||||
} from './types/anki';
|
||||
import { AiConfig } from './types/integrations';
|
||||
import type { KnownWordMaturityTier } from './types/subtitle';
|
||||
@@ -52,7 +50,6 @@ import {
|
||||
withUpdateProgress,
|
||||
UiFeedbackState,
|
||||
} from './anki-integration/ui-feedback';
|
||||
import { applyCardKindFlagFields, resolveWordCardKindSetting } from './anki-integration/card-kinds';
|
||||
import { KnownWordCacheManager } from './anki-integration/known-word-cache';
|
||||
import { PollingRunner } from './anki-integration/polling';
|
||||
import type { AnkiConnectProxyServer } from './anki-integration/anki-connect-proxy';
|
||||
@@ -86,6 +83,8 @@ interface NoteInfo {
|
||||
fields: Record<string, { value: string }>;
|
||||
}
|
||||
|
||||
type CardKind = 'sentence' | 'audio' | 'word-and-sentence';
|
||||
|
||||
function trimToNonEmptyString(value: unknown): string | null {
|
||||
if (typeof value !== 'string') return null;
|
||||
const trimmed = value.trim();
|
||||
@@ -841,7 +840,6 @@ export class AnkiIntegration {
|
||||
kikuEnabled: boolean;
|
||||
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
|
||||
kikuDeleteDuplicateInAuto: boolean;
|
||||
wordCardKind: WordCardKind;
|
||||
} {
|
||||
const lapis = this.getLapisConfig();
|
||||
const kiku = this.getKikuConfig();
|
||||
@@ -854,7 +852,6 @@ export class AnkiIntegration {
|
||||
kikuEnabled: kiku.enabled,
|
||||
kikuFieldGrouping: (kiku.fieldGrouping || 'disabled') as 'auto' | 'manual' | 'disabled',
|
||||
kikuDeleteDuplicateInAuto: kiku.deleteDuplicateInAuto !== false,
|
||||
wordCardKind: resolveWordCardKindSetting(this.config.lapisKiku?.wordCardKind),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1318,9 +1315,79 @@ export class AnkiIntegration {
|
||||
availableFieldNames: string[],
|
||||
cardKind: CardKind,
|
||||
): void {
|
||||
applyCardKindFlagFields(updatedFields, cardKind, (preferredName) =>
|
||||
this.resolveFieldName(availableFieldNames, preferredName),
|
||||
const audioFlagNames = ['IsAudioCard'];
|
||||
|
||||
if (cardKind === 'word-and-sentence') {
|
||||
const wordAndSentenceFlag = this.resolveFieldName(
|
||||
availableFieldNames,
|
||||
'IsWordAndSentenceCard',
|
||||
);
|
||||
if (!wordAndSentenceFlag) {
|
||||
return;
|
||||
}
|
||||
updatedFields[wordAndSentenceFlag] = 'x';
|
||||
|
||||
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
|
||||
if (sentenceFlag && sentenceFlag !== wordAndSentenceFlag) {
|
||||
updatedFields[sentenceFlag] = '';
|
||||
}
|
||||
|
||||
for (const audioFlagName of audioFlagNames) {
|
||||
const resolved = this.resolveFieldName(availableFieldNames, audioFlagName);
|
||||
if (resolved && resolved !== wordAndSentenceFlag) {
|
||||
updatedFields[resolved] = '';
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (cardKind === 'sentence') {
|
||||
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
|
||||
if (sentenceFlag) {
|
||||
updatedFields[sentenceFlag] = 'x';
|
||||
}
|
||||
|
||||
for (const audioFlagName of audioFlagNames) {
|
||||
const resolved = this.resolveFieldName(availableFieldNames, audioFlagName);
|
||||
if (resolved && resolved !== sentenceFlag) {
|
||||
updatedFields[resolved] = '';
|
||||
}
|
||||
}
|
||||
|
||||
const wordAndSentenceFlag = this.resolveFieldName(
|
||||
availableFieldNames,
|
||||
'IsWordAndSentenceCard',
|
||||
);
|
||||
if (wordAndSentenceFlag && wordAndSentenceFlag !== sentenceFlag) {
|
||||
updatedFields[wordAndSentenceFlag] = '';
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const resolvedAudioFlags = Array.from(
|
||||
new Set(
|
||||
audioFlagNames
|
||||
.map((name) => this.resolveFieldName(availableFieldNames, name))
|
||||
.filter((name): name is string => Boolean(name)),
|
||||
),
|
||||
);
|
||||
const audioFlagName = resolvedAudioFlags[0] || null;
|
||||
if (audioFlagName) {
|
||||
updatedFields[audioFlagName] = 'x';
|
||||
}
|
||||
for (const extraAudioFlag of resolvedAudioFlags.slice(1)) {
|
||||
updatedFields[extraAudioFlag] = '';
|
||||
}
|
||||
|
||||
const sentenceFlag = this.resolveFieldName(availableFieldNames, 'IsSentenceCard');
|
||||
if (sentenceFlag && sentenceFlag !== audioFlagName) {
|
||||
updatedFields[sentenceFlag] = '';
|
||||
}
|
||||
|
||||
const wordAndSentenceFlag = this.resolveFieldName(availableFieldNames, 'IsWordAndSentenceCard');
|
||||
if (wordAndSentenceFlag && wordAndSentenceFlag !== audioFlagName) {
|
||||
updatedFields[wordAndSentenceFlag] = '';
|
||||
}
|
||||
}
|
||||
|
||||
private async showNotification(
|
||||
|
||||
@@ -4,23 +4,29 @@ import test from 'node:test';
|
||||
import { CardCreationService } from './card-creation';
|
||||
import { toMpvEdlValue } from './mpv-edl-test-utils';
|
||||
import type { MediaInput } from '../media-generator';
|
||||
import type { AnkiConnectConfig, CardKind } from '../types/anki';
|
||||
import { applyCardKindFlagFields } from './card-kinds';
|
||||
import type { AnkiConnectConfig } from '../types/anki';
|
||||
|
||||
type CardCreationDeps = ConstructorParameters<typeof CardCreationService>[0];
|
||||
|
||||
function setCardTypeFields(
|
||||
function setWordAndSentenceCardTypeFields(
|
||||
updatedFields: Record<string, string>,
|
||||
availableFieldNames: string[],
|
||||
cardKind: CardKind,
|
||||
cardKind: 'sentence' | 'audio' | 'word-and-sentence',
|
||||
): void {
|
||||
applyCardKindFlagFields(
|
||||
updatedFields,
|
||||
cardKind,
|
||||
(preferredName) =>
|
||||
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ??
|
||||
null,
|
||||
);
|
||||
if (cardKind !== 'word-and-sentence') return;
|
||||
|
||||
const resolveFieldName = (preferredName: string): string | null =>
|
||||
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
|
||||
const wordAndSentenceFlag = resolveFieldName('IsWordAndSentenceCard');
|
||||
if (!wordAndSentenceFlag) return;
|
||||
|
||||
updatedFields[wordAndSentenceFlag] = 'x';
|
||||
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
|
||||
const resolved = resolveFieldName(flagName);
|
||||
if (resolved && resolved !== wordAndSentenceFlag) {
|
||||
updatedFields[resolved] = '';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createManualUpdateService(overrides: Partial<CardCreationDeps> = {}): {
|
||||
@@ -211,7 +217,7 @@ test('manual clipboard subtitle update marks Kiku word cards as word-and-sentenc
|
||||
kikuFieldGrouping: 'disabled',
|
||||
kikuDeleteDuplicateInAuto: false,
|
||||
}),
|
||||
setCardTypeFields,
|
||||
setCardTypeFields: setWordAndSentenceCardTypeFields,
|
||||
});
|
||||
|
||||
await service.updateLastAddedFromClipboard('字幕');
|
||||
|
||||
@@ -3,7 +3,7 @@ import {
|
||||
getConfiguredWordFieldName,
|
||||
getPreferredWordValueFromExtractedFields,
|
||||
} from '../anki-field-config';
|
||||
import { AnkiConnectConfig, type CardKind, type WordCardKind } from '../types/anki';
|
||||
import { AnkiConnectConfig } from '../types/anki';
|
||||
import { createLogger } from '../logger';
|
||||
import type { MediaInput } from '../media-input';
|
||||
import { SubtitleTimingTracker } from '../subtitle-timing-tracker';
|
||||
@@ -15,7 +15,7 @@ import {
|
||||
resolveAudioStreamIndexForMediaGeneration,
|
||||
type MediaGenerationInputResolverOptions,
|
||||
} from './media-source';
|
||||
import { resolveWordCardKind } from './note-field-utils';
|
||||
import { shouldMarkWordAndSentenceCard } from './note-field-utils';
|
||||
import type { PendingYoutubeMediaUpdate } from './pending-youtube-media';
|
||||
import { resolveMpvVolumeScale } from './mpv-volume';
|
||||
|
||||
@@ -42,6 +42,8 @@ export interface CardCreationNoteInfo {
|
||||
fields: Record<string, { value: string }>;
|
||||
}
|
||||
|
||||
type CardKind = 'sentence' | 'audio' | 'word-and-sentence';
|
||||
|
||||
interface CardCreationClient {
|
||||
addNote(
|
||||
deck: string,
|
||||
@@ -134,7 +136,6 @@ interface CardCreationDeps {
|
||||
kikuEnabled: boolean;
|
||||
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
|
||||
kikuDeleteDuplicateInAuto: boolean;
|
||||
wordCardKind?: WordCardKind;
|
||||
};
|
||||
getFallbackDurationSeconds: () => number;
|
||||
appendKnownWordsFromNoteInfo: (noteInfo: CardCreationNoteInfo) => void;
|
||||
@@ -273,9 +274,12 @@ export class CardCreationService {
|
||||
if (sentenceField) {
|
||||
const processedSentence = this.deps.processSentence(sentence, fields);
|
||||
updatedFields[sentenceField] = processedSentence;
|
||||
const wordCardKind = resolveWordCardKind(noteInfo, sentenceCardConfig);
|
||||
if (wordCardKind) {
|
||||
this.deps.setCardTypeFields(updatedFields, Object.keys(noteInfo.fields), wordCardKind);
|
||||
if (shouldMarkWordAndSentenceCard(noteInfo, sentenceCardConfig)) {
|
||||
this.deps.setCardTypeFields(
|
||||
updatedFields,
|
||||
Object.keys(noteInfo.fields),
|
||||
'word-and-sentence',
|
||||
);
|
||||
}
|
||||
updatePerformed = true;
|
||||
}
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { applyCardKindFlagFields } from './card-kinds';
|
||||
|
||||
function resolverFor(availableFieldNames: string[]) {
|
||||
return (preferredName: string): string | null =>
|
||||
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
|
||||
}
|
||||
|
||||
const KIKU_FLAG_FIELDS = ['IsWordAndSentenceCard', 'IsClickCard', 'IsSentenceCard', 'IsAudioCard'];
|
||||
|
||||
test('flags the requested card kind and clears the others', () => {
|
||||
const fields: Record<string, string> = {};
|
||||
|
||||
applyCardKindFlagFields(fields, 'click', resolverFor(KIKU_FLAG_FIELDS));
|
||||
|
||||
assert.deepEqual(fields, {
|
||||
IsClickCard: 'x',
|
||||
IsWordAndSentenceCard: '',
|
||||
IsSentenceCard: '',
|
||||
IsAudioCard: '',
|
||||
});
|
||||
});
|
||||
|
||||
test('matches flag fields case-insensitively', () => {
|
||||
const fields: Record<string, string> = {};
|
||||
|
||||
applyCardKindFlagFields(fields, 'word-and-sentence', resolverFor(['iswordandsentencecard']));
|
||||
|
||||
assert.deepEqual(fields, { iswordandsentencecard: 'x' });
|
||||
});
|
||||
|
||||
test('leaves flags untouched when the note type has no flag for a word card kind', () => {
|
||||
const fields: Record<string, string> = {};
|
||||
|
||||
applyCardKindFlagFields(
|
||||
fields,
|
||||
'click',
|
||||
resolverFor(['IsWordAndSentenceCard', 'IsSentenceCard']),
|
||||
);
|
||||
|
||||
assert.deepEqual(fields, {});
|
||||
});
|
||||
|
||||
test('clears stale flags for explicit mine actions even without the target flag', () => {
|
||||
const fields: Record<string, string> = {};
|
||||
|
||||
applyCardKindFlagFields(
|
||||
fields,
|
||||
'audio',
|
||||
resolverFor(['IsWordAndSentenceCard', 'IsSentenceCard']),
|
||||
);
|
||||
|
||||
assert.deepEqual(fields, { IsWordAndSentenceCard: '', IsSentenceCard: '' });
|
||||
});
|
||||
|
||||
test('does not blank the target flag it just set', () => {
|
||||
const fields: Record<string, string> = {};
|
||||
|
||||
applyCardKindFlagFields(fields, 'sentence', resolverFor(['IsSentenceCard']));
|
||||
|
||||
assert.deepEqual(fields, { IsSentenceCard: 'x' });
|
||||
});
|
||||
@@ -1,63 +0,0 @@
|
||||
import type { CardKind, WordCardKind } from '../types/anki';
|
||||
|
||||
/**
|
||||
* Kiku/Lapis note types decide which card a note generates from mutually exclusive
|
||||
* `Is...Card` flag fields. Setting one always means clearing the others.
|
||||
*/
|
||||
export const CARD_KIND_FLAG_FIELDS: Record<CardKind, string> = {
|
||||
'word-and-sentence': 'IsWordAndSentenceCard',
|
||||
click: 'IsClickCard',
|
||||
sentence: 'IsSentenceCard',
|
||||
audio: 'IsAudioCard',
|
||||
};
|
||||
|
||||
export const WORD_CARD_KINDS: readonly WordCardKind[] = [
|
||||
'word-and-sentence',
|
||||
'click',
|
||||
'sentence',
|
||||
'audio',
|
||||
'none',
|
||||
];
|
||||
|
||||
export const DEFAULT_WORD_CARD_KIND: WordCardKind = 'word-and-sentence';
|
||||
|
||||
/**
|
||||
* Card kinds SubMiner marks on its own initiative (word cards). They are only applied
|
||||
* when the note type actually carries the matching flag field, so plain note types keep
|
||||
* their fields untouched.
|
||||
*/
|
||||
const IMPLICIT_CARD_KINDS = new Set<CardKind>(['word-and-sentence', 'click']);
|
||||
|
||||
export function isWordCardKind(value: unknown): value is WordCardKind {
|
||||
return typeof value === 'string' && WORD_CARD_KINDS.includes(value as WordCardKind);
|
||||
}
|
||||
|
||||
export function resolveWordCardKindSetting(value: unknown): WordCardKind {
|
||||
return isWordCardKind(value) ? value : DEFAULT_WORD_CARD_KIND;
|
||||
}
|
||||
|
||||
/**
|
||||
* Flags `cardKind` on the note and clears every other card-kind flag it has, so the note
|
||||
* never ends up claiming to be two kinds of card at once.
|
||||
*/
|
||||
export function applyCardKindFlagFields(
|
||||
updatedFields: Record<string, string>,
|
||||
cardKind: CardKind,
|
||||
resolveFieldName: (preferredName: string) => string | null,
|
||||
): void {
|
||||
const targetFlag = resolveFieldName(CARD_KIND_FLAG_FIELDS[cardKind]);
|
||||
if (!targetFlag && IMPLICIT_CARD_KINDS.has(cardKind)) {
|
||||
return;
|
||||
}
|
||||
if (targetFlag) {
|
||||
updatedFields[targetFlag] = 'x';
|
||||
}
|
||||
|
||||
for (const [kind, flagName] of Object.entries(CARD_KIND_FLAG_FIELDS)) {
|
||||
if (kind === cardKind) continue;
|
||||
const resolved = resolveFieldName(flagName);
|
||||
if (resolved && resolved !== targetFlag) {
|
||||
updatedFields[resolved] = '';
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -69,18 +69,6 @@ function createMaturityHarness(config: AnkiConnectConfig): {
|
||||
};
|
||||
}
|
||||
|
||||
// The four queries a maturity refresh issues, in one place so a query-string
|
||||
// change lands in a single spot.
|
||||
function setTierQueries(
|
||||
clientState: { findNotesByQuery: Map<string, number[]> },
|
||||
tiers: { all: number[]; mature: number[]; young: number[]; learning: number[] },
|
||||
): void {
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', tiers.all);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', tiers.mature);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', tiers.young);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', tiers.learning);
|
||||
}
|
||||
|
||||
function maturityConfig(overrides: Partial<AnkiConnectConfig> = {}): AnkiConnectConfig {
|
||||
return {
|
||||
deck: 'Mining',
|
||||
@@ -143,7 +131,10 @@ test('refresh fetches tier sets and getKnownWordTier classifies notes', async ()
|
||||
const { manager, calls, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2, 3, 4], mature: [1], young: [2], learning: [3] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2, 3, 4]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', [2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [3]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '犬' } } },
|
||||
@@ -171,7 +162,10 @@ test('a note with cards in several tiers counts as its most mature card', async
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1], mature: [1], young: [1], learning: [1] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [1]);
|
||||
clientState.notesInfoResult = [{ noteId: 1, fields: { Word: { value: '猫' } } }];
|
||||
|
||||
await manager.refresh(true);
|
||||
@@ -186,7 +180,10 @@ test('a word matched by several notes takes the most mature note tier', async ()
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [], young: [2], learning: [1] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', [2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [1]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '猫' } } },
|
||||
@@ -204,7 +201,10 @@ test('tiers are reading-aware for words with several readings', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [1], young: [], learning: [2] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [2]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '床' }, Reading: { value: 'とこ' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '床' }, Reading: { value: 'ゆか' } } },
|
||||
@@ -227,7 +227,10 @@ test('reading-only fallback resolves tiers unless opted out', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1], mature: [1], young: [], learning: [] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', []);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '警告' }, Reading: { value: 'けいこく' } } },
|
||||
];
|
||||
@@ -302,7 +305,10 @@ test('tiers persist to v4 state and reload without refetching', async () => {
|
||||
|
||||
try {
|
||||
Date.now = () => 120_000;
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [1], young: [], learning: [2] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [2]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '犬' } } },
|
||||
@@ -353,7 +359,10 @@ test('appendFromNoteInfo preserves an existing maturity tier', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [7, 8], mature: [7], young: [], learning: [8] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [7, 8]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [7]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [8]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 7, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 8, fields: { Word: { value: '犬' } } },
|
||||
@@ -380,7 +389,10 @@ test('getKnownWordMatchNoteIds reports the notes behind a tier', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2, 3], mature: [1], young: [2], learning: [3] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2, 3]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', [2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [3]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '床' }, Reading: { value: 'とこ' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '床' }, Reading: { value: 'ゆか' } } },
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { resolveWordCardKind, type NoteFieldValueInfo } from './note-field-utils';
|
||||
|
||||
function kikuNote(values: Record<string, string> = {}): NoteFieldValueInfo {
|
||||
const defaults: Record<string, string> = {
|
||||
Expression: '単語',
|
||||
Sentence: '',
|
||||
IsWordAndSentenceCard: '',
|
||||
IsClickCard: '',
|
||||
IsSentenceCard: '',
|
||||
IsAudioCard: '',
|
||||
};
|
||||
return {
|
||||
fields: Object.fromEntries(
|
||||
Object.entries({ ...defaults, ...values }).map(([name, value]) => [name, { value }]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
test('marks word-and-sentence cards by default when Kiku is enabled', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote(), { lapisEnabled: false, kikuEnabled: true }),
|
||||
'word-and-sentence',
|
||||
);
|
||||
});
|
||||
|
||||
test('honors the configured word card kind', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote(), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
wordCardKind: 'click',
|
||||
}),
|
||||
'click',
|
||||
);
|
||||
});
|
||||
|
||||
test('marks nothing when neither Kiku nor Lapis is enabled', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote(), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: false,
|
||||
wordCardKind: 'click',
|
||||
}),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
test('marks nothing when the word card kind is "none"', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote(), {
|
||||
lapisEnabled: true,
|
||||
kikuEnabled: false,
|
||||
wordCardKind: 'none',
|
||||
}),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
test('falls back to the default kind for an unrecognized setting', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote(), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
wordCardKind: 'bogus' as never,
|
||||
}),
|
||||
'word-and-sentence',
|
||||
);
|
||||
});
|
||||
|
||||
test('marks nothing when the note type lacks the configured flag field', () => {
|
||||
const note: NoteFieldValueInfo = {
|
||||
fields: { Expression: { value: '単語' }, Sentence: { value: '' } },
|
||||
};
|
||||
|
||||
assert.equal(
|
||||
resolveWordCardKind(note, { lapisEnabled: false, kikuEnabled: true, wordCardKind: 'click' }),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
test('leaves cards already mined as sentence or audio cards alone', () => {
|
||||
for (const flagField of ['IsSentenceCard', 'IsAudioCard']) {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote({ [flagField]: 'x' }), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
wordCardKind: 'click',
|
||||
}),
|
||||
null,
|
||||
flagField,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('re-affirms the configured kind when the note already carries its flag', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote({ IsSentenceCard: 'x' }), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
wordCardKind: 'sentence',
|
||||
}),
|
||||
'sentence',
|
||||
);
|
||||
});
|
||||
|
||||
test('overrides a differently flagged word card', () => {
|
||||
assert.equal(
|
||||
resolveWordCardKind(kikuNote({ IsWordAndSentenceCard: 'x' }), {
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
wordCardKind: 'click',
|
||||
}),
|
||||
'click',
|
||||
);
|
||||
});
|
||||
@@ -1,13 +1,3 @@
|
||||
import type { CardKind, WordCardKind } from '../types/anki';
|
||||
import { createLogger } from '../logger';
|
||||
import {
|
||||
CARD_KIND_FLAG_FIELDS,
|
||||
DEFAULT_WORD_CARD_KIND,
|
||||
resolveWordCardKindSetting,
|
||||
} from './card-kinds';
|
||||
|
||||
const log = createLogger('anki').child('integration.note-fields');
|
||||
|
||||
export interface NoteFieldValueInfo {
|
||||
fields: Record<string, { value: string }>;
|
||||
}
|
||||
@@ -26,57 +16,22 @@ export function hasNoteFieldValue(noteInfo: NoteFieldValueInfo, preferredName: s
|
||||
return (getNoteFieldValue(noteInfo, preferredName) ?? '').trim().length > 0;
|
||||
}
|
||||
|
||||
/** Flags set only by an explicit mine action; a note carrying one is not a word card. */
|
||||
const EXPLICIT_CARD_FLAG_FIELDS = [CARD_KIND_FLAG_FIELDS.sentence, CARD_KIND_FLAG_FIELDS.audio];
|
||||
|
||||
const warnedMissingFlagFields = new Set<CardKind>();
|
||||
|
||||
function warnMissingFlagFieldOnce(wordCardKind: CardKind, flagField: string): void {
|
||||
if (wordCardKind === DEFAULT_WORD_CARD_KIND || warnedMissingFlagFields.has(wordCardKind)) {
|
||||
// The default kind is also the fallback for plain note types, so its absence is expected.
|
||||
return;
|
||||
}
|
||||
warnedMissingFlagFields.add(wordCardKind);
|
||||
log.warn(
|
||||
`Word card type "${wordCardKind}" is configured but the note has no ${flagField} field; leaving card type flags unchanged.`,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Card kind to flag when SubMiner fills a word card's sentence, or null to leave the
|
||||
* card-kind flags alone. Kiku/Lapis only: other note types have no such fields.
|
||||
*/
|
||||
export function resolveWordCardKind(
|
||||
export function shouldMarkWordAndSentenceCard(
|
||||
noteInfo: NoteFieldValueInfo,
|
||||
sentenceCardConfig: {
|
||||
lapisEnabled: boolean;
|
||||
kikuEnabled: boolean;
|
||||
wordCardKind?: WordCardKind;
|
||||
},
|
||||
): CardKind | null {
|
||||
sentenceCardConfig: { lapisEnabled: boolean; kikuEnabled: boolean },
|
||||
): boolean {
|
||||
if (!sentenceCardConfig.lapisEnabled && !sentenceCardConfig.kikuEnabled) {
|
||||
return null;
|
||||
return false;
|
||||
}
|
||||
|
||||
const wordCardKind = resolveWordCardKindSetting(sentenceCardConfig.wordCardKind);
|
||||
if (wordCardKind === 'none') {
|
||||
return null;
|
||||
const wordAndSentenceValue = getNoteFieldValue(noteInfo, 'IsWordAndSentenceCard');
|
||||
if (wordAndSentenceValue === null) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const flagField = CARD_KIND_FLAG_FIELDS[wordCardKind];
|
||||
const flagValue = getNoteFieldValue(noteInfo, flagField);
|
||||
if (flagValue === null) {
|
||||
// Note type has no flag field for the configured kind.
|
||||
warnMissingFlagFieldOnce(wordCardKind, flagField);
|
||||
return null;
|
||||
if (wordAndSentenceValue.trim().length > 0) {
|
||||
return true;
|
||||
}
|
||||
if (flagValue.trim().length > 0) {
|
||||
return wordCardKind;
|
||||
}
|
||||
|
||||
const alreadyExplicitCard = EXPLICIT_CARD_FLAG_FIELDS.some(
|
||||
(fieldName) =>
|
||||
fieldName.toLowerCase() !== flagField.toLowerCase() && hasNoteFieldValue(noteInfo, fieldName),
|
||||
return (
|
||||
!hasNoteFieldValue(noteInfo, 'IsSentenceCard') && !hasNoteFieldValue(noteInfo, 'IsAudioCard')
|
||||
);
|
||||
return alreadyExplicitCard ? null : wordCardKind;
|
||||
}
|
||||
|
||||
@@ -6,21 +6,26 @@ import {
|
||||
type NoteUpdateWorkflowNoteInfo,
|
||||
} from './note-update-workflow';
|
||||
import type { SubtitleMiningContext } from '../types/subtitle';
|
||||
import type { CardKind } from '../types/anki';
|
||||
import { applyCardKindFlagFields } from './card-kinds';
|
||||
|
||||
function setCardTypeFields(
|
||||
function setWordAndSentenceCardTypeFields(
|
||||
updatedFields: Record<string, string>,
|
||||
availableFieldNames: string[],
|
||||
cardKind: CardKind,
|
||||
cardKind: 'word-and-sentence',
|
||||
): void {
|
||||
applyCardKindFlagFields(
|
||||
updatedFields,
|
||||
cardKind,
|
||||
(preferredName) =>
|
||||
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ??
|
||||
null,
|
||||
);
|
||||
assert.equal(cardKind, 'word-and-sentence');
|
||||
const resolveFieldName = (preferredName: string): string | null =>
|
||||
availableFieldNames.find((name) => name.toLowerCase() === preferredName.toLowerCase()) ?? null;
|
||||
|
||||
const wordAndSentenceFlag = resolveFieldName('IsWordAndSentenceCard');
|
||||
if (!wordAndSentenceFlag) return;
|
||||
|
||||
updatedFields[wordAndSentenceFlag] = 'x';
|
||||
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
|
||||
const resolved = resolveFieldName(flagName);
|
||||
if (resolved && resolved !== wordAndSentenceFlag) {
|
||||
updatedFields[resolved] = '';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createWorkflowHarness() {
|
||||
@@ -74,7 +79,7 @@ function createWorkflowHarness() {
|
||||
handleFieldGroupingManual: async (_originalNoteId, _newNoteId, _newNoteInfo, _expression) =>
|
||||
false,
|
||||
processSentence: (text: string, _noteFields: Record<string, string>) => text,
|
||||
setCardTypeFields,
|
||||
setCardTypeFields: setWordAndSentenceCardTypeFields,
|
||||
resolveConfiguredFieldName: (noteInfo: NoteUpdateWorkflowNoteInfo, preferred?: string) => {
|
||||
if (!preferred) return null;
|
||||
const names = Object.keys(noteInfo.fields);
|
||||
@@ -178,73 +183,6 @@ test('NoteUpdateWorkflow marks enriched Kiku word cards as word-and-sentence car
|
||||
});
|
||||
});
|
||||
|
||||
test('NoteUpdateWorkflow marks the configured word card kind instead of word-and-sentence', async () => {
|
||||
const harness = createWorkflowHarness();
|
||||
harness.deps.getEffectiveSentenceCardConfig = () => ({
|
||||
sentenceField: 'Sentence',
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
kikuFieldGrouping: 'manual',
|
||||
wordCardKind: 'click',
|
||||
});
|
||||
harness.deps.client.notesInfo = async () =>
|
||||
[
|
||||
{
|
||||
noteId: 42,
|
||||
fields: {
|
||||
Expression: { value: 'taberu' },
|
||||
Sentence: { value: '' },
|
||||
IsWordAndSentenceCard: { value: 'x' },
|
||||
IsClickCard: { value: '' },
|
||||
IsSentenceCard: { value: '' },
|
||||
IsAudioCard: { value: '' },
|
||||
},
|
||||
},
|
||||
] satisfies NoteUpdateWorkflowNoteInfo[];
|
||||
|
||||
await harness.workflow.execute(42);
|
||||
|
||||
assert.equal(harness.updates.length, 1);
|
||||
assert.deepEqual(harness.updates[0]?.fields, {
|
||||
Sentence: 'subtitle-text',
|
||||
IsClickCard: 'x',
|
||||
IsWordAndSentenceCard: '',
|
||||
IsSentenceCard: '',
|
||||
IsAudioCard: '',
|
||||
});
|
||||
});
|
||||
|
||||
test('NoteUpdateWorkflow leaves card type flags alone when the word card kind is none', async () => {
|
||||
const harness = createWorkflowHarness();
|
||||
harness.deps.getEffectiveSentenceCardConfig = () => ({
|
||||
sentenceField: 'Sentence',
|
||||
lapisEnabled: false,
|
||||
kikuEnabled: true,
|
||||
kikuFieldGrouping: 'manual',
|
||||
wordCardKind: 'none',
|
||||
});
|
||||
harness.deps.client.notesInfo = async () =>
|
||||
[
|
||||
{
|
||||
noteId: 42,
|
||||
fields: {
|
||||
Expression: { value: 'taberu' },
|
||||
Sentence: { value: '' },
|
||||
IsWordAndSentenceCard: { value: '' },
|
||||
IsSentenceCard: { value: '' },
|
||||
IsAudioCard: { value: '' },
|
||||
},
|
||||
},
|
||||
] satisfies NoteUpdateWorkflowNoteInfo[];
|
||||
|
||||
await harness.workflow.execute(42);
|
||||
|
||||
assert.equal(harness.updates.length, 1);
|
||||
assert.deepEqual(harness.updates[0]?.fields, {
|
||||
Sentence: 'subtitle-text',
|
||||
});
|
||||
});
|
||||
|
||||
test('NoteUpdateWorkflow does not set Kiku card flags when Lapis and Kiku are disabled', async () => {
|
||||
const harness = createWorkflowHarness();
|
||||
harness.deps.client.notesInfo = async () =>
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import { DEFAULT_ANKI_CONNECT_CONFIG } from '../config';
|
||||
import { getPreferredWordValueFromExtractedFields } from '../anki-field-config';
|
||||
import type { SubtitleMiningContext } from '../types/subtitle';
|
||||
import type { CardKind, WordCardKind } from '../types/anki';
|
||||
import { resolveWordCardKind } from './note-field-utils';
|
||||
import { shouldMarkWordAndSentenceCard } from './note-field-utils';
|
||||
|
||||
export interface NoteUpdateWorkflowNoteInfo {
|
||||
noteId: number;
|
||||
@@ -40,7 +39,6 @@ export interface NoteUpdateWorkflowDeps {
|
||||
lapisEnabled: boolean;
|
||||
kikuEnabled: boolean;
|
||||
kikuFieldGrouping: 'auto' | 'manual' | 'disabled';
|
||||
wordCardKind?: WordCardKind;
|
||||
};
|
||||
appendKnownWordsFromNoteInfo: (noteInfo: NoteUpdateWorkflowNoteInfo) => void;
|
||||
extractFields: (fields: Record<string, { value: string }>) => Record<string, string>;
|
||||
@@ -69,7 +67,7 @@ export interface NoteUpdateWorkflowDeps {
|
||||
setCardTypeFields: (
|
||||
updatedFields: Record<string, string>,
|
||||
availableFieldNames: string[],
|
||||
cardKind: CardKind,
|
||||
cardKind: 'word-and-sentence',
|
||||
) => void;
|
||||
resolveConfiguredFieldName: (
|
||||
noteInfo: NoteUpdateWorkflowNoteInfo,
|
||||
@@ -209,9 +207,12 @@ export class NoteUpdateWorkflow {
|
||||
if (sentenceField && currentSubtitleText) {
|
||||
const processedSentence = this.deps.processSentence(currentSubtitleText, fields);
|
||||
updatedFields[sentenceField] = processedSentence;
|
||||
const wordCardKind = resolveWordCardKind(noteInfo, sentenceCardConfig);
|
||||
if (wordCardKind) {
|
||||
this.deps.setCardTypeFields(updatedFields, Object.keys(noteInfo.fields), wordCardKind);
|
||||
if (shouldMarkWordAndSentenceCard(noteInfo, sentenceCardConfig)) {
|
||||
this.deps.setCardTypeFields(
|
||||
updatedFields,
|
||||
Object.keys(noteInfo.fields),
|
||||
'word-and-sentence',
|
||||
);
|
||||
}
|
||||
updatePerformed = true;
|
||||
}
|
||||
|
||||
@@ -116,10 +116,6 @@ export function normalizeAnkiIntegrationConfig(config: AnkiConnectConfig): AnkiC
|
||||
...DEFAULT_ANKI_CONNECT_CONFIG.isKiku,
|
||||
...(config.isKiku ?? {}),
|
||||
},
|
||||
lapisKiku: {
|
||||
...DEFAULT_ANKI_CONNECT_CONFIG.lapisKiku,
|
||||
...(config.lapisKiku ?? {}),
|
||||
},
|
||||
} as AnkiConnectConfig;
|
||||
}
|
||||
|
||||
@@ -209,10 +205,6 @@ export class AnkiIntegrationRuntime {
|
||||
patch.isKiku !== undefined
|
||||
? { ...this.config.isKiku, ...patch.isKiku }
|
||||
: this.config.isKiku,
|
||||
lapisKiku:
|
||||
patch.lapisKiku !== undefined
|
||||
? { ...this.config.lapisKiku, ...patch.lapisKiku }
|
||||
: this.config.lapisKiku,
|
||||
};
|
||||
this.config = normalizeAnkiIntegrationConfig(mergedConfig);
|
||||
this.deps.onConfigChanged?.(this.config);
|
||||
|
||||
@@ -2738,43 +2738,6 @@ test('ignores deprecated isLapis sentence-card field overrides', () => {
|
||||
);
|
||||
});
|
||||
|
||||
test('accepts a Kiku/Lapis word card kind and warns on an unknown one', () => {
|
||||
const dir = makeTempDir();
|
||||
fs.writeFileSync(
|
||||
path.join(dir, 'config.jsonc'),
|
||||
`{
|
||||
"ankiConnect": {
|
||||
"isKiku": { "enabled": true },
|
||||
"lapisKiku": { "wordCardKind": "click" }
|
||||
}
|
||||
}`,
|
||||
'utf-8',
|
||||
);
|
||||
|
||||
const service = new ConfigService(dir);
|
||||
assert.equal(service.getConfig().ankiConnect.lapisKiku.wordCardKind, 'click');
|
||||
assert.equal(service.getWarnings().length, 0);
|
||||
|
||||
const invalidDir = makeTempDir();
|
||||
fs.writeFileSync(
|
||||
path.join(invalidDir, 'config.jsonc'),
|
||||
`{
|
||||
"ankiConnect": {
|
||||
"lapisKiku": { "wordCardKind": "isClickCard" }
|
||||
}
|
||||
}`,
|
||||
'utf-8',
|
||||
);
|
||||
|
||||
const invalidService = new ConfigService(invalidDir);
|
||||
assert.equal(invalidService.getConfig().ankiConnect.lapisKiku.wordCardKind, 'word-and-sentence');
|
||||
assert.ok(
|
||||
invalidService
|
||||
.getWarnings()
|
||||
.some((warning) => warning.path === 'ankiConnect.lapisKiku.wordCardKind'),
|
||||
);
|
||||
});
|
||||
|
||||
test('accepts valid ankiConnect knownWords deck object', () => {
|
||||
const dir = makeTempDir();
|
||||
fs.writeFileSync(
|
||||
|
||||
@@ -91,9 +91,6 @@ export const INTEGRATIONS_DEFAULT_CONFIG: Pick<
|
||||
fieldGrouping: 'disabled',
|
||||
deleteDuplicateInAuto: true,
|
||||
},
|
||||
lapisKiku: {
|
||||
wordCardKind: 'word-and-sentence',
|
||||
},
|
||||
},
|
||||
jimaku: {
|
||||
apiBaseUrl: 'https://jimaku.cc',
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { ResolvedConfig } from '../../types/config';
|
||||
import { WORD_CARD_KINDS } from '../../anki-integration/card-kinds';
|
||||
import { MPV_LAUNCH_MODE_VALUES } from '../../shared/mpv-launch-mode';
|
||||
import {
|
||||
NOTIFICATION_TYPE_VALUES,
|
||||
@@ -375,21 +374,6 @@ export function buildIntegrationConfigOptionRegistry(
|
||||
defaultValue: defaultConfig.ankiConnect.isLapis.sentenceCardModel,
|
||||
description: 'Note type name used by Lapis sentence cards.',
|
||||
},
|
||||
{
|
||||
path: 'ankiConnect.lapisKiku.wordCardKind',
|
||||
kind: 'enum',
|
||||
enumValues: WORD_CARD_KINDS,
|
||||
enumLabels: {
|
||||
'word-and-sentence': 'Word and sentence card (IsWordAndSentenceCard)',
|
||||
click: 'Click card (IsClickCard)',
|
||||
sentence: 'Sentence card (IsSentenceCard)',
|
||||
audio: 'Audio card (IsAudioCard)',
|
||||
none: 'Leave card type flags untouched',
|
||||
},
|
||||
defaultValue: defaultConfig.ankiConnect.lapisKiku.wordCardKind,
|
||||
description:
|
||||
'Card-type flag SubMiner marks on Kiku/Lapis word cards: word-and-sentence (IsWordAndSentenceCard), click (IsClickCard), sentence (IsSentenceCard), audio (IsAudioCard), or none to leave the flags untouched. Only applies when isKiku.enabled or isLapis.enabled; the other card-type flags are cleared so a note never claims two kinds. Cards mined with the dedicated sentence/audio actions keep their own flag.',
|
||||
},
|
||||
{
|
||||
path: 'ankiConnect.metadata.pattern',
|
||||
kind: 'string',
|
||||
|
||||
@@ -135,7 +135,7 @@ const INTEGRATION_TEMPLATE_SECTIONS: ConfigTemplateSection[] = [
|
||||
title: 'AnkiConnect Integration',
|
||||
description: ['Automatic Anki updates and media generation options.'],
|
||||
notes: [
|
||||
'Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, isKiku.fieldGrouping, and lapisKiku.wordCardKind update live while SubMiner is running.',
|
||||
'Hot-reload: ankiConnect.ai.enabled, media.normalizeAudio/mirrorMpvVolume, knownWords, nPlusOne, fields.word/audio/image/sentence/miscInfo, behavior.autoUpdateNewCards, isLapis.sentenceCardModel, and isKiku.fieldGrouping update live while SubMiner is running.',
|
||||
'Shared AI provider transport settings are read from top-level ai and typically require restart.',
|
||||
'Most other AnkiConnect settings still require restart.',
|
||||
],
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import type { ResolveContext } from './context';
|
||||
import { initializeAnkiConnectResolution } from './anki-connect/initialize';
|
||||
import { applyAnkiKikuResolution } from './anki-connect/kiku';
|
||||
import { applyAnkiLapisKikuResolution } from './anki-connect/lapis-kiku';
|
||||
import { applyAnkiKnownWordsResolution } from './anki-connect/known-words';
|
||||
import { applyAnkiLegacyResolution } from './anki-connect/legacy';
|
||||
import { applyAnkiModernResolution } from './anki-connect/modern';
|
||||
@@ -23,5 +22,4 @@ export function applyAnkiConnectResolution(context: ResolveContext): void {
|
||||
applyAnkiLegacyResolution(context, ankiConnect, behavior, fields, media, metadata);
|
||||
applyAnkiKnownWordsResolution(context, ankiConnect, behavior);
|
||||
applyAnkiKikuResolution(context);
|
||||
applyAnkiLapisKikuResolution(context, ankiConnect);
|
||||
}
|
||||
|
||||
@@ -77,8 +77,5 @@ export function initializeAnkiConnectResolution(
|
||||
? (ankiConnect.isKiku as (typeof context.resolved)['ankiConnect']['isKiku'])
|
||||
: {}),
|
||||
},
|
||||
lapisKiku: {
|
||||
...context.resolved.ankiConnect.lapisKiku,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,39 +0,0 @@
|
||||
import { isWordCardKind, WORD_CARD_KINDS } from '../../../anki-integration/card-kinds';
|
||||
import { DEFAULT_CONFIG } from '../../definitions';
|
||||
import type { ResolveContext } from '../context';
|
||||
import { isObject } from '../shared';
|
||||
|
||||
export function applyAnkiLapisKikuResolution(
|
||||
context: ResolveContext,
|
||||
ankiConnect: Record<string, unknown>,
|
||||
): void {
|
||||
if (!isObject(ankiConnect.lapisKiku)) {
|
||||
if (ankiConnect.lapisKiku !== undefined) {
|
||||
context.warn(
|
||||
'ankiConnect.lapisKiku',
|
||||
ankiConnect.lapisKiku,
|
||||
context.resolved.ankiConnect.lapisKiku,
|
||||
'Expected object.',
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const wordCardKind = ankiConnect.lapisKiku.wordCardKind;
|
||||
if (wordCardKind === undefined) {
|
||||
return;
|
||||
}
|
||||
if (isWordCardKind(wordCardKind)) {
|
||||
context.resolved.ankiConnect.lapisKiku.wordCardKind = wordCardKind;
|
||||
return;
|
||||
}
|
||||
|
||||
context.warn(
|
||||
'ankiConnect.lapisKiku.wordCardKind',
|
||||
wordCardKind,
|
||||
DEFAULT_CONFIG.ankiConnect.lapisKiku.wordCardKind,
|
||||
`Expected one of ${WORD_CARD_KINDS.join(', ')}.`,
|
||||
);
|
||||
context.resolved.ankiConnect.lapisKiku.wordCardKind =
|
||||
DEFAULT_CONFIG.ankiConnect.lapisKiku.wordCardKind;
|
||||
}
|
||||
@@ -221,7 +221,6 @@ const LABEL_OVERRIDES: Record<string, string> = {
|
||||
'ankiConnect.nPlusOne.enabled': 'Enabled',
|
||||
'ankiConnect.isLapis.enabled': 'Enable Lapis Features',
|
||||
'ankiConnect.isKiku.enabled': 'Enable Kiku Features',
|
||||
'ankiConnect.lapisKiku.wordCardKind': 'Word Card Type',
|
||||
'stats.toggleKey': 'Toggle Stats Overlay',
|
||||
'shortcuts.openCharacterDictionaryManager': 'Open Character Dictionary Manager',
|
||||
'subtitleSidebar.pauseVideoOnHover': 'Pause Video On Hover - Sidebar',
|
||||
@@ -256,8 +255,6 @@ const DESCRIPTION_OVERRIDES: Record<string, string> = {
|
||||
'Enable Lapis-specific mining behavior and sentence-card model targeting. When Kiku is enabled, Lapis features still work and Kiku-specific features are added on top.',
|
||||
'ankiConnect.isLapis.sentenceCardModel':
|
||||
'Anki note type used for Lapis sentence cards. Select from note types reported by AnkiConnect.',
|
||||
'ankiConnect.lapisKiku.wordCardKind':
|
||||
'Card-type flag marked on mined word cards: word-and-sentence sets IsWordAndSentenceCard, click sets IsClickCard, sentence sets IsSentenceCard, audio sets IsAudioCard, and none leaves the flags untouched. The other flags are cleared so a note never claims two card types. Requires Kiku or Lapis features to be enabled. Applies to Yomitan auto-enrichment, manual clipboard updates, and stats-dashboard word mining; cards mined with Mine Sentence or Mine Audio keep their own flag.',
|
||||
'subtitleStyle.css':
|
||||
'CSS declarations applied to primary subtitles. Includes color, background-color, and all font properties.',
|
||||
'subtitleStyle.secondary.css':
|
||||
@@ -404,11 +401,7 @@ function categoryAndSection(path: string): { category: ConfigSettingsCategory; s
|
||||
if (path.startsWith('ankiConnect.media.')) {
|
||||
return { category: 'mining-anki', section: 'Media Capture' };
|
||||
}
|
||||
if (
|
||||
path.startsWith('ankiConnect.isKiku.') ||
|
||||
path.startsWith('ankiConnect.isLapis.') ||
|
||||
path.startsWith('ankiConnect.lapisKiku.')
|
||||
) {
|
||||
if (path.startsWith('ankiConnect.isKiku.') || path.startsWith('ankiConnect.isLapis.')) {
|
||||
return { category: 'mining-anki', section: 'Kiku/Lapis Features' };
|
||||
}
|
||||
if (path.startsWith('ankiConnect.ai.')) {
|
||||
@@ -709,7 +702,6 @@ function restartBehaviorForPath(path: string): ConfigSettingsRestartBehavior {
|
||||
path === 'ankiConnect.fields.miscInfo' ||
|
||||
path === 'ankiConnect.isLapis.sentenceCardModel' ||
|
||||
path === 'ankiConnect.isKiku.fieldGrouping' ||
|
||||
path === 'ankiConnect.lapisKiku.wordCardKind' ||
|
||||
path === 'mpv.aniskipEnabled' ||
|
||||
path === 'mpv.aniskipButtonKey' ||
|
||||
path === 'stats.toggleKey' ||
|
||||
|
||||
@@ -1100,56 +1100,12 @@ describe('stats server API routes', () => {
|
||||
assert.deepEqual(assignments, [{ animeId: 1, body }]);
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/:animeId/cover returns revalidated cover art', async () => {
|
||||
it('GET /api/stats/anime/:animeId/cover returns cover art', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/anime/1/cover');
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(res.headers.get('content-type'), 'image/jpeg');
|
||||
assert.equal(res.headers.get('cache-control'), 'no-cache');
|
||||
assert.ok(res.headers.get('etag'));
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/:animeId/cover answers 304 for a matching ETag', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const first = await app.request('/api/stats/anime/1/cover');
|
||||
const etag = first.headers.get('etag');
|
||||
assert.ok(etag);
|
||||
|
||||
const cached = await app.request('/api/stats/anime/1/cover', {
|
||||
headers: { 'If-None-Match': `W/${etag}, "other"` },
|
||||
});
|
||||
assert.equal(cached.status, 304);
|
||||
assert.equal(cached.headers.get('etag'), etag);
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/:animeId/cover resends art when the ETag no longer matches', async () => {
|
||||
// A relinked AniList entry swaps the bytes behind the same cover URL.
|
||||
let coverBlob = Buffer.from([0xff, 0xd8, 0xff, 0xd9]);
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getAnimeCoverArt: async () => ({
|
||||
videoId: 1,
|
||||
anilistId: 21858,
|
||||
coverUrl: 'https://example.com/cover.jpg',
|
||||
coverBlob,
|
||||
titleRomaji: 'Little Witch Academia',
|
||||
titleEnglish: 'Little Witch Academia',
|
||||
episodesTotal: 25,
|
||||
fetchedAtMs: Date.now(),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
const first = await app.request('/api/stats/anime/1/cover');
|
||||
const staleEtag = first.headers.get('etag');
|
||||
assert.ok(staleEtag);
|
||||
|
||||
coverBlob = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||
const res = await app.request('/api/stats/anime/1/cover', {
|
||||
headers: { 'If-None-Match': staleEtag },
|
||||
});
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(res.headers.get('content-type'), 'image/png');
|
||||
assert.notEqual(res.headers.get('etag'), staleEtag);
|
||||
assert.equal(res.headers.get('cache-control'), 'public, max-age=86400');
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/:animeId/cover serves detected cover MIME type', async () => {
|
||||
@@ -2454,80 +2410,6 @@ Aligned English subtitle
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/mine-card marks the configured Kiku word card kind', async () => {
|
||||
await withTempDir(async (dir) => {
|
||||
const sourcePath = path.join(dir, 'episode.mkv');
|
||||
fs.writeFileSync(sourcePath, 'fake media');
|
||||
|
||||
await withFakeAnkiConnect(
|
||||
async (requests, url) => {
|
||||
const app = createStatsApp(createMockTracker(), {
|
||||
addYomitanNote: async () => 777,
|
||||
createMediaGenerator: () => ({
|
||||
generateAudio: async () => null,
|
||||
generateScreenshot: async () => null,
|
||||
generateAnimatedImage: async () => null,
|
||||
}),
|
||||
ankiConnectConfig: {
|
||||
url,
|
||||
deck: 'Mining',
|
||||
fields: {
|
||||
image: 'Picture',
|
||||
sentence: 'Sentence',
|
||||
},
|
||||
media: {
|
||||
generateAudio: false,
|
||||
generateImage: false,
|
||||
},
|
||||
isKiku: {
|
||||
enabled: true,
|
||||
fieldGrouping: 'disabled',
|
||||
deleteDuplicateInAuto: true,
|
||||
},
|
||||
lapisKiku: {
|
||||
wordCardKind: 'click',
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const res = await app.request('/api/stats/mine-card?mode=word', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourcePath,
|
||||
startMs: 1_000,
|
||||
endMs: 2_000,
|
||||
sentence: '猫を見た',
|
||||
word: '猫',
|
||||
videoTitle: 'Episode 1',
|
||||
}),
|
||||
});
|
||||
|
||||
const body = await res.json();
|
||||
assert.equal(res.status, 200, JSON.stringify(body));
|
||||
|
||||
const updateRequest = requests.find((request) => request.action === 'updateNoteFields');
|
||||
const fields = updateRequest?.params?.note?.fields ?? {};
|
||||
assert.equal(fields.IsClickCard, 'x');
|
||||
assert.equal(fields.IsWordAndSentenceCard, '');
|
||||
assert.equal(fields.IsSentenceCard, '');
|
||||
assert.equal(fields.IsAudioCard, '');
|
||||
},
|
||||
{
|
||||
notesInfoFields: {
|
||||
Expression: { value: '猫' },
|
||||
Sentence: { value: '' },
|
||||
Picture: { value: '' },
|
||||
IsWordAndSentenceCard: { value: '' },
|
||||
IsClickCard: { value: '' },
|
||||
IsSentenceCard: { value: '' },
|
||||
IsAudioCard: { value: '' },
|
||||
},
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/mine-card writes word mining sentence audio and image together', async () => {
|
||||
await withTempDir(async (dir) => {
|
||||
const sourcePath = path.join(dir, 'episode.mkv');
|
||||
@@ -2991,39 +2873,6 @@ Aligned English subtitle
|
||||
assert.equal(deleteCalls, 0);
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/:animeId deletes the whole library entry', async () => {
|
||||
let deletedAnimeId: number | null = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
deleteAnime: async (animeId: number) => {
|
||||
deletedAnimeId = animeId;
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/7', { method: 'DELETE' });
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(deletedAnimeId, 7);
|
||||
assert.deepEqual(await res.json(), { ok: true });
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/:animeId rejects non-positive anime ids', async () => {
|
||||
let deleteCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
deleteAnime: async () => {
|
||||
deleteCalls += 1;
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/0', { method: 'DELETE' });
|
||||
|
||||
assert.equal(res.status, 400);
|
||||
assert.equal(deleteCalls, 0);
|
||||
});
|
||||
|
||||
it('POST /api/stats/anki/browse returns 400 for missing noteId', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/anki/browse', { method: 'POST' });
|
||||
|
||||
@@ -143,50 +143,3 @@ test('anilist update queue persists and reloads from disk', () => {
|
||||
});
|
||||
assert.equal(queueB.nextReady(Number.MAX_SAFE_INTEGER)?.title, 'Persist Demo');
|
||||
});
|
||||
|
||||
test('drops queued items whose persisted mediaId is not a positive integer', () => {
|
||||
const filePath = path.join(
|
||||
fs.mkdtempSync(path.join(os.tmpdir(), 'anilist-queue-mediaid-')),
|
||||
'queue.json',
|
||||
);
|
||||
const base = {
|
||||
episode: 1,
|
||||
createdAt: 1,
|
||||
attemptCount: 0,
|
||||
nextAttemptAt: 0,
|
||||
lastError: null,
|
||||
};
|
||||
fs.writeFileSync(
|
||||
filePath,
|
||||
JSON.stringify({
|
||||
pending: [
|
||||
{ ...base, key: 'ok-absent', title: 'A' },
|
||||
{ ...base, key: 'ok-null', title: 'B', mediaId: null },
|
||||
{ ...base, key: 'ok-valid', title: 'C', mediaId: 108489 },
|
||||
{ ...base, key: 'bad-string', title: 'D', mediaId: '108489' },
|
||||
{ ...base, key: 'bad-zero', title: 'E', mediaId: 0 },
|
||||
{ ...base, key: 'bad-negative', title: 'F', mediaId: -3 },
|
||||
{ ...base, key: 'bad-float', title: 'G', mediaId: 1.5 },
|
||||
{ ...base, key: 'bad-object', title: 'H', mediaId: { id: 1 } },
|
||||
],
|
||||
deadLetter: [],
|
||||
}),
|
||||
'utf-8',
|
||||
);
|
||||
|
||||
const queue = createAnilistUpdateQueue(filePath, {
|
||||
info: () => {},
|
||||
warn: () => {},
|
||||
error: () => {},
|
||||
});
|
||||
|
||||
const keptKeys: string[] = [];
|
||||
for (let i = 0; i < 8; i += 1) {
|
||||
const next = queue.nextReady(1000);
|
||||
if (!next) break;
|
||||
keptKeys.push(next.key);
|
||||
queue.markSuccess(next.key);
|
||||
}
|
||||
|
||||
assert.deepEqual(keptKeys, ['ok-absent', 'ok-null', 'ok-valid']);
|
||||
});
|
||||
|
||||
@@ -10,8 +10,6 @@ export interface AnilistQueuedUpdate {
|
||||
key: string;
|
||||
title: string;
|
||||
season?: number | null;
|
||||
/** Pinned AniList media id from a manual override, when one applied at enqueue time. */
|
||||
mediaId?: number | null;
|
||||
episode: number;
|
||||
createdAt: number;
|
||||
attemptCount: number;
|
||||
@@ -31,31 +29,13 @@ export interface AnilistRetryQueueSnapshot {
|
||||
}
|
||||
|
||||
export interface AnilistUpdateQueue {
|
||||
enqueue: (
|
||||
key: string,
|
||||
title: string,
|
||||
episode: number,
|
||||
season?: number | null,
|
||||
mediaId?: number | null,
|
||||
) => void;
|
||||
enqueue: (key: string, title: string, episode: number, season?: number | null) => void;
|
||||
nextReady: (nowMs?: number) => AnilistQueuedUpdate | null;
|
||||
markSuccess: (key: string) => void;
|
||||
markFailure: (key: string, reason: string, nowMs?: number) => void;
|
||||
getSnapshot: (nowMs?: number) => AnilistRetryQueueSnapshot;
|
||||
}
|
||||
|
||||
/**
|
||||
* A persisted mediaId pins the update to a specific AniList entry, so a corrupted or
|
||||
* hand-edited queue file must not be able to feed a bogus id straight to the mutation.
|
||||
*/
|
||||
function isValidPersistedMediaId(value: unknown): boolean {
|
||||
return (
|
||||
value === undefined ||
|
||||
value === null ||
|
||||
(typeof value === 'number' && Number.isInteger(value) && value > 0)
|
||||
);
|
||||
}
|
||||
|
||||
function clampBackoffMs(attemptCount: number): number {
|
||||
const computed = INITIAL_BACKOFF_MS * Math.pow(2, Math.max(0, attemptCount - 1));
|
||||
return Math.min(MAX_BACKOFF_MS, computed);
|
||||
@@ -102,7 +82,6 @@ export function createAnilistUpdateQueue(
|
||||
typeof item.createdAt === 'number' &&
|
||||
typeof item.attemptCount === 'number' &&
|
||||
typeof item.nextAttemptAt === 'number' &&
|
||||
isValidPersistedMediaId(item.mediaId) &&
|
||||
(typeof item.lastError === 'string' || item.lastError === null),
|
||||
)
|
||||
.slice(0, MAX_ITEMS);
|
||||
@@ -117,7 +96,6 @@ export function createAnilistUpdateQueue(
|
||||
typeof item.createdAt === 'number' &&
|
||||
typeof item.attemptCount === 'number' &&
|
||||
typeof item.nextAttemptAt === 'number' &&
|
||||
isValidPersistedMediaId(item.mediaId) &&
|
||||
(typeof item.lastError === 'string' || item.lastError === null),
|
||||
)
|
||||
.slice(0, MAX_ITEMS);
|
||||
@@ -129,13 +107,7 @@ export function createAnilistUpdateQueue(
|
||||
load();
|
||||
|
||||
return {
|
||||
enqueue(
|
||||
key: string,
|
||||
title: string,
|
||||
episode: number,
|
||||
season: number | null = null,
|
||||
mediaId: number | null = null,
|
||||
): void {
|
||||
enqueue(key: string, title: string, episode: number, season: number | null = null): void {
|
||||
const existing =
|
||||
pending.find((item) => item.key === key) || deadLetter.find((item) => item.key === key);
|
||||
if (existing) {
|
||||
@@ -148,7 +120,6 @@ export function createAnilistUpdateQueue(
|
||||
key,
|
||||
title,
|
||||
season,
|
||||
mediaId,
|
||||
episode,
|
||||
createdAt: Date.now(),
|
||||
attemptCount: 0,
|
||||
|
||||
@@ -376,7 +376,7 @@ test('updateAnilistPostWatchProgress returns non-retryable error when media is n
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress resolves later seasons through the sequel chain', async () => {
|
||||
test('updateAnilistPostWatchProgress prefers season-specific AniList matches', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
const searchTerms: string[] = [];
|
||||
let call = 0;
|
||||
@@ -389,45 +389,21 @@ test('updateAnilistPostWatchProgress resolves later seasons through the sequel c
|
||||
data: {
|
||||
Page: {
|
||||
media: [
|
||||
{ id: 101, episodes: 12, format: 'TV', title: { english: 'Demo Show' } },
|
||||
{ id: 202, episodes: 12, format: 'TV', title: { english: 'Demo Show Sequel' } },
|
||||
{ id: 202, episodes: 12, title: { english: 'Demo Show Season 2' } },
|
||||
{ id: 101, episodes: 12, title: { english: 'Demo Show' } },
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
if (call === 2) {
|
||||
assert.equal(body.variables?.id, 101);
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
Media: {
|
||||
relations: {
|
||||
edges: [
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: {
|
||||
id: 202,
|
||||
type: 'ANIME',
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
title: { english: 'Demo Show Sequel' },
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
if (call === 3) {
|
||||
assert.equal(body.variables?.mediaId, 202);
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
Media: { id: 202, mediaListEntry: { progress: 1, status: 'CURRENT' } },
|
||||
Media: { id: 202, mediaListEntry: null },
|
||||
},
|
||||
});
|
||||
}
|
||||
assert.equal(body.variables?.mediaId, 202);
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
SaveMediaListEntry: { progress: 2, status: 'CURRENT' },
|
||||
@@ -439,118 +415,16 @@ test('updateAnilistPostWatchProgress resolves later seasons through the sequel c
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Demo Show', 2, {
|
||||
season: 2,
|
||||
});
|
||||
// Searches the bare title: AniList has no "Demo Show Season 2" entry to find.
|
||||
assert.deepEqual(searchTerms, ['Demo Show']);
|
||||
assert.equal(result.status, 'updated');
|
||||
assert.match(result.message, /Demo Show Sequel/);
|
||||
assert.equal(call, 4);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress refuses to update season 1 when the season is unresolvable', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let call = 0;
|
||||
globalThis.fetch = (async () => {
|
||||
call += 1;
|
||||
if (call === 1) {
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
Page: {
|
||||
media: [{ id: 101, episodes: 12, format: 'TV', title: { english: 'Demo Show' } }],
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
return createJsonResponse({ data: { Media: { relations: { edges: [] } } } });
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Demo Show', 2, { season: 3 });
|
||||
assert.deepEqual(searchTerms, ['Demo Show Season 2']);
|
||||
assert.equal(result.status, 'error');
|
||||
assert.equal(result.retryable, false);
|
||||
assert.match(result.message, /could not find season 3/i);
|
||||
// Stops after search + relation lookup: no entry lookup, no save.
|
||||
assert.match(result.message, /not in your AniList Planning or Watching list/i);
|
||||
assert.equal(call, 2);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress uses a pinned media id without searching', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let call = 0;
|
||||
globalThis.fetch = (async (_input, init) => {
|
||||
call += 1;
|
||||
const body = JSON.parse(String(init?.body)) as { variables?: Record<string, unknown> };
|
||||
if (call === 1) {
|
||||
assert.equal(body.variables?.mediaId, 108489);
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
Media: {
|
||||
id: 108489,
|
||||
episodes: 12,
|
||||
title: { english: 'Pinned Show' },
|
||||
mediaListEntry: { progress: 4, status: 'CURRENT' },
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
assert.equal(body.variables?.mediaId, 108489);
|
||||
return createJsonResponse({
|
||||
data: { SaveMediaListEntry: { progress: 5, status: 'CURRENT' } },
|
||||
});
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Wrong Guess', 5, {
|
||||
season: 3,
|
||||
mediaId: 108489,
|
||||
});
|
||||
assert.equal(result.status, 'updated');
|
||||
assert.match(result.message, /Pinned Show/);
|
||||
assert.equal(call, 2);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress marks the final episode completed for a pinned media id', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let call = 0;
|
||||
globalThis.fetch = (async (_input, init) => {
|
||||
call += 1;
|
||||
const body = JSON.parse(String(init?.body)) as { variables?: Record<string, unknown> };
|
||||
if (call === 1) {
|
||||
return createJsonResponse({
|
||||
data: {
|
||||
Media: {
|
||||
id: 108489,
|
||||
episodes: 12,
|
||||
title: { english: 'Pinned Show' },
|
||||
mediaListEntry: { progress: 11, status: 'CURRENT' },
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
assert.equal(body.variables?.status, 'COMPLETED');
|
||||
return createJsonResponse({
|
||||
data: { SaveMediaListEntry: { progress: 12, status: 'COMPLETED' } },
|
||||
});
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Pinned Show', 12, {
|
||||
mediaId: 108489,
|
||||
});
|
||||
assert.equal(result.status, 'updated');
|
||||
assert.match(result.message, /completed/i);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress does not update rewatching entries', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let call = 0;
|
||||
@@ -605,33 +479,3 @@ test('updateAnilistPostWatchProgress returns error when search fails', async ()
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress does not requeue a search that matched nothing', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async () =>
|
||||
createJsonResponse({ data: { Page: { media: [] } } })) as typeof fetch;
|
||||
|
||||
try {
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Unknown Show', 3);
|
||||
assert.equal(result.status, 'error');
|
||||
assert.equal(result.retryable, false);
|
||||
assert.match(result.message, /no matches/i);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('updateAnilistPostWatchProgress still allows retry when the search itself fails', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async () =>
|
||||
createJsonResponse({ errors: [{ message: 'upstream exploded' }] })) as typeof fetch;
|
||||
|
||||
try {
|
||||
const result = await updateAnilistPostWatchProgress('token', 'Demo Show', 3);
|
||||
assert.equal(result.status, 'error');
|
||||
assert.notEqual(result.retryable, false);
|
||||
assert.match(result.message, /search failed/i);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -3,7 +3,6 @@ import * as path from 'path';
|
||||
|
||||
import { parseMediaInfo } from '../../../jimaku/utils';
|
||||
import type { AnilistRateLimiter } from './rate-limiter';
|
||||
import { resolveAnilistSeasonMedia } from './season-resolver';
|
||||
|
||||
const ANILIST_GRAPHQL_URL = 'https://graphql.anilist.co';
|
||||
|
||||
@@ -25,12 +24,6 @@ export interface AnilistPostWatchUpdateResult {
|
||||
export interface AnilistPostWatchUpdateOptions {
|
||||
rateLimiter?: AnilistRateLimiter;
|
||||
season?: number | null;
|
||||
/**
|
||||
* Pinned AniList media id (from a character dictionary manual override). When set,
|
||||
* the search/season resolution is skipped entirely.
|
||||
*/
|
||||
mediaId?: number | null;
|
||||
logInfo?: (message: string) => void;
|
||||
}
|
||||
|
||||
interface AnilistGraphQlError {
|
||||
@@ -42,15 +35,23 @@ interface AnilistGraphQlResponse<T> {
|
||||
errors?: AnilistGraphQlError[];
|
||||
}
|
||||
|
||||
interface AnilistSearchData {
|
||||
Page?: {
|
||||
media?: Array<{
|
||||
id: number;
|
||||
episodes: number | null;
|
||||
title?: {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
native?: string | null;
|
||||
};
|
||||
}>;
|
||||
};
|
||||
}
|
||||
|
||||
interface AnilistMediaEntryData {
|
||||
Media?: {
|
||||
id: number;
|
||||
episodes?: number | null;
|
||||
title?: {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
native?: string | null;
|
||||
} | null;
|
||||
mediaListEntry?: {
|
||||
progress?: number | null;
|
||||
status?: string | null;
|
||||
@@ -153,6 +154,32 @@ function buildGuessitTitle(title: string, alternativeTitle: string | null): stri
|
||||
return title;
|
||||
}
|
||||
|
||||
function normalizeTitle(text: string): string {
|
||||
return text.trim().toLowerCase().replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
function titleMentionsSeason(title: string, season: number): boolean {
|
||||
const normalized = normalizeTitle(title);
|
||||
return (
|
||||
normalized.includes(`season ${season}`) ||
|
||||
normalized.includes(`s${String(season).padStart(2, '0')}`) ||
|
||||
normalized.includes(`s${season}`)
|
||||
);
|
||||
}
|
||||
|
||||
function buildSearchCandidates(title: string, season: number | null | undefined): string[] {
|
||||
const trimmed = title.trim();
|
||||
if (!trimmed) return [];
|
||||
const candidates =
|
||||
typeof season === 'number' &&
|
||||
Number.isInteger(season) &&
|
||||
season > 1 &&
|
||||
!titleMentionsSeason(trimmed, season)
|
||||
? [`${trimmed} Season ${season}`, trimmed]
|
||||
: [trimmed];
|
||||
return candidates.filter((candidate, index, all) => all.indexOf(candidate) === index);
|
||||
}
|
||||
|
||||
async function anilistGraphQl<T>(
|
||||
accessToken: string,
|
||||
query: string,
|
||||
@@ -189,22 +216,38 @@ function firstErrorMessage<T>(response: AnilistGraphQlResponse<T>): string | nul
|
||||
return firstError?.message ?? null;
|
||||
}
|
||||
|
||||
/** The season resolver signals failure by throwing; anilistGraphQl reports it in-band. */
|
||||
function createAnilistSeasonQueryExecutor(
|
||||
accessToken: string,
|
||||
options: AnilistPostWatchUpdateOptions,
|
||||
) {
|
||||
return async <T>(query: string, variables: Record<string, unknown>): Promise<T> => {
|
||||
const response = await anilistGraphQl<T>(accessToken, query, variables, options);
|
||||
const error = firstErrorMessage(response);
|
||||
if (error) {
|
||||
throw new Error(error);
|
||||
}
|
||||
if (!response.data) {
|
||||
throw new Error('AniList response missing data');
|
||||
}
|
||||
return response.data;
|
||||
};
|
||||
function pickBestSearchResult(
|
||||
title: string,
|
||||
episode: number,
|
||||
media: Array<{
|
||||
id: number;
|
||||
episodes: number | null;
|
||||
title?: {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
native?: string | null;
|
||||
};
|
||||
}>,
|
||||
): { id: number; title: string; episodes: number | null } | null {
|
||||
const filtered = media.filter((item) => {
|
||||
const totalEpisodes = item.episodes;
|
||||
return totalEpisodes === null || totalEpisodes >= episode;
|
||||
});
|
||||
const candidates = filtered.length > 0 ? filtered : media;
|
||||
if (candidates.length === 0) return null;
|
||||
|
||||
const normalizedTarget = normalizeTitle(title);
|
||||
const exact = candidates.find((item) => {
|
||||
const titles = [item.title?.romaji, item.title?.english, item.title?.native]
|
||||
.filter((value): value is string => typeof value === 'string')
|
||||
.map((value) => normalizeTitle(value));
|
||||
return titles.includes(normalizedTarget);
|
||||
});
|
||||
|
||||
const selected = exact ?? candidates[0]!;
|
||||
const selectedTitle =
|
||||
selected.title?.english || selected.title?.romaji || selected.title?.native || title;
|
||||
return { id: selected.id, title: selectedTitle, episodes: selected.episodes };
|
||||
}
|
||||
|
||||
function isUpdateableListStatus(status: string | null | undefined): boolean {
|
||||
@@ -278,53 +321,52 @@ export async function updateAnilistPostWatchProgress(
|
||||
episode: number,
|
||||
options: AnilistPostWatchUpdateOptions = {},
|
||||
): Promise<AnilistPostWatchUpdateResult> {
|
||||
const pinnedMediaId =
|
||||
typeof options.mediaId === 'number' && Number.isInteger(options.mediaId) && options.mediaId > 0
|
||||
? options.mediaId
|
||||
: null;
|
||||
|
||||
let mediaId = pinnedMediaId;
|
||||
let resolvedTitle: string | null = null;
|
||||
let resolvedEpisodes: number | null = null;
|
||||
|
||||
if (mediaId === null) {
|
||||
let resolution: Awaited<ReturnType<typeof resolveAnilistSeasonMedia>>;
|
||||
try {
|
||||
resolution = await resolveAnilistSeasonMedia(
|
||||
{ title, season: options.season, episode },
|
||||
{
|
||||
execute: createAnilistSeasonQueryExecutor(accessToken, options),
|
||||
logInfo: options.logInfo,
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
return {
|
||||
status: 'error',
|
||||
message: `AniList search failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
};
|
||||
let media: NonNullable<NonNullable<AnilistSearchData['Page']>['media']> = [];
|
||||
let searchError: string | null = null;
|
||||
let pickTitle = title;
|
||||
const searchCandidates = buildSearchCandidates(title, options.season);
|
||||
for (const search of searchCandidates) {
|
||||
const searchResponse = await anilistGraphQl<AnilistSearchData>(
|
||||
accessToken,
|
||||
`
|
||||
query ($search: String!) {
|
||||
Page(perPage: 5) {
|
||||
media(search: $search, type: ANIME) {
|
||||
id
|
||||
episodes
|
||||
title {
|
||||
romaji
|
||||
english
|
||||
native
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`,
|
||||
{ search },
|
||||
options,
|
||||
);
|
||||
searchError = firstErrorMessage(searchResponse);
|
||||
if (searchError) {
|
||||
break;
|
||||
}
|
||||
|
||||
if (!resolution) {
|
||||
// A well-formed search that matched nothing is deterministic for this title, so
|
||||
// requeueing it just burns rate limit. Transient failures throw and are caught above.
|
||||
return {
|
||||
status: 'error',
|
||||
retryable: false,
|
||||
message: 'AniList search returned no matches.',
|
||||
};
|
||||
}
|
||||
if (!resolution.seasonResolved) {
|
||||
// Updating the season 1 entry here is worse than not updating at all.
|
||||
return {
|
||||
status: 'error',
|
||||
retryable: false,
|
||||
message: `AniList update skipped: could not find season ${resolution.requestedSeason} of "${title}" (only matched "${resolution.title}"). Pick the right entry with the character dictionary AniList override.`,
|
||||
};
|
||||
media = searchResponse.data?.Page?.media ?? [];
|
||||
if (media.length > 0) {
|
||||
pickTitle = search;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
mediaId = resolution.id;
|
||||
resolvedTitle = resolution.title;
|
||||
resolvedEpisodes = resolution.episodes;
|
||||
if (searchError) {
|
||||
return {
|
||||
status: 'error',
|
||||
message: `AniList search failed: ${searchError}`,
|
||||
};
|
||||
}
|
||||
|
||||
const picked = pickBestSearchResult(pickTitle, episode, media);
|
||||
if (!picked) {
|
||||
return { status: 'error', message: 'AniList search returned no matches.' };
|
||||
}
|
||||
|
||||
const entryResponse = await anilistGraphQl<AnilistMediaEntryData>(
|
||||
@@ -333,12 +375,6 @@ export async function updateAnilistPostWatchProgress(
|
||||
query ($mediaId: Int!) {
|
||||
Media(id: $mediaId, type: ANIME) {
|
||||
id
|
||||
episodes
|
||||
title {
|
||||
romaji
|
||||
english
|
||||
native
|
||||
}
|
||||
mediaListEntry {
|
||||
progress
|
||||
status
|
||||
@@ -346,7 +382,7 @@ export async function updateAnilistPostWatchProgress(
|
||||
}
|
||||
}
|
||||
`,
|
||||
{ mediaId },
|
||||
{ mediaId: picked.id },
|
||||
options,
|
||||
);
|
||||
const entryError = firstErrorMessage(entryResponse);
|
||||
@@ -357,34 +393,21 @@ export async function updateAnilistPostWatchProgress(
|
||||
};
|
||||
}
|
||||
|
||||
const entryMedia = entryResponse.data?.Media ?? null;
|
||||
const pickedTitle =
|
||||
resolvedTitle ||
|
||||
entryMedia?.title?.english?.trim() ||
|
||||
entryMedia?.title?.romaji?.trim() ||
|
||||
entryMedia?.title?.native?.trim() ||
|
||||
title;
|
||||
const pickedEpisodes =
|
||||
resolvedEpisodes ??
|
||||
(typeof entryMedia?.episodes === 'number' && entryMedia.episodes > 0
|
||||
? entryMedia.episodes
|
||||
: null);
|
||||
|
||||
const entry = entryMedia?.mediaListEntry ?? null;
|
||||
const entry = entryResponse.data?.Media?.mediaListEntry ?? null;
|
||||
if (!entry || !isUpdateableListStatus(entry.status)) {
|
||||
return {
|
||||
status: 'error',
|
||||
retryable: false,
|
||||
message: `AniList update not possible: "${pickedTitle}" is ${formatListStatus(entry?.status)}. Add it to Planning or Watching, then mark watched again.`,
|
||||
message: `AniList update not possible: "${picked.title}" is ${formatListStatus(entry?.status)}. Add it to Planning or Watching, then mark watched again.`,
|
||||
};
|
||||
}
|
||||
|
||||
const currentProgress = entry.progress ?? 0;
|
||||
const shouldMarkCompleted = isKnownFinalEpisode(pickedEpisodes, episode);
|
||||
const shouldMarkCompleted = isKnownFinalEpisode(picked.episodes, episode);
|
||||
if (typeof currentProgress === 'number' && currentProgress >= episode && !shouldMarkCompleted) {
|
||||
return {
|
||||
status: 'skipped',
|
||||
message: `AniList already at episode ${currentProgress} (${pickedTitle}).`,
|
||||
message: `AniList already at episode ${currentProgress} (${picked.title}).`,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -399,7 +422,7 @@ export async function updateAnilistPostWatchProgress(
|
||||
}
|
||||
`,
|
||||
{
|
||||
mediaId,
|
||||
mediaId: picked.id,
|
||||
progress: episode,
|
||||
status: shouldMarkCompleted ? 'COMPLETED' : 'CURRENT',
|
||||
},
|
||||
@@ -413,7 +436,7 @@ export async function updateAnilistPostWatchProgress(
|
||||
return {
|
||||
status: 'updated',
|
||||
message: shouldMarkCompleted
|
||||
? `AniList updated "${pickedTitle}" to episode ${episode} and marked it completed.`
|
||||
: `AniList updated "${pickedTitle}" to episode ${episode}.`,
|
||||
? `AniList updated "${picked.title}" to episode ${episode} and marked it completed.`
|
||||
: `AniList updated "${picked.title}" to episode ${episode}.`,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -201,45 +201,17 @@ test('fetchIfMissing uses guessit primary title and season when available', asyn
|
||||
});
|
||||
|
||||
const searchCalls: Array<{ search: string }> = [];
|
||||
const relationCalls: number[] = [];
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const raw = (init?.body as string | undefined) ?? '';
|
||||
const payload = JSON.parse(raw) as { variables: { search?: string; id?: number } };
|
||||
const payload = JSON.parse(raw) as { variables: { search: string } };
|
||||
const search = payload.variables.search;
|
||||
searchCalls.push({ search });
|
||||
|
||||
if (typeof payload.variables.id === 'number') {
|
||||
relationCalls.push(payload.variables.id);
|
||||
return Promise.resolve(
|
||||
createJsonResponse({
|
||||
data: {
|
||||
Media: {
|
||||
relations: {
|
||||
edges: [
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: {
|
||||
id: 20,
|
||||
type: 'ANIME',
|
||||
episodes: 25,
|
||||
format: 'TV',
|
||||
seasonYear: 2017,
|
||||
coverImage: { large: 'https://images.test/cover-s2.jpg', medium: null },
|
||||
title: {
|
||||
romaji: 'Little Witch Academia 2',
|
||||
english: 'Little Witch Academia 2',
|
||||
native: null,
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
if (search.includes('Season 2')) {
|
||||
return Promise.resolve(createJsonResponse({ data: { Page: { media: [] } } }));
|
||||
}
|
||||
|
||||
searchCalls.push({ search: String(payload.variables.search) });
|
||||
return Promise.resolve(
|
||||
createJsonResponse({
|
||||
data: {
|
||||
@@ -248,8 +220,6 @@ test('fetchIfMissing uses guessit primary title and season when available', asyn
|
||||
{
|
||||
id: 19,
|
||||
episodes: 24,
|
||||
format: 'TV',
|
||||
seasonYear: 2013,
|
||||
coverImage: { large: 'https://images.test/cover.jpg', medium: null },
|
||||
title: {
|
||||
romaji: 'Little Witch Academia',
|
||||
@@ -281,11 +251,9 @@ test('fetchIfMissing uses guessit primary title and season when available', asyn
|
||||
const stored = getCoverArt(db, videoId);
|
||||
|
||||
assert.equal(fetched, true);
|
||||
// One search on the bare title, then a sequel hop to reach season 2.
|
||||
assert.equal(searchCalls.length, 1);
|
||||
assert.equal(searchCalls[0]!.search, 'Little Witch Academia');
|
||||
assert.deepEqual(relationCalls, [19]);
|
||||
assert.equal(stored?.anilistId, 20);
|
||||
assert.equal(searchCalls.length, 2);
|
||||
assert.equal(searchCalls[0]!.search, 'Little Witch Academia Season 2');
|
||||
assert.equal(stored?.anilistId, 19);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
db.close();
|
||||
@@ -356,187 +324,3 @@ test('fetchIfMissing falls back to internal parser when guessit throws', async (
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('fetchIfMissing caches a no-match when the season cannot be resolved', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cover-fetcher-unresolved.mkv', {
|
||||
canonicalTitle: 'Unresolved Show (2013) - S03E01 - Something [1080p].mkv',
|
||||
sourcePath: '/tmp/cover-fetcher-unresolved.mkv',
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
sourceUrl: null,
|
||||
});
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url;
|
||||
if (!url.includes('graphql')) {
|
||||
return Promise.resolve(
|
||||
new Response(Buffer.from('01020304'), {
|
||||
status: 200,
|
||||
headers: { 'content-type': 'image/png' },
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
const payload = JSON.parse(String(init?.body ?? '{}')) as {
|
||||
variables?: { search?: string; id?: number };
|
||||
};
|
||||
if (typeof payload.variables?.id === 'number') {
|
||||
// No sequel edges, so season 3 cannot be reached from the season 1 anchor.
|
||||
return Promise.resolve(createJsonResponse({ data: { Media: { relations: { edges: [] } } } }));
|
||||
}
|
||||
return Promise.resolve(
|
||||
createJsonResponse({
|
||||
data: {
|
||||
Page: {
|
||||
media: [
|
||||
{
|
||||
id: 55,
|
||||
episodes: 13,
|
||||
format: 'TV',
|
||||
seasonYear: 2013,
|
||||
coverImage: { large: 'https://images.test/s1.jpg', medium: null },
|
||||
title: { romaji: 'Unresolved Show', english: 'Unresolved Show', native: null },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
const fetcher = createCoverArtFetcher(
|
||||
{ acquire: async () => {}, recordResponse: () => {} },
|
||||
console,
|
||||
{
|
||||
runGuessit: async () =>
|
||||
JSON.stringify({ title: 'Unresolved Show', season: 3, episode: 1, year: 2013 }),
|
||||
},
|
||||
);
|
||||
|
||||
const fetched = await fetcher.fetchIfMissing(db, videoId, 'Unresolved Show');
|
||||
const stored = getCoverArt(db, videoId);
|
||||
|
||||
assert.equal(fetched, false);
|
||||
// Storing the season 1 artwork would leave a blob with no AniList id, which the
|
||||
// `existing.coverBlob` early return serves forever - the season could then never
|
||||
// re-resolve. A plain no-match keeps the existing retry window in play instead.
|
||||
assert.equal(stored?.coverBlob, null);
|
||||
assert.equal(stored?.coverUrl, null);
|
||||
assert.equal(stored?.anilistId, null);
|
||||
assert.equal(stored?.episodesTotal, null);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('fetchIfMissing re-resolves an unresolved season once AniList publishes the relation', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cover-fetcher-recovers.mkv', {
|
||||
canonicalTitle: 'Recovering Show (2013) - S02E01 - Something [1080p].mkv',
|
||||
sourcePath: '/tmp/cover-fetcher-recovers.mkv',
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
sourceUrl: null,
|
||||
});
|
||||
|
||||
let sequelPublished = false;
|
||||
const originalFetch = globalThis.fetch;
|
||||
const originalNow = Date.now;
|
||||
globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url;
|
||||
if (!url.includes('graphql')) {
|
||||
return Promise.resolve(
|
||||
new Response(Buffer.from('01020304'), {
|
||||
status: 200,
|
||||
headers: { 'content-type': 'image/png' },
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
const payload = JSON.parse(String(init?.body ?? '{}')) as {
|
||||
variables?: { search?: string; id?: number };
|
||||
};
|
||||
if (typeof payload.variables?.id === 'number') {
|
||||
return Promise.resolve(
|
||||
createJsonResponse({
|
||||
data: {
|
||||
Media: {
|
||||
relations: {
|
||||
edges: sequelPublished
|
||||
? [
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: {
|
||||
id: 66,
|
||||
type: 'ANIME',
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
seasonYear: 2015,
|
||||
coverImage: { large: 'https://images.test/s2.jpg', medium: null },
|
||||
title: { romaji: 'Recovering Show 2', english: null, native: null },
|
||||
},
|
||||
},
|
||||
]
|
||||
: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
return Promise.resolve(
|
||||
createJsonResponse({
|
||||
data: {
|
||||
Page: {
|
||||
media: [
|
||||
{
|
||||
id: 65,
|
||||
episodes: 13,
|
||||
format: 'TV',
|
||||
seasonYear: 2013,
|
||||
coverImage: { large: 'https://images.test/s1.jpg', medium: null },
|
||||
title: { romaji: 'Recovering Show', english: null, native: null },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
const fetcher = createCoverArtFetcher(
|
||||
{ acquire: async () => {}, recordResponse: () => {} },
|
||||
console,
|
||||
{
|
||||
runGuessit: async () =>
|
||||
JSON.stringify({ title: 'Recovering Show', season: 2, episode: 1, year: 2013 }),
|
||||
},
|
||||
);
|
||||
|
||||
assert.equal(await fetcher.fetchIfMissing(db, videoId, 'Recovering Show'), false);
|
||||
assert.equal(getCoverArt(db, videoId)?.anilistId, null);
|
||||
|
||||
// AniList publishes the sequel relation, and the no-match retry window elapses.
|
||||
sequelPublished = true;
|
||||
const base = originalNow();
|
||||
Date.now = () => base + 10 * 60 * 1000;
|
||||
|
||||
assert.equal(await fetcher.fetchIfMissing(db, videoId, 'Recovering Show'), true);
|
||||
const recovered = getCoverArt(db, videoId);
|
||||
assert.equal(recovered?.anilistId, 66);
|
||||
assert.equal(recovered?.episodesTotal, 12);
|
||||
} finally {
|
||||
Date.now = originalNow;
|
||||
globalThis.fetch = originalFetch;
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -11,15 +11,43 @@ import {
|
||||
runGuessit,
|
||||
type GuessAnilistMediaInfoDeps,
|
||||
} from './anilist-updater';
|
||||
import {
|
||||
resolveAnilistSeasonMedia,
|
||||
type AnilistQueryExecutor,
|
||||
type AnilistSeasonResolution,
|
||||
} from './season-resolver';
|
||||
|
||||
const ANILIST_GRAPHQL_URL = 'https://graphql.anilist.co';
|
||||
const NO_MATCH_RETRY_MS = 5 * 60 * 1000;
|
||||
|
||||
const SEARCH_QUERY = `
|
||||
query ($search: String!) {
|
||||
Page(perPage: 5) {
|
||||
media(search: $search, type: ANIME) {
|
||||
id
|
||||
episodes
|
||||
season
|
||||
seasonYear
|
||||
coverImage { large medium }
|
||||
title { romaji english native }
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
interface AnilistMedia {
|
||||
id: number;
|
||||
episodes: number | null;
|
||||
season: string | null;
|
||||
seasonYear: number | null;
|
||||
coverImage: { large: string | null; medium: string | null } | null;
|
||||
title: { romaji: string | null; english: string | null; native: string | null } | null;
|
||||
}
|
||||
|
||||
interface AnilistSearchResponse {
|
||||
data?: {
|
||||
Page?: {
|
||||
media?: AnilistMedia[];
|
||||
};
|
||||
};
|
||||
errors?: Array<{ message?: string }>;
|
||||
}
|
||||
|
||||
export interface CoverArtFetcher {
|
||||
fetchIfMissing(db: DatabaseSync, videoId: number, canonicalTitle: string): Promise<boolean>;
|
||||
}
|
||||
@@ -71,45 +99,151 @@ export function stripFilenameTags(raw: string): string {
|
||||
return title.trim().replace(/\s{2,}/g, ' ');
|
||||
}
|
||||
|
||||
class AnilistRateLimitedError extends Error {
|
||||
constructor() {
|
||||
super('Anilist rate limit reached');
|
||||
this.name = 'AnilistRateLimitedError';
|
||||
}
|
||||
function removeSeasonHint(title: string): string {
|
||||
return title
|
||||
.replace(/\bseason\s*\d+\b/gi, '')
|
||||
.replace(/\s{2,}/g, ' ')
|
||||
.trim();
|
||||
}
|
||||
|
||||
async function executeAnilistQuery<T>(
|
||||
function normalizeTitle(text: string): string {
|
||||
return text.trim().toLowerCase().replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
function extractCandidateSeasonHints(text: string): Set<number> {
|
||||
const normalized = normalizeTitle(text);
|
||||
const matches = [
|
||||
...normalized.matchAll(/\bseason\s*(\d{1,2})\b/gi),
|
||||
...normalized.matchAll(/\bs(\d{1,2})(?:\b|\D)/gi),
|
||||
];
|
||||
const values = new Set<number>();
|
||||
for (const match of matches) {
|
||||
const value = Number.parseInt(match[1]!, 10);
|
||||
if (Number.isInteger(value)) {
|
||||
values.add(value);
|
||||
}
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
function isSeasonMentioned(titles: string[], season: number | null): boolean {
|
||||
if (!season) {
|
||||
return false;
|
||||
}
|
||||
const hints = titles.flatMap((title) => [...extractCandidateSeasonHints(title)]);
|
||||
return hints.includes(season);
|
||||
}
|
||||
|
||||
function pickBestSearchResult(
|
||||
title: string,
|
||||
episode: number | null,
|
||||
season: number | null,
|
||||
media: AnilistMedia[],
|
||||
): { id: number; title: string } | null {
|
||||
const cleanedTitle = removeSeasonHint(title);
|
||||
const targets = [title, cleanedTitle]
|
||||
.map(normalizeTitle)
|
||||
.map((value) => value.trim())
|
||||
.filter((value, index, all) => value.length > 0 && all.indexOf(value) === index);
|
||||
|
||||
const filtered =
|
||||
episode === null
|
||||
? media
|
||||
: media.filter((item) => {
|
||||
const total = item.episodes;
|
||||
return total === null || total >= episode;
|
||||
});
|
||||
const candidates = filtered.length > 0 ? filtered : media;
|
||||
if (candidates.length === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const scored = candidates.map((item) => {
|
||||
const candidateTitles = [item.title?.romaji, item.title?.english, item.title?.native]
|
||||
.filter((value): value is string => typeof value === 'string')
|
||||
.map((value) => normalizeTitle(value));
|
||||
|
||||
let score = 0;
|
||||
|
||||
for (const target of targets) {
|
||||
if (candidateTitles.includes(target)) {
|
||||
score += 120;
|
||||
continue;
|
||||
}
|
||||
if (candidateTitles.some((itemTitle) => itemTitle.includes(target))) {
|
||||
score += 30;
|
||||
}
|
||||
if (candidateTitles.some((itemTitle) => target.includes(itemTitle))) {
|
||||
score += 10;
|
||||
}
|
||||
}
|
||||
|
||||
if (episode !== null && item.episodes === episode) {
|
||||
score += 20;
|
||||
}
|
||||
|
||||
if (season !== null && isSeasonMentioned(candidateTitles, season)) {
|
||||
score += 15;
|
||||
}
|
||||
|
||||
return { item, score };
|
||||
});
|
||||
|
||||
scored.sort((a, b) => {
|
||||
if (b.score !== a.score) return b.score - a.score;
|
||||
return b.item.id - a.item.id;
|
||||
});
|
||||
|
||||
const selected = scored[0]!;
|
||||
const selectedTitle =
|
||||
selected.item.title?.english ??
|
||||
selected.item.title?.romaji ??
|
||||
selected.item.title?.native ??
|
||||
title;
|
||||
return { id: selected.item.id, title: selectedTitle };
|
||||
}
|
||||
|
||||
function buildSearchCandidates(parsed: CoverArtCandidate): string[] {
|
||||
const candidateTitles = [
|
||||
...(parsed.source === 'guessit' && parsed.season !== null && parsed.season > 1
|
||||
? [`${parsed.title} Season ${parsed.season}`]
|
||||
: []),
|
||||
parsed.title,
|
||||
];
|
||||
return candidateTitles
|
||||
.map((title) => title.trim())
|
||||
.filter((title, index, all) => title.length > 0 && all.indexOf(title) === index);
|
||||
}
|
||||
|
||||
async function searchAnilist(
|
||||
rateLimiter: AnilistRateLimiter,
|
||||
query: string,
|
||||
variables: Record<string, unknown>,
|
||||
): Promise<T> {
|
||||
title: string,
|
||||
): Promise<{ media: AnilistMedia[]; rateLimited: boolean }> {
|
||||
await rateLimiter.acquire();
|
||||
|
||||
const res = await fetch(ANILIST_GRAPHQL_URL, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
|
||||
body: JSON.stringify({ query, variables }),
|
||||
body: JSON.stringify({ query: SEARCH_QUERY, variables: { search: title } }),
|
||||
});
|
||||
|
||||
rateLimiter.recordResponse(res.headers);
|
||||
|
||||
if (res.status === 429) {
|
||||
throw new AnilistRateLimitedError();
|
||||
return { media: [], rateLimited: true };
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
throw new Error(`Anilist search failed: ${res.status} ${res.statusText}`);
|
||||
}
|
||||
|
||||
const json = (await res.json()) as { data?: T; errors?: Array<{ message?: string }> };
|
||||
const firstError = json.errors?.find((entry) => Boolean(entry?.message));
|
||||
if (firstError?.message) {
|
||||
throw new Error(firstError.message);
|
||||
const json = (await res.json()) as AnilistSearchResponse;
|
||||
const mediaList = json.data?.Page?.media;
|
||||
if (!mediaList || mediaList.length === 0) {
|
||||
return { media: [], rateLimited: false };
|
||||
}
|
||||
if (!json.data) {
|
||||
throw new Error('Anilist response missing data');
|
||||
}
|
||||
return json.data;
|
||||
|
||||
return { media: mediaList, rateLimited: false };
|
||||
}
|
||||
|
||||
async function downloadImage(url: string): Promise<Buffer | null> {
|
||||
@@ -242,57 +376,49 @@ export function createCoverArtFetcher(
|
||||
|
||||
const parsedInfo = await resolveMediaInfo(db, videoId, canonicalTitle);
|
||||
const searchBase = parsedInfo?.title ?? cleaned;
|
||||
const searchTitles = searchBase === cleaned ? [searchBase] : ([searchBase, cleaned] as const);
|
||||
const searchCandidates = parsedInfo ? buildSearchCandidates(parsedInfo) : [cleaned];
|
||||
|
||||
const execute: AnilistQueryExecutor = (query, variables) =>
|
||||
executeAnilistQuery(rateLimiter, query, variables);
|
||||
const effectiveCandidates = searchCandidates.includes(cleaned)
|
||||
? searchCandidates
|
||||
: [...searchCandidates, cleaned];
|
||||
|
||||
let resolution: AnilistSeasonResolution | null = null;
|
||||
try {
|
||||
for (const searchTitle of searchTitles) {
|
||||
logger.info('cover-art: searching Anilist for "%s" (videoId=%d)', searchTitle, videoId);
|
||||
resolution = await resolveAnilistSeasonMedia(
|
||||
{
|
||||
title: searchTitle,
|
||||
season: parsedInfo?.season ?? null,
|
||||
episode: parsedInfo?.episode ?? null,
|
||||
},
|
||||
{ execute, logInfo: (message) => logger.info('%s', message) },
|
||||
let selected: AnilistMedia | null = null;
|
||||
let rateLimited = false;
|
||||
|
||||
for (const candidate of effectiveCandidates) {
|
||||
logger.info('cover-art: searching Anilist for "%s" (videoId=%d)', candidate, videoId);
|
||||
|
||||
try {
|
||||
const result = await searchAnilist(rateLimiter, candidate);
|
||||
rateLimited = result.rateLimited;
|
||||
if (result.media.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const picked = pickBestSearchResult(
|
||||
searchBase,
|
||||
parsedInfo?.episode ?? null,
|
||||
parsedInfo?.season ?? null,
|
||||
result.media,
|
||||
);
|
||||
if (resolution) break;
|
||||
}
|
||||
} catch (err) {
|
||||
if (err instanceof AnilistRateLimitedError) {
|
||||
logger.warn('cover-art: rate-limited by Anilist, skipping videoId=%d', videoId);
|
||||
if (picked) {
|
||||
const match = result.media.find((media) => media.id === picked.id);
|
||||
if (match) {
|
||||
selected = match;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error('cover-art: Anilist search error for "%s": %s', candidate, err);
|
||||
return false;
|
||||
}
|
||||
logger.error('cover-art: Anilist search error for "%s": %s', searchBase, err);
|
||||
}
|
||||
|
||||
if (rateLimited) {
|
||||
logger.warn('cover-art: rate-limited by Anilist, skipping videoId=%d', videoId);
|
||||
return false;
|
||||
}
|
||||
|
||||
if (resolution && !resolution.seasonResolved) {
|
||||
// Only the season 1 entry was found. Storing its artwork would leave a cover with
|
||||
// no AniList id, which the `existing.coverBlob` early return above serves forever,
|
||||
// so the season could never re-resolve once AniList publishes the relation.
|
||||
// Caching a plain no-match instead reuses the NO_MATCH_RETRY_MS retry window.
|
||||
logger.warn(
|
||||
'cover-art: could not find season %d of "%s" (only matched "%s"), caching no-match',
|
||||
resolution.requestedSeason,
|
||||
searchBase,
|
||||
resolution.title,
|
||||
);
|
||||
upsertCoverArt(db, videoId, {
|
||||
anilistId: null,
|
||||
coverUrl: null,
|
||||
coverBlob: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
episodesTotal: null,
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
const selected = resolution?.media ?? null;
|
||||
if (!selected) {
|
||||
logger.info('cover-art: no Anilist results for "%s", caching no-match', searchBase);
|
||||
upsertCoverArt(db, videoId, {
|
||||
|
||||
@@ -1,307 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
import {
|
||||
ANILIST_SEASON_RELATIONS_QUERY,
|
||||
ANILIST_SEASON_SEARCH_QUERY,
|
||||
pickByAirOrder,
|
||||
resolveAnilistSeasonMedia,
|
||||
stripSeasonSuffix,
|
||||
type AnilistSeasonMedia,
|
||||
} from './season-resolver';
|
||||
|
||||
/** Real AniList payloads for "My Teen Romantic Comedy SNAFU", trimmed to used fields. */
|
||||
const OREGAIRU_SEARCH: AnilistSeasonMedia[] = [
|
||||
{
|
||||
id: 14813,
|
||||
episodes: 13,
|
||||
format: 'TV',
|
||||
seasonYear: 2013,
|
||||
title: {
|
||||
romaji: 'Yahari Ore no Seishun Love Come wa Machigatteiru.',
|
||||
english: 'My Teen Romantic Comedy SNAFU',
|
||||
native: null,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 18753,
|
||||
episodes: 1,
|
||||
format: 'OVA',
|
||||
seasonYear: 2013,
|
||||
title: { romaji: null, english: 'My Teen Romantic Comedy SNAFU OVA', native: null },
|
||||
},
|
||||
{
|
||||
id: 108489,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
seasonYear: 2020,
|
||||
title: {
|
||||
romaji: 'Yahari Ore no Seishun Love Come wa Machigatteiru. Kan',
|
||||
english: 'My Teen Romantic Comedy SNAFU Climax!',
|
||||
native: null,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 20698,
|
||||
episodes: 13,
|
||||
format: 'TV',
|
||||
seasonYear: 2015,
|
||||
title: {
|
||||
romaji: 'Yahari Ore no Seishun Love Come wa Machigatteiru. Zoku',
|
||||
english: 'My Teen Romantic Comedy SNAFU TOO!',
|
||||
native: null,
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
const OREGAIRU_RELATIONS: Record<
|
||||
number,
|
||||
Array<{ relationType: string; node: AnilistSeasonMedia }>
|
||||
> = {
|
||||
14813: [
|
||||
{
|
||||
relationType: 'SIDE_STORY',
|
||||
node: OREGAIRU_SEARCH[1]!,
|
||||
},
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: OREGAIRU_SEARCH[3]!,
|
||||
},
|
||||
],
|
||||
20698: [
|
||||
{
|
||||
relationType: 'PREQUEL',
|
||||
node: OREGAIRU_SEARCH[0]!,
|
||||
},
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: OREGAIRU_SEARCH[2]!,
|
||||
},
|
||||
],
|
||||
108489: [
|
||||
{
|
||||
relationType: 'PREQUEL',
|
||||
node: OREGAIRU_SEARCH[3]!,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
function createExecutor(
|
||||
search: AnilistSeasonMedia[],
|
||||
relations: Record<number, Array<{ relationType: string; node: AnilistSeasonMedia }>> = {},
|
||||
) {
|
||||
const searches: string[] = [];
|
||||
const relationLookups: number[] = [];
|
||||
const execute = async <T>(query: string, variables: Record<string, unknown>): Promise<T> => {
|
||||
if (query === ANILIST_SEASON_SEARCH_QUERY) {
|
||||
searches.push(String(variables.search));
|
||||
return { Page: { media: search } } as T;
|
||||
}
|
||||
if (query === ANILIST_SEASON_RELATIONS_QUERY) {
|
||||
const id = Number(variables.id);
|
||||
relationLookups.push(id);
|
||||
return {
|
||||
Media: {
|
||||
relations: {
|
||||
edges: (relations[id] ?? []).map((edge) => ({
|
||||
relationType: edge.relationType,
|
||||
node: { ...edge.node, type: 'ANIME' },
|
||||
})),
|
||||
},
|
||||
},
|
||||
} as T;
|
||||
}
|
||||
throw new Error(`unexpected query: ${query}`);
|
||||
};
|
||||
return { execute, searches, relationLookups };
|
||||
}
|
||||
|
||||
test('stripSeasonSuffix drops release-name season markers', () => {
|
||||
assert.equal(stripSeasonSuffix('Some Show Season 3'), 'Some Show');
|
||||
assert.equal(stripSeasonSuffix('Some Show S3'), 'Some Show');
|
||||
assert.equal(stripSeasonSuffix('Some Show 2nd Season'), 'Some Show');
|
||||
assert.equal(stripSeasonSuffix('Some Show'), 'Some Show');
|
||||
});
|
||||
|
||||
test('resolves season 3 by walking sequel relations', async () => {
|
||||
const { execute, searches, relationLookups } = createExecutor(
|
||||
OREGAIRU_SEARCH,
|
||||
OREGAIRU_RELATIONS,
|
||||
);
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 3, episode: 1 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 108489);
|
||||
assert.equal(result?.title, 'My Teen Romantic Comedy SNAFU Climax!');
|
||||
assert.equal(result?.episodes, 12);
|
||||
assert.equal(result?.seasonResolved, true);
|
||||
assert.equal(result?.via, 'sequel-chain');
|
||||
// Searches the bare title only; "Season 3" returns nothing on AniList.
|
||||
assert.deepEqual(searches, ['My Teen Romantic Comedy SNAFU']);
|
||||
assert.deepEqual(relationLookups, [14813, 20698]);
|
||||
});
|
||||
|
||||
test('season 1 resolves to the anchor without relation lookups', async () => {
|
||||
const { execute, relationLookups } = createExecutor(OREGAIRU_SEARCH, OREGAIRU_RELATIONS);
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 1, episode: 5 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 14813);
|
||||
assert.equal(result?.seasonResolved, true);
|
||||
assert.equal(result?.via, 'anchor');
|
||||
assert.deepEqual(relationLookups, []);
|
||||
});
|
||||
|
||||
test('strips a season marker already present in the parsed title', async () => {
|
||||
const { execute, searches } = createExecutor(OREGAIRU_SEARCH, OREGAIRU_RELATIONS);
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU Season 2', season: 2, episode: 3 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.deepEqual(searches, ['My Teen Romantic Comedy SNAFU']);
|
||||
assert.equal(result?.id, 20698);
|
||||
assert.equal(result?.seasonResolved, true);
|
||||
});
|
||||
|
||||
test('falls back to air order when the sequel chain is broken', async () => {
|
||||
const { execute } = createExecutor(OREGAIRU_SEARCH, {});
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 3, episode: 1 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 108489);
|
||||
assert.equal(result?.via, 'air-order');
|
||||
assert.equal(result?.seasonResolved, true);
|
||||
});
|
||||
|
||||
test('reports seasonResolved false rather than silently using season 1', async () => {
|
||||
const { execute } = createExecutor([OREGAIRU_SEARCH[0]!], {});
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 3, episode: 1 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 14813);
|
||||
assert.equal(result?.seasonResolved, false);
|
||||
assert.equal(result?.via, 'anchor');
|
||||
assert.equal(result?.requestedSeason, 3);
|
||||
});
|
||||
|
||||
test('does not filter the anchor by an episode count the season 1 entry cannot have', async () => {
|
||||
const { execute } = createExecutor(OREGAIRU_SEARCH, OREGAIRU_RELATIONS);
|
||||
// Season 2 episode 13 exceeds nothing, but season 3 episode 13 exceeds Climax!'s 12.
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 2, episode: 13 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 20698);
|
||||
assert.equal(result?.seasonResolved, true);
|
||||
});
|
||||
|
||||
test('prefers the TV sequel when a franchise branches into other formats', async () => {
|
||||
const branching: Record<number, Array<{ relationType: string; node: AnilistSeasonMedia }>> = {
|
||||
1: [
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: {
|
||||
id: 3,
|
||||
episodes: 6,
|
||||
format: 'ONA',
|
||||
seasonYear: 2019,
|
||||
title: { english: 'Spinoff' },
|
||||
},
|
||||
},
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: { id: 2, episodes: 12, format: 'TV', seasonYear: 2020, title: { english: 'Show 2' } },
|
||||
},
|
||||
],
|
||||
};
|
||||
const { execute } = createExecutor(
|
||||
[{ id: 1, episodes: 12, format: 'TV', seasonYear: 2018, title: { english: 'Show' } }],
|
||||
branching,
|
||||
);
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Show', season: 2 }, { execute });
|
||||
|
||||
assert.equal(result?.id, 2);
|
||||
assert.equal(result?.via, 'sequel-chain');
|
||||
});
|
||||
|
||||
test('air-order fallback refuses when the anchor is not the earliest entry', () => {
|
||||
const anchor: AnilistSeasonMedia = {
|
||||
id: 2,
|
||||
format: 'TV',
|
||||
seasonYear: 2020,
|
||||
title: { english: 'Show Later' },
|
||||
};
|
||||
const picked = pickByAirOrder(anchor, 2, [
|
||||
anchor,
|
||||
{ id: 1, format: 'TV', seasonYear: 2015, title: { english: 'Show Earlier' } },
|
||||
]);
|
||||
|
||||
assert.equal(picked, null);
|
||||
});
|
||||
|
||||
test('returns null when the search yields nothing', async () => {
|
||||
const { execute } = createExecutor([]);
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Nothing', season: 2 }, { execute });
|
||||
|
||||
assert.equal(result, null);
|
||||
});
|
||||
|
||||
test('refuses a season beyond the sequel-hop cap instead of returning a partial walk', async () => {
|
||||
// A 30-entry sequel chain: hopping the capped number of times would land on the wrong
|
||||
// season and report it resolved, so the walk must decline outright.
|
||||
const chain: Record<number, Array<{ relationType: string; node: AnilistSeasonMedia }>> = {};
|
||||
for (let id = 1; id < 30; id += 1) {
|
||||
chain[id] = [
|
||||
{
|
||||
relationType: 'SEQUEL',
|
||||
node: {
|
||||
id: id + 1,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
seasonYear: 2000 + id,
|
||||
title: { english: `Show ${id + 1}` },
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
const { execute, relationLookups } = createExecutor(
|
||||
[{ id: 1, episodes: 12, format: 'TV', seasonYear: 2000, title: { english: 'Show' } }],
|
||||
chain,
|
||||
);
|
||||
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Show', season: 25 }, { execute });
|
||||
|
||||
assert.equal(result?.id, 1);
|
||||
assert.equal(result?.seasonResolved, false);
|
||||
assert.equal(result?.via, 'anchor');
|
||||
// Declines before spending any relation requests.
|
||||
assert.deepEqual(relationLookups, []);
|
||||
});
|
||||
|
||||
test('air-order fallback declines when a franchise entry has no air year', async () => {
|
||||
// The middle season has no year, so ordering it would shift every later season by one.
|
||||
const { execute } = createExecutor(
|
||||
[
|
||||
{ id: 1, episodes: 12, format: 'TV', seasonYear: 2013, title: { english: 'Show' } },
|
||||
{ id: 2, episodes: 12, format: 'TV', title: { english: 'Show Zoku' } },
|
||||
{ id: 3, episodes: 12, format: 'TV', seasonYear: 2020, title: { english: 'Show Kan' } },
|
||||
],
|
||||
{},
|
||||
);
|
||||
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Show', season: 2 }, { execute });
|
||||
|
||||
assert.equal(result?.id, 1);
|
||||
assert.equal(result?.seasonResolved, false);
|
||||
assert.equal(result?.via, 'anchor');
|
||||
});
|
||||
@@ -1,407 +0,0 @@
|
||||
/**
|
||||
* AniList has no concept of "season N": sequels are separate media with their own
|
||||
* titles (Zoku, Kan, 2nd Season, ...). Searching "<title> Season 3" therefore returns
|
||||
* nothing and callers silently fall back to the season 1 entry.
|
||||
*
|
||||
* This module resolves a parsed (title, season) pair to the right media by locating a
|
||||
* season 1 anchor and then walking AniList SEQUEL relations, with an air-order fallback
|
||||
* when the relation chain is incomplete. When the seasonal entry cannot be located it
|
||||
* reports `seasonResolved: false` so callers can refuse to act instead of guessing.
|
||||
*/
|
||||
|
||||
export interface AnilistSeasonMediaTitle {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
native?: string | null;
|
||||
}
|
||||
|
||||
export interface AnilistSeasonMedia {
|
||||
id: number;
|
||||
episodes?: number | null;
|
||||
format?: string | null;
|
||||
seasonYear?: number | null;
|
||||
startDate?: { year?: number | null } | null;
|
||||
synonyms?: Array<string | null> | null;
|
||||
coverImage?: { large?: string | null; medium?: string | null } | null;
|
||||
title?: AnilistSeasonMediaTitle | null;
|
||||
}
|
||||
|
||||
export type AnilistQueryExecutor = <T>(
|
||||
query: string,
|
||||
variables: Record<string, unknown>,
|
||||
) => Promise<T>;
|
||||
|
||||
export type AnilistSeasonResolutionVia = 'anchor' | 'sequel-chain' | 'air-order';
|
||||
|
||||
export interface AnilistSeasonResolution {
|
||||
id: number;
|
||||
title: string;
|
||||
episodes: number | null;
|
||||
media: AnilistSeasonMedia;
|
||||
/** False when a season >= 2 was requested but no seasonal entry could be located. */
|
||||
seasonResolved: boolean;
|
||||
requestedSeason: number | null;
|
||||
via: AnilistSeasonResolutionVia;
|
||||
}
|
||||
|
||||
export interface ResolveAnilistSeasonMediaInput {
|
||||
title: string;
|
||||
season?: number | null;
|
||||
episode?: number | null;
|
||||
}
|
||||
|
||||
export interface ResolveAnilistSeasonMediaDeps {
|
||||
execute: AnilistQueryExecutor;
|
||||
logInfo?: (message: string) => void;
|
||||
}
|
||||
|
||||
const MEDIA_FIELDS = `
|
||||
id
|
||||
episodes
|
||||
format
|
||||
seasonYear
|
||||
startDate { year }
|
||||
synonyms
|
||||
coverImage { large medium }
|
||||
title { romaji english native }
|
||||
`;
|
||||
|
||||
export const ANILIST_SEASON_SEARCH_QUERY = `
|
||||
query ($search: String!) {
|
||||
Page(perPage: 10) {
|
||||
media(search: $search, type: ANIME, sort: [SEARCH_MATCH, POPULARITY_DESC]) {
|
||||
${MEDIA_FIELDS}
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
export const ANILIST_SEASON_RELATIONS_QUERY = `
|
||||
query ($id: Int!) {
|
||||
Media(id: $id, type: ANIME) {
|
||||
id
|
||||
relations {
|
||||
edges {
|
||||
relationType
|
||||
node {
|
||||
type
|
||||
${MEDIA_FIELDS}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
interface AnilistSeasonSearchResponse {
|
||||
Page?: {
|
||||
media?: AnilistSeasonMedia[] | null;
|
||||
} | null;
|
||||
}
|
||||
|
||||
interface AnilistSeasonRelationsResponse {
|
||||
Media?: {
|
||||
relations?: {
|
||||
edges?: Array<{
|
||||
relationType?: string | null;
|
||||
node?: (AnilistSeasonMedia & { type?: string | null }) | null;
|
||||
} | null> | null;
|
||||
} | null;
|
||||
} | null;
|
||||
}
|
||||
|
||||
/** Formats that can carry a numbered TV season, best first. */
|
||||
const SEASONAL_FORMAT_PRIORITY = ['TV', 'TV_SHORT', 'ONA'];
|
||||
|
||||
const MAX_SEQUEL_HOPS = 12;
|
||||
|
||||
function normalizeTitle(value: string): string {
|
||||
return value.trim().toLowerCase().replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops season markers a release name carries but AniList titles never do,
|
||||
* so "Some Show Season 3" and "Some Show S3" both search as "Some Show".
|
||||
*/
|
||||
export function stripSeasonSuffix(title: string): string {
|
||||
return title
|
||||
.replace(/\bseason\s*\d{1,2}\b/gi, ' ')
|
||||
.replace(/\b\d{1,2}(?:st|nd|rd|th)\s+season\b/gi, ' ')
|
||||
.replace(/\bs\d{1,2}\b/gi, ' ')
|
||||
.replace(/\s{2,}/g, ' ')
|
||||
.trim();
|
||||
}
|
||||
|
||||
function mediaTitles(media: AnilistSeasonMedia): string[] {
|
||||
const synonyms = Array.isArray(media.synonyms) ? media.synonyms : [];
|
||||
return [media.title?.english, media.title?.romaji, media.title?.native, ...synonyms]
|
||||
.filter((value): value is string => typeof value === 'string' && value.trim().length > 0)
|
||||
.map((value) => normalizeTitle(value));
|
||||
}
|
||||
|
||||
function displayTitle(media: AnilistSeasonMedia, fallback: string): string {
|
||||
return (
|
||||
media.title?.english?.trim() ||
|
||||
media.title?.romaji?.trim() ||
|
||||
media.title?.native?.trim() ||
|
||||
fallback.trim()
|
||||
);
|
||||
}
|
||||
|
||||
function episodeCount(media: AnilistSeasonMedia): number | null {
|
||||
return typeof media.episodes === 'number' && media.episodes > 0 ? media.episodes : null;
|
||||
}
|
||||
|
||||
function airYear(media: AnilistSeasonMedia): number | null {
|
||||
if (typeof media.seasonYear === 'number' && media.seasonYear > 0) {
|
||||
return media.seasonYear;
|
||||
}
|
||||
const startYear = media.startDate?.year;
|
||||
return typeof startYear === 'number' && startYear > 0 ? startYear : null;
|
||||
}
|
||||
|
||||
function isSeasonalFormat(media: AnilistSeasonMedia): boolean {
|
||||
const format = (media.format || '').toUpperCase();
|
||||
return SEASONAL_FORMAT_PRIORITY.includes(format);
|
||||
}
|
||||
|
||||
function formatRank(media: AnilistSeasonMedia): number {
|
||||
const index = SEASONAL_FORMAT_PRIORITY.indexOf((media.format || '').toUpperCase());
|
||||
return index < 0 ? SEASONAL_FORMAT_PRIORITY.length : index;
|
||||
}
|
||||
|
||||
function toResolution(
|
||||
media: AnilistSeasonMedia,
|
||||
fallbackTitle: string,
|
||||
season: number | null,
|
||||
via: AnilistSeasonResolutionVia,
|
||||
seasonResolved: boolean,
|
||||
): AnilistSeasonResolution {
|
||||
return {
|
||||
id: media.id,
|
||||
title: displayTitle(media, fallbackTitle),
|
||||
episodes: episodeCount(media),
|
||||
media,
|
||||
seasonResolved,
|
||||
requestedSeason: season,
|
||||
via,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Picks the franchise anchor (season 1 entry) for a search result set. The anchor is
|
||||
* matched on title alone - season is handled by walking relations from here.
|
||||
*/
|
||||
export function pickAnchorMedia(
|
||||
title: string,
|
||||
media: AnilistSeasonMedia[],
|
||||
options: { episode?: number | null } = {},
|
||||
): AnilistSeasonMedia | null {
|
||||
if (media.length === 0) return null;
|
||||
|
||||
const episode = options.episode;
|
||||
const episodeFiltered =
|
||||
typeof episode === 'number' && episode > 0
|
||||
? media.filter((entry) => {
|
||||
const total = episodeCount(entry);
|
||||
return total === null || total >= episode;
|
||||
})
|
||||
: media;
|
||||
const pool = episodeFiltered.length > 0 ? episodeFiltered : media;
|
||||
|
||||
const targets = [normalizeTitle(title), normalizeTitle(stripSeasonSuffix(title))].filter(
|
||||
(value, index, all) => value.length > 0 && all.indexOf(value) === index,
|
||||
);
|
||||
|
||||
const scored = pool.map((entry, index) => {
|
||||
const candidateTitles = mediaTitles(entry);
|
||||
let score = 0;
|
||||
for (const target of targets) {
|
||||
if (candidateTitles.includes(target)) {
|
||||
score += 120;
|
||||
continue;
|
||||
}
|
||||
if (candidateTitles.some((candidate) => candidate.startsWith(target))) {
|
||||
score += 40;
|
||||
} else if (candidateTitles.some((candidate) => candidate.includes(target))) {
|
||||
score += 25;
|
||||
}
|
||||
if (candidateTitles.some((candidate) => target.includes(candidate))) {
|
||||
score += 10;
|
||||
}
|
||||
}
|
||||
if (isSeasonalFormat(entry)) {
|
||||
score += 30;
|
||||
}
|
||||
return { entry, score, index };
|
||||
});
|
||||
|
||||
scored.sort((a, b) => {
|
||||
if (b.score !== a.score) return b.score - a.score;
|
||||
if (a.index !== b.index) return a.index - b.index;
|
||||
return a.entry.id - b.entry.id;
|
||||
});
|
||||
|
||||
return scored[0]?.entry ?? null;
|
||||
}
|
||||
|
||||
async function walkSequelChain(
|
||||
anchor: AnilistSeasonMedia,
|
||||
season: number,
|
||||
deps: ResolveAnilistSeasonMediaDeps,
|
||||
): Promise<AnilistSeasonMedia | null> {
|
||||
// Walking fewer hops than requested would land on the wrong season and report it as
|
||||
// resolved, so refuse instead and let the caller fall through to its guarded fallback.
|
||||
const hops = season - 1;
|
||||
if (hops > MAX_SEQUEL_HOPS) {
|
||||
return null;
|
||||
}
|
||||
const visited = new Set<number>([anchor.id]);
|
||||
let current = anchor;
|
||||
|
||||
for (let hop = 0; hop < hops; hop += 1) {
|
||||
// Transport errors propagate: a failed hop must not be mistaken for "no sequel exists".
|
||||
const response = await deps.execute<AnilistSeasonRelationsResponse>(
|
||||
ANILIST_SEASON_RELATIONS_QUERY,
|
||||
{ id: current.id },
|
||||
);
|
||||
|
||||
const sequels = (response.Media?.relations?.edges ?? [])
|
||||
.map((edge) => edge ?? null)
|
||||
.filter(
|
||||
(
|
||||
edge,
|
||||
): edge is {
|
||||
relationType?: string | null;
|
||||
node: AnilistSeasonMedia & { type?: string | null };
|
||||
} =>
|
||||
Boolean(edge?.node) &&
|
||||
(edge?.relationType || '').toUpperCase() === 'SEQUEL' &&
|
||||
(edge?.node?.type || 'ANIME').toUpperCase() === 'ANIME',
|
||||
)
|
||||
.map((edge) => edge.node)
|
||||
.filter((node) => !visited.has(node.id));
|
||||
|
||||
if (sequels.length === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// A franchise can branch (a TV sequel plus an ONA spinoff); prefer the TV line.
|
||||
sequels.sort((a, b) => {
|
||||
const rankDelta = formatRank(a) - formatRank(b);
|
||||
if (rankDelta !== 0) return rankDelta;
|
||||
const yearDelta =
|
||||
(airYear(a) ?? Number.MAX_SAFE_INTEGER) - (airYear(b) ?? Number.MAX_SAFE_INTEGER);
|
||||
if (yearDelta !== 0) return yearDelta;
|
||||
return a.id - b.id;
|
||||
});
|
||||
|
||||
current = sequels[0]!;
|
||||
visited.add(current.id);
|
||||
}
|
||||
|
||||
return current.id === anchor.id ? null : current;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback for franchises whose SEQUEL edges are missing or route through a format we
|
||||
* skipped: order the franchise's seasonal entries by air date and index by season.
|
||||
*/
|
||||
export function pickByAirOrder(
|
||||
anchor: AnilistSeasonMedia,
|
||||
season: number,
|
||||
media: AnilistSeasonMedia[],
|
||||
): AnilistSeasonMedia | null {
|
||||
const anchorTitles = mediaTitles(anchor).map((value) => stripSeasonSuffix(value));
|
||||
const anchorBase = anchorTitles.sort((a, b) => a.length - b.length)[0];
|
||||
if (!anchorBase) return null;
|
||||
|
||||
const franchise = media.filter((entry) => {
|
||||
if (!isSeasonalFormat(entry)) return false;
|
||||
if (entry.id === anchor.id) return true;
|
||||
return mediaTitles(entry).some((candidate) =>
|
||||
stripSeasonSuffix(candidate).includes(anchorBase),
|
||||
);
|
||||
});
|
||||
if (franchise.length < season) return null;
|
||||
|
||||
// Ordering by air date is only meaningful when every entry has one: an unknown year
|
||||
// would sort last and shift the season index while still reporting a resolved match.
|
||||
if (franchise.some((entry) => airYear(entry) === null)) return null;
|
||||
|
||||
const ordered = [...franchise].sort((a, b) => {
|
||||
const yearDelta = (airYear(a) ?? 0) - (airYear(b) ?? 0);
|
||||
if (yearDelta !== 0) return yearDelta;
|
||||
return a.id - b.id;
|
||||
});
|
||||
|
||||
// The anchor has to be the first entry, otherwise this ordering is not a season list.
|
||||
if (ordered[0]?.id !== anchor.id) return null;
|
||||
|
||||
return ordered[season - 1] ?? null;
|
||||
}
|
||||
|
||||
export async function resolveAnilistSeasonMedia(
|
||||
input: ResolveAnilistSeasonMediaInput,
|
||||
deps: ResolveAnilistSeasonMediaDeps,
|
||||
): Promise<AnilistSeasonResolution | null> {
|
||||
const searchTitle = stripSeasonSuffix(input.title).trim() || input.title.trim();
|
||||
if (!searchTitle) return null;
|
||||
|
||||
const season =
|
||||
typeof input.season === 'number' && Number.isInteger(input.season) && input.season > 0
|
||||
? input.season
|
||||
: null;
|
||||
|
||||
const response = await deps.execute<AnilistSeasonSearchResponse>(ANILIST_SEASON_SEARCH_QUERY, {
|
||||
search: searchTitle,
|
||||
});
|
||||
const media = (response.Page?.media ?? []).filter(
|
||||
(entry): entry is AnilistSeasonMedia => Boolean(entry) && typeof entry.id === 'number',
|
||||
);
|
||||
if (media.length === 0) return null;
|
||||
|
||||
// Season 1 (or unknown) resolves against the episode count; later seasons must not,
|
||||
// because the anchor is season 1 and may be shorter than the requested episode.
|
||||
const anchor = pickAnchorMedia(searchTitle, media, {
|
||||
episode: season === null || season <= 1 ? input.episode : null,
|
||||
});
|
||||
if (!anchor) return null;
|
||||
|
||||
if (season === null || season <= 1) {
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', true);
|
||||
}
|
||||
|
||||
let chainError: unknown = null;
|
||||
let viaChain: AnilistSeasonMedia | null = null;
|
||||
try {
|
||||
viaChain = await walkSequelChain(anchor, season, deps);
|
||||
} catch (error) {
|
||||
chainError = error;
|
||||
}
|
||||
if (viaChain) {
|
||||
deps.logInfo?.(
|
||||
`[anilist] season ${season} of "${searchTitle}" resolved via sequel chain: ${displayTitle(viaChain, searchTitle)} (${viaChain.id})`,
|
||||
);
|
||||
return toResolution(viaChain, searchTitle, season, 'sequel-chain', true);
|
||||
}
|
||||
|
||||
const viaAirOrder = pickByAirOrder(anchor, season, media);
|
||||
if (viaAirOrder) {
|
||||
deps.logInfo?.(
|
||||
`[anilist] season ${season} of "${searchTitle}" resolved via air order: ${displayTitle(viaAirOrder, searchTitle)} (${viaAirOrder.id})`,
|
||||
);
|
||||
return toResolution(viaAirOrder, searchTitle, season, 'air-order', true);
|
||||
}
|
||||
|
||||
// The chain failed for transport reasons rather than because the season is absent;
|
||||
// surface that so callers retry instead of reporting an unresolvable season.
|
||||
if (chainError) {
|
||||
throw chainError;
|
||||
}
|
||||
|
||||
deps.logInfo?.(
|
||||
`[anilist] could not resolve season ${season} of "${searchTitle}"; falling back to ${displayTitle(anchor, searchTitle)} (${anchor.id})`,
|
||||
);
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', false);
|
||||
}
|
||||
@@ -85,7 +85,6 @@ const HOT_RELOAD_EXACT_OR_PREFIX_PATHS = [
|
||||
'ankiConnect.fields.miscInfo',
|
||||
'ankiConnect.isLapis.sentenceCardModel',
|
||||
'ankiConnect.isKiku.fieldGrouping',
|
||||
'ankiConnect.lapisKiku.wordCardKind',
|
||||
] as const;
|
||||
|
||||
function hotReloadFieldForChangedPath(path: string): string | null {
|
||||
|
||||
@@ -1466,190 +1466,6 @@ test('deleteVideo ignores the currently active video and keeps new writes flusha
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Attach the derived rows a real session would produce to every recorded line
|
||||
* of `animeId`: word/kanji entries and their occurrences, monthly rollups, and
|
||||
* cover art backed by a shared blob.
|
||||
*/
|
||||
function seedDerivedAnimeData(db: DatabaseSync, animeId: number): void {
|
||||
const lines = db
|
||||
.prepare(
|
||||
'SELECT line_id AS lineId, CREATED_DATE AS seenMs FROM imm_subtitle_lines WHERE anime_id = ?',
|
||||
)
|
||||
.all(animeId) as Array<{ lineId: number; seenMs: number }>;
|
||||
assert.ok(lines.length > 0, 'expected recorded subtitle lines to decorate');
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (9001, '天気', '天気', 'てんき', 'noun', '名詞', 0, 0, 0)`,
|
||||
).run();
|
||||
db.prepare(
|
||||
`INSERT INTO imm_kanji(id, kanji, first_seen, last_seen, frequency) VALUES (9101, '気', 0, 0, 0)`,
|
||||
).run();
|
||||
|
||||
const insertWordOccurrence = db.prepare(
|
||||
'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, 9001, 1, ?)',
|
||||
);
|
||||
const insertKanjiOccurrence = db.prepare(
|
||||
'INSERT INTO imm_kanji_line_occurrences(line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, 9101, 1, ?)',
|
||||
);
|
||||
for (const line of lines) {
|
||||
insertWordOccurrence.run(line.lineId, line.seenMs);
|
||||
insertKanjiOccurrence.run(line.lineId, line.seenMs);
|
||||
}
|
||||
db.prepare(
|
||||
`UPDATE imm_words SET frequency = ?, first_seen = ?, last_seen = ? WHERE id = 9001`,
|
||||
).run(lines.length, 0, 0);
|
||||
|
||||
const videoIds = (
|
||||
db
|
||||
.prepare('SELECT video_id AS videoId FROM imm_videos WHERE anime_id = ?')
|
||||
.all(animeId) as Array<{ videoId: number }>
|
||||
).map((row) => row.videoId);
|
||||
const insertMonthlyRollup = db.prepare(
|
||||
`INSERT INTO imm_monthly_rollups(rollup_month, video_id, total_sessions, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (202401, ?, 1, '0', '0')`,
|
||||
);
|
||||
const insertArt = db.prepare(
|
||||
`INSERT INTO imm_media_art(video_id, anilist_id, cover_url, cover_blob, cover_blob_hash, fetched_at_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, 4242, 'https://example.test/cover.jpg', NULL, 'deadbeef', '0', '0', '0')`,
|
||||
);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_cover_art_blobs(blob_hash, cover_blob, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES ('deadbeef', X'FFD8FFD9', '0', '0')`,
|
||||
).run();
|
||||
for (const videoId of videoIds) {
|
||||
insertMonthlyRollup.run(videoId);
|
||||
insertArt.run(videoId);
|
||||
}
|
||||
}
|
||||
|
||||
test('deleteAnime removes every episode, session and library row for the title', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
|
||||
for (const episode of ['S02E05', 'S02E06']) {
|
||||
tracker = new Ctor({ dbPath });
|
||||
tracker.handleMediaChange(`/tmp/Little Witch Academia ${episode}.mkv`, `Episode ${episode}`);
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
tracker.recordSubtitleLine('今日は晴れです', 0, 1.2);
|
||||
tracker.recordCardsMined(1);
|
||||
tracker.destroy();
|
||||
tracker = null;
|
||||
}
|
||||
|
||||
tracker = new Ctor({ dbPath });
|
||||
const privateApi = tracker as unknown as { db: DatabaseSync };
|
||||
const animeId = (
|
||||
privateApi.db.prepare('SELECT anime_id FROM imm_anime LIMIT 1').get() as {
|
||||
anime_id: number;
|
||||
} | null
|
||||
)?.anime_id;
|
||||
assert.ok(animeId);
|
||||
|
||||
// The tokenizer does not run in this harness, so attach vocabulary, kanji,
|
||||
// rollups and cover art to the recorded lines by hand. Without them the
|
||||
// "everything is gone" assertions below would pass against empty tables.
|
||||
seedDerivedAnimeData(privateApi.db, animeId);
|
||||
|
||||
const countOf = (sql: string): number =>
|
||||
(privateApi.db.prepare(sql).get() as { total: number }).total;
|
||||
for (const table of [
|
||||
'imm_words',
|
||||
'imm_kanji',
|
||||
'imm_word_line_occurrences',
|
||||
'imm_kanji_line_occurrences',
|
||||
'imm_daily_rollups',
|
||||
'imm_monthly_rollups',
|
||||
'imm_media_art',
|
||||
'imm_cover_art_blobs',
|
||||
]) {
|
||||
assert.ok(
|
||||
countOf(`SELECT COUNT(*) AS total FROM ${table}`) > 0,
|
||||
`precondition: ${table} should hold rows before the delete`,
|
||||
);
|
||||
}
|
||||
|
||||
const libraryBefore = await tracker.getAnimeLibrary();
|
||||
assert.equal(libraryBefore.length, 1);
|
||||
assert.equal(libraryBefore[0]?.episodeCount, 2);
|
||||
|
||||
await tracker.deleteAnime(animeId);
|
||||
|
||||
const libraryAfter = await tracker.getAnimeLibrary();
|
||||
assert.equal(libraryAfter.length, 0);
|
||||
|
||||
for (const table of [
|
||||
'imm_anime',
|
||||
'imm_lifetime_anime',
|
||||
'imm_videos',
|
||||
'imm_sessions',
|
||||
'imm_subtitle_lines',
|
||||
'imm_daily_rollups',
|
||||
'imm_monthly_rollups',
|
||||
'imm_lifetime_media',
|
||||
'imm_words',
|
||||
'imm_kanji',
|
||||
'imm_word_line_occurrences',
|
||||
'imm_kanji_line_occurrences',
|
||||
'imm_media_art',
|
||||
'imm_cover_art_blobs',
|
||||
]) {
|
||||
assert.equal(
|
||||
countOf(`SELECT COUNT(*) AS total FROM ${table}`),
|
||||
0,
|
||||
`${table} should be empty after deleting the only title`,
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleteAnime ignores the title of the currently active session', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath });
|
||||
tracker.handleMediaChange('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
|
||||
const privateApi = tracker as unknown as {
|
||||
db: DatabaseSync;
|
||||
sessionState: { sessionId: number; videoId: number } | null;
|
||||
};
|
||||
const videoId = privateApi.sessionState?.videoId;
|
||||
assert.ok(videoId);
|
||||
const animeId = (
|
||||
privateApi.db.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?').get(videoId) as {
|
||||
anime_id: number | null;
|
||||
} | null
|
||||
)?.anime_id;
|
||||
assert.ok(animeId);
|
||||
|
||||
await tracker.deleteAnime(animeId);
|
||||
|
||||
const animeCountRow = privateApi.db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_anime WHERE anime_id = ?')
|
||||
.get(animeId) as { total: number };
|
||||
const videoCountRow = privateApi.db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_videos WHERE video_id = ?')
|
||||
.get(videoId) as { total: number };
|
||||
|
||||
assert.equal(animeCountRow.total, 1);
|
||||
assert.equal(videoCountRow.total, 1);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('handleMediaChange links parsed anime metadata on the active video row', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
@@ -83,7 +83,6 @@ import {
|
||||
} from './immersion-tracker/query-library';
|
||||
import {
|
||||
cleanupVocabularyStats,
|
||||
deleteAnime as deleteAnimeQuery,
|
||||
deleteSession as deleteSessionQuery,
|
||||
deleteSessions as deleteSessionsQuery,
|
||||
deleteVideo as deleteVideoQuery,
|
||||
@@ -734,28 +733,6 @@ export class ImmersionTrackerService {
|
||||
deleteVideoQuery(this.db, videoId);
|
||||
}
|
||||
|
||||
async deleteAnime(animeId: number): Promise<void> {
|
||||
// The active video's anime link is assigned asynchronously after the title
|
||||
// is parsed, so a guard reading imm_videos too early sees a null and lets
|
||||
// the delete through — then the late update recreates the anime row.
|
||||
const pendingVideoId = this.sessionState?.videoId;
|
||||
if (pendingVideoId !== undefined) {
|
||||
await this.pendingAnimeMetadataUpdates.get(pendingVideoId);
|
||||
}
|
||||
|
||||
const activeVideoId = this.sessionState?.videoId;
|
||||
if (activeVideoId !== undefined) {
|
||||
const activeAnime = this.db
|
||||
.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?')
|
||||
.get(activeVideoId) as { anime_id: number | null } | null;
|
||||
if (activeAnime?.anime_id === animeId) {
|
||||
this.logger.warn(`Ignoring delete request for active immersion anime ${animeId}`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
deleteAnimeQuery(this.db, animeId);
|
||||
}
|
||||
|
||||
async reassignAnimeAnilist(
|
||||
animeId: number,
|
||||
info: {
|
||||
|
||||
@@ -1,331 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database } from '../sqlite.js';
|
||||
import type { DatabaseSync } from '../sqlite.js';
|
||||
import { ensureSchema } from '../storage.js';
|
||||
import { deleteSession, deleteSessions, deleteVideo } from '../query-maintenance.js';
|
||||
|
||||
const DAY_MS = 86_400_000;
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
|
||||
function makeDbPath(): string {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-lexical-removal-test-'));
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) return;
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* Seed two episodes of one anime, each with one ended session.
|
||||
*
|
||||
* `lines` places a word occurrence on a specific day so tests can control which
|
||||
* session holds a word's first/last occurrence.
|
||||
*/
|
||||
function seed(
|
||||
db: DatabaseSync,
|
||||
lines: Array<{ session: 1 | 2; wordId: number; dayOffset: number; count?: number }>,
|
||||
options: { legacyRows?: boolean } = {},
|
||||
): void {
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'show', 'Show', ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'v1', 1, 'Ep 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS}),
|
||||
(2, 'v2', 1, 'Ep 2', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 's1', 1, '${BASE_MS}', '${BASE_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS}),
|
||||
(2, 's2', 2, '${BASE_MS + DAY_MS}', '${BASE_MS + DAY_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS});
|
||||
`);
|
||||
|
||||
const insertLine = db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, 1, ?, ?, ?, ?)`,
|
||||
);
|
||||
const insertWord = db.prepare(
|
||||
`INSERT OR IGNORE INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (?, ?, ?, '', 'noun', '名詞', 0, 0, 0)`,
|
||||
);
|
||||
// `legacyRows` reproduces databases written before the seen_ms column existed,
|
||||
// where the timestamp has to be read back off the subtitle line.
|
||||
const insertOccurrence = options.legacyRows
|
||||
? db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (?, ?, ?)`,
|
||||
)
|
||||
: db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)`,
|
||||
);
|
||||
|
||||
let lineId = 0;
|
||||
for (const line of lines) {
|
||||
lineId += 1;
|
||||
const seenMs = BASE_MS + line.dayOffset * DAY_MS;
|
||||
insertLine.run(lineId, line.session, line.session, lineId, `line ${lineId}`, seenMs, seenMs);
|
||||
insertWord.run(line.wordId, `語${line.wordId}`, `語${line.wordId}`);
|
||||
if (options.legacyRows) {
|
||||
insertOccurrence.run(lineId, line.wordId, line.count ?? 1);
|
||||
} else {
|
||||
insertOccurrence.run(lineId, line.wordId, line.count ?? 1, seenMs);
|
||||
}
|
||||
}
|
||||
|
||||
// Match what the tracker maintains: aggregates derived from the occurrences.
|
||||
db.exec(`
|
||||
UPDATE imm_words SET
|
||||
frequency = (
|
||||
SELECT COALESCE(SUM(o.occurrence_count), 0)
|
||||
FROM imm_word_line_occurrences o WHERE o.word_id = imm_words.id
|
||||
),
|
||||
first_seen = (
|
||||
SELECT MIN(sl.CREATED_DATE) / 1000
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE o.word_id = imm_words.id
|
||||
),
|
||||
last_seen = (
|
||||
SELECT MAX(sl.LAST_UPDATE_DATE) / 1000
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE o.word_id = imm_words.id
|
||||
)
|
||||
`);
|
||||
}
|
||||
|
||||
function createDb(
|
||||
lines: Parameters<typeof seed>[1],
|
||||
options: Parameters<typeof seed>[2] = {},
|
||||
): { db: DatabaseSync; dbPath: string } {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
seed(db, lines, options);
|
||||
return { db, dbPath };
|
||||
}
|
||||
|
||||
function readWord(
|
||||
db: DatabaseSync,
|
||||
wordId: number,
|
||||
): { frequency: number; firstSeen: number; lastSeen: number } | null {
|
||||
return (
|
||||
(db
|
||||
.prepare(
|
||||
'SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen FROM imm_words WHERE id = ?',
|
||||
)
|
||||
.get(wordId) as { frequency: number; firstSeen: number; lastSeen: number } | null) ?? null
|
||||
);
|
||||
}
|
||||
|
||||
test('deleting a session subtracts only the occurrences it removed', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 10, dayOffset: 0, count: 3 },
|
||||
{ session: 2, wordId: 10, dayOffset: 1, count: 4 },
|
||||
]);
|
||||
|
||||
try {
|
||||
assert.equal(readWord(db, 10)?.frequency, 7);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 10)?.frequency, 4, 'only session 1 occurrences are subtracted');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the session that held a word first drops the word entirely', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 11, dayOffset: 0 },
|
||||
{ session: 2, wordId: 12, dayOffset: 1 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSession(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 11), null, 'word seen only in the deleted session is removed');
|
||||
assert.ok(readWord(db, 12), 'word seen elsewhere survives');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the earliest session moves first_seen forward to the surviving line', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 13, dayOffset: 0 },
|
||||
{ session: 2, wordId: 13, dayOffset: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
assert.equal(readWord(db, 13)?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 13);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(
|
||||
word?.firstSeen,
|
||||
Math.floor((BASE_MS + 5 * DAY_MS) / 1000),
|
||||
'first_seen advances to the remaining occurrence',
|
||||
);
|
||||
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 5 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the latest session moves last_seen back to the surviving line', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 14, dayOffset: 0 },
|
||||
{ session: 2, wordId: 14, dayOffset: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSession(db, 2);
|
||||
|
||||
const word = readWord(db, 14);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(word?.lastSeen, Math.floor(BASE_MS / 1000), 'last_seen falls back to session 1');
|
||||
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting an interior occurrence leaves the surrounding extremes untouched', () => {
|
||||
// Session 2 carries the middle occurrence; sessions bracket it in time.
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 15, dayOffset: 0 },
|
||||
{ session: 2, wordId: 15, dayOffset: 3 },
|
||||
{ session: 1, wordId: 15, dayOffset: 9 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSessions(db, [2]);
|
||||
|
||||
const word = readWord(db, 15);
|
||||
assert.equal(word?.frequency, 2);
|
||||
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 9 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a stored frequency that has drifted low is repaired instead of dropping a live word', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 16, dayOffset: 0, count: 5 },
|
||||
{ session: 2, wordId: 16, dayOffset: 4, count: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
// Simulate drift: the stored total is lower than the occurrences justify, so
|
||||
// naive subtraction would take the word to zero while rows still reference it.
|
||||
db.prepare('UPDATE imm_words SET frequency = 5 WHERE id = ?').run(16);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 16);
|
||||
assert.ok(word, 'word with surviving occurrences is not deleted');
|
||||
assert.equal(word?.frequency, 5, 'frequency is recomputed from the surviving occurrences');
|
||||
assert.equal(word?.firstSeen, Math.floor((BASE_MS + 4 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting a video subtracts every occurrence carried by its lines', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 17, dayOffset: 0, count: 2 },
|
||||
{ session: 1, wordId: 17, dayOffset: 1, count: 3 },
|
||||
{ session: 2, wordId: 17, dayOffset: 2, count: 4 },
|
||||
{ session: 2, wordId: 18, dayOffset: 2, count: 1 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteVideo(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 17)?.frequency, 4, 'both lines from video 1 are subtracted');
|
||||
assert.equal(readWord(db, 18)?.frequency, 1, 'untouched video keeps its counts');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('occurrence rows written before the seen_ms column still resolve their dates', () => {
|
||||
const { db, dbPath } = createDb(
|
||||
[
|
||||
{ session: 1, wordId: 20, dayOffset: 0 },
|
||||
{ session: 2, wordId: 20, dayOffset: 6 },
|
||||
],
|
||||
{ legacyRows: true },
|
||||
);
|
||||
|
||||
try {
|
||||
assert.equal(
|
||||
(
|
||||
db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_word_line_occurrences WHERE seen_ms IS NULL')
|
||||
.get() as { total: number }
|
||||
).total,
|
||||
2,
|
||||
'precondition: rows carry no denormalised timestamp',
|
||||
);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 20);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(
|
||||
word?.firstSeen,
|
||||
Math.floor((BASE_MS + 6 * DAY_MS) / 1000),
|
||||
'falls back to the subtitle line timestamp',
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('upgrading an older database backfills seen_ms from the subtitle lines', () => {
|
||||
const { db, dbPath } = createDb(
|
||||
[
|
||||
{ session: 1, wordId: 21, dayOffset: 0 },
|
||||
{ session: 2, wordId: 21, dayOffset: 2 },
|
||||
],
|
||||
{ legacyRows: true },
|
||||
);
|
||||
|
||||
try {
|
||||
// Re-run ensureSchema the way a pre-19 database would be opened.
|
||||
db.exec('DELETE FROM imm_schema_version');
|
||||
db.exec(`INSERT INTO imm_schema_version(schema_version, applied_at_ms) VALUES (18, '0')`);
|
||||
ensureSchema(db);
|
||||
|
||||
const rows = db
|
||||
.prepare('SELECT line_id AS lineId, seen_ms AS seenMs FROM imm_word_line_occurrences')
|
||||
.all() as Array<{ lineId: number; seenMs: number | null }>;
|
||||
assert.equal(rows.length, 2);
|
||||
for (const row of rows) {
|
||||
assert.ok(row.seenMs, `line ${row.lineId} should have a backfilled timestamp`);
|
||||
}
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.seenMs).sort((a, b) => Number(a) - Number(b)),
|
||||
[BASE_MS, BASE_MS + 2 * DAY_MS],
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
@@ -5,7 +5,6 @@ import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database } from '../sqlite.js';
|
||||
import {
|
||||
applyPragmas,
|
||||
createTrackerPreparedStatements,
|
||||
ensureSchema,
|
||||
getOrCreateAnimeRecord,
|
||||
@@ -58,16 +57,6 @@ function makeDbPath(): string {
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
// Match the runtime's SQLite tuning (WAL + synchronous=NORMAL). Without it these
|
||||
// tests run at SQLite's defaults, where every statement is its own fsync-ing
|
||||
// transaction; on slow CI disks that is enough to push insert-heavy cases past
|
||||
// the 5s per-test timeout.
|
||||
function openTestDb(dbPath: string) {
|
||||
const db = new Database(dbPath);
|
||||
applyPragmas(db);
|
||||
return db;
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) {
|
||||
@@ -183,7 +172,7 @@ function withMockNowMs<T>(fixedDateMs: string | number, run: () => T): T {
|
||||
|
||||
test('getSessionSummaries returns sessionId and canonicalTitle', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -242,7 +231,7 @@ test('getSessionSummaries returns sessionId and canonicalTitle', () => {
|
||||
|
||||
test('getAnimeEpisodes prefers the latest session media position when the latest session is still active', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -310,7 +299,7 @@ test('getAnimeEpisodes prefers the latest session media position when the latest
|
||||
|
||||
test('getAnimeEpisodes includes unwatched episodes for the anime', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -408,7 +397,7 @@ test('getAnimeEpisodes includes unwatched episodes for the anime', () => {
|
||||
|
||||
test('getAnimeEpisodes falls back to the latest subtitle segment end when session progress checkpoints are missing', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -480,7 +469,7 @@ test('getAnimeEpisodes falls back to the latest subtitle segment end when sessio
|
||||
|
||||
test('getAnimeEpisodes ignores zero-valued session checkpoints and falls back to subtitle progress', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -552,7 +541,7 @@ test('getAnimeEpisodes ignores zero-valued session checkpoints and falls back to
|
||||
|
||||
test('getSessionTimeline returns the full session when no limit is provided', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -603,7 +592,7 @@ test('getSessionTimeline returns the full session when no limit is provided', ()
|
||||
|
||||
test('getDailyRollups limits by distinct days (not rows)', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -636,7 +625,7 @@ test('getDailyRollups limits by distinct days (not rows)', () => {
|
||||
|
||||
test('getTrendsDashboard returns chart-ready aggregated series', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -790,7 +779,7 @@ test('getTrendsDashboard returns chart-ready aggregated series', () => {
|
||||
|
||||
test('getTrendsDashboard redacts legacy Jellyfin stream titles', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -880,7 +869,7 @@ test('getTrendsDashboard redacts legacy Jellyfin stream titles', () => {
|
||||
|
||||
test('getTrendsDashboard keeps local-midnight session buckets separate', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1016,7 +1005,7 @@ test('getTrendsDashboard keeps local-midnight session buckets separate', () => {
|
||||
|
||||
test('getTrendsDashboard 30d day range zero-fills empty calendar days', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
withMockNowMs('1772395200000', () => {
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1080,7 +1069,7 @@ test('getTrendsDashboard 30d day range zero-fills empty calendar days', () => {
|
||||
|
||||
test('getTrendsDashboard skips empty calendar days when zero-fill is disabled', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
withMockNowMs('1772395200000', () => {
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1134,7 +1123,7 @@ test(
|
||||
{ timeout: 20_000 },
|
||||
() => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
withMockNowMs('1772395200000', () => {
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1194,7 +1183,7 @@ test(
|
||||
|
||||
test('getTrendsDashboard month grouping spans every touched calendar month and keeps progress monthly', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
withMockNowMs('1772395200000', () => {
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1417,7 +1406,7 @@ test('getTrendsDashboard month grouping spans every touched calendar month and k
|
||||
|
||||
test('getQueryHints reads all-time totals from lifetime summary', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1487,7 +1476,7 @@ test('getQueryHints reads all-time totals from lifetime summary', () => {
|
||||
|
||||
test('getQueryHints computes weekly new-word cutoff from calendar midnights', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
withMockNowMs('1773601200000', () => {
|
||||
try {
|
||||
@@ -1539,7 +1528,7 @@ test('getQueryHints computes weekly new-word cutoff from calendar midnights', ()
|
||||
|
||||
test('word-count read models use filtered persisted occurrences with raw fallback', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1622,7 +1611,7 @@ test('word-count read models use filtered persisted occurrences with raw fallbac
|
||||
|
||||
test('rollups keep persisted totals when retained-session word counts are partial', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1683,7 +1672,7 @@ test('rollups keep persisted totals when retained-session word counts are partia
|
||||
|
||||
test('getQueryHints counts new words by distinct headword first-seen time', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1748,7 +1737,7 @@ test('getQueryHints counts new words by distinct headword first-seen time', () =
|
||||
|
||||
test('getSessionSummaries with no telemetry returns zero aggregates', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1782,7 +1771,7 @@ test('getSessionSummaries with no telemetry returns zero aggregates', () => {
|
||||
|
||||
test('getSessionSummaries uses denormalized session metrics for ended sessions without telemetry', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1834,7 +1823,7 @@ test('getSessionSummaries uses denormalized session metrics for ended sessions w
|
||||
|
||||
test('getVocabularyStats returns rows ordered by frequency descending', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1877,7 +1866,7 @@ test('getVocabularyStats returns rows ordered by frequency descending', () => {
|
||||
|
||||
test('getVocabularyStats filters rows that fail tokenizer vocabulary rules', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1932,7 +1921,7 @@ test('getVocabularyStats filters rows that fail tokenizer vocabulary rules', ()
|
||||
|
||||
test('getVocabularyStats pages past hidden rows until enough visible rows are collected', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1967,7 +1956,7 @@ test('getVocabularyStats pages past hidden rows until enough visible rows are co
|
||||
|
||||
test('getVocabularyStats returns empty array when no words exist', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -1981,7 +1970,7 @@ test('getVocabularyStats returns empty array when no words exist', () => {
|
||||
|
||||
test('cleanupVocabularyStats repairs stored POS metadata and removes excluded imm_words rows', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2111,7 +2100,7 @@ test('cleanupVocabularyStats repairs stored POS metadata and removes excluded im
|
||||
|
||||
test('getDailyRollups returns all rows for the most recent rollup days', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const insertRollup = db.prepare(
|
||||
@@ -2147,7 +2136,7 @@ test('getDailyRollups returns all rows for the most recent rollup days', () => {
|
||||
|
||||
test('getMonthlyRollups returns all rows for the most recent rollup months', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const insertRollup = db.prepare(
|
||||
@@ -2178,7 +2167,7 @@ test('getMonthlyRollups returns all rows for the most recent rollup months', ()
|
||||
|
||||
test('getMonthlyRollups derives rate metrics from stored monthly totals', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const insertRollup = db.prepare(
|
||||
@@ -2209,7 +2198,7 @@ test('getMonthlyRollups derives rate metrics from stored monthly totals', () =>
|
||||
|
||||
test('getAnimeDailyRollups returns all rows for the most recent rollup days', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const insertRollup = db.prepare(
|
||||
@@ -2265,7 +2254,7 @@ test('getAnimeDailyRollups returns all rows for the most recent rollup days', ()
|
||||
|
||||
test('cleanupVocabularyStats merges repaired duplicates instead of violating the imm_words unique key', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2378,7 +2367,7 @@ test('cleanupVocabularyStats merges repaired duplicates instead of violating the
|
||||
|
||||
test('getKanjiStats returns rows ordered by frequency descending', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2416,7 +2405,7 @@ test('getKanjiStats returns rows ordered by frequency descending', () => {
|
||||
|
||||
test('getKanjiStats returns empty array when no kanji exist', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2430,7 +2419,7 @@ test('getKanjiStats returns empty array when no kanji exist', () => {
|
||||
|
||||
test('getSessionEvents returns events ordered by ts_ms ascending', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2491,7 +2480,7 @@ test('getSessionEvents returns events ordered by ts_ms ascending', () => {
|
||||
|
||||
test('getSessionEvents round-trips wall-clock timestamps written through event inserts', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2535,7 +2524,7 @@ test('getSessionEvents round-trips wall-clock timestamps written through event i
|
||||
|
||||
test('getSessionEvents returns empty array for session with no events', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2558,7 +2547,7 @@ test('getSessionEvents returns empty array for session with no events', () => {
|
||||
|
||||
test('getSessionEvents filters events to the requested session id', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2624,7 +2613,7 @@ test('getSessionEvents filters events to the requested session id', () => {
|
||||
|
||||
test('getSessionEvents respects limit parameter', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2667,7 +2656,7 @@ test('getSessionEvents respects limit parameter', () => {
|
||||
|
||||
test('getSessionEvents filters by event type before applying limit', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2739,7 +2728,7 @@ test('getSessionEvents filters by event type before applying limit', () => {
|
||||
|
||||
test('getSessionWordsByLine joins word occurrences through imm_words.id', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -2814,7 +2803,7 @@ test('getSessionWordsByLine joins word occurrences through imm_words.id', () =>
|
||||
|
||||
test('anime-level queries group by anime_id and preserve episode-level rows', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3094,7 +3083,7 @@ test('anime-level queries group by anime_id and preserve episode-level rows', ()
|
||||
|
||||
test('anime library and detail still return lifetime rows without retained sessions', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3191,7 +3180,7 @@ test('anime library and detail still return lifetime rows without retained sessi
|
||||
|
||||
test('anime and media detail prefer lifetime totals over partial retained sessions', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3281,7 +3270,7 @@ test('anime and media detail prefer lifetime totals over partial retained sessio
|
||||
|
||||
test('media detail resolves retained sessions before lifetime summary exists', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3334,7 +3323,7 @@ test('media detail resolves retained sessions before lifetime summary exists', (
|
||||
|
||||
test('media library and detail queries read lifetime totals', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3426,7 +3415,7 @@ test('media library and detail queries read lifetime totals', () => {
|
||||
|
||||
test('media library and detail queries include joined youtube metadata when present', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3523,7 +3512,7 @@ test('media library and detail queries include joined youtube metadata when pres
|
||||
|
||||
test('cover art queries reuse a shared blob across duplicate anime art rows', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3650,7 +3639,7 @@ test('cover art queries reuse a shared blob across duplicate anime art rows', ()
|
||||
|
||||
test('upsertCoverArt prefers freshly fetched bytes over a reused shared hash', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3696,7 +3685,7 @@ test('upsertCoverArt prefers freshly fetched bytes over a reused shared hash', (
|
||||
|
||||
test('anime/media detail and episode queries use ended-session metrics when telemetry rows are absent', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3838,7 +3827,7 @@ test('anime/media detail and episode queries use ended-session metrics when tele
|
||||
|
||||
test('getWordOccurrences maps a normalized word back to anime, video, and subtitle line context', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -3913,7 +3902,7 @@ test('getWordOccurrences maps a normalized word back to anime, video, and subtit
|
||||
|
||||
test('searchSubtitleSentences searches known subtitle lines and returns media context', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4010,7 +3999,7 @@ test('searchSubtitleSentences searches known subtitle lines and returns media co
|
||||
|
||||
test('searchSubtitleSentences searches subtitle lines by resolved headword candidates', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4094,7 +4083,7 @@ test('searchSubtitleSentences searches subtitle lines by resolved headword candi
|
||||
|
||||
test('getKanjiOccurrences maps a kanji back to anime, video, and subtitle line context', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4169,7 +4158,7 @@ test('getKanjiOccurrences maps a kanji back to anime, video, and subtitle line c
|
||||
|
||||
test('deleteSession removes the session and all associated session-scoped rows', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4317,7 +4306,7 @@ test('deleteSession removes the session and all associated session-scoped rows',
|
||||
|
||||
test('deleteSession rebuilds word and kanji aggregates from retained subtitle lines', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4473,7 +4462,7 @@ test('deleteSession rebuilds word and kanji aggregates from retained subtitle li
|
||||
|
||||
test('deleteSession removes zero-session media from library and trends', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4678,7 +4667,7 @@ test('deleteSession removes zero-session media from library and trends', () => {
|
||||
|
||||
test('getTrendsDashboard builds librarySummary with per-title aggregates', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4785,7 +4774,7 @@ test('getTrendsDashboard builds librarySummary with per-title aggregates', () =>
|
||||
|
||||
test('getTrendsDashboard librarySummary returns null lookupsPerHundred when words is zero', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4867,7 +4856,7 @@ test('getTrendsDashboard librarySummary returns null lookupsPerHundred when word
|
||||
|
||||
test('getTrendsDashboard rollup word metrics keep persisted totals over partial session counts', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4982,7 +4971,7 @@ test('getTrendsDashboard rollup word metrics keep persisted totals over partial
|
||||
|
||||
test('getTrendsDashboard librarySummary is empty when no rollups exist', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
@@ -4993,145 +4982,3 @@ test('getTrendsDashboard librarySummary is empty when no rollups exist', () => {
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('getVocabularyStats counts the distinct anime each word appeared in', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const startedAtMs = 1_700_000_000_000;
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs}),
|
||||
(22, 'vocab-b', 'Vocab B', ${startedAtMs}, ${startedAtMs});
|
||||
`);
|
||||
const videoIds = [1, 2].map((n) =>
|
||||
getOrCreateVideoRecord(db, `local:/tmp/vocab-anime-${n}.mkv`, {
|
||||
canonicalTitle: `Vocab Anime ${n}`,
|
||||
sourcePath: `/tmp/vocab-anime-${n}.mkv`,
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
}),
|
||||
);
|
||||
const sessionIds = videoIds.map(
|
||||
(videoId) => startSessionRecord(db, videoId, startedAtMs).sessionId,
|
||||
);
|
||||
|
||||
// 猫 spans both titles, twice in the first, so the count has to deduplicate
|
||||
// by anime rather than just tallying occurrence rows. 犬 stays in one title.
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 1,
|
||||
occurrenceCount: 5,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 2,
|
||||
occurrenceCount: 1,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[1]!,
|
||||
videoId: videoIds[1]!,
|
||||
animeId: 22,
|
||||
lineIndex: 3,
|
||||
occurrenceCount: 4,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 4,
|
||||
occurrenceCount: 2,
|
||||
startedAtMs,
|
||||
word: '犬',
|
||||
reading: 'いぬ',
|
||||
});
|
||||
|
||||
const rows = getVocabularyStats(db, 10);
|
||||
const cat = rows.find((row) => row.headword === '猫');
|
||||
const dog = rows.find((row) => row.headword === '犬');
|
||||
|
||||
assert.equal(cat?.animeCount, 2, '猫 was seen in two anime across three lines');
|
||||
assert.equal(dog?.animeCount, 1, '犬 was seen in one');
|
||||
assert.equal(cat?.frequency, 10, 'frequency still aggregates across both titles');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('getVocabularyStats still applies part-of-speech exclusions', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const startedAtMs = 1_700_000_000_000;
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs});
|
||||
`);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/vocab-exclude.mkv', {
|
||||
canonicalTitle: 'Vocab Exclude',
|
||||
sourcePath: '/tmp/vocab-exclude.mkv',
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
|
||||
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId,
|
||||
videoId,
|
||||
animeId: 11,
|
||||
lineIndex: 1,
|
||||
occurrenceCount: 9,
|
||||
startedAtMs,
|
||||
word: '走る',
|
||||
reading: 'はしる',
|
||||
partOfSpeech: 'verb',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId,
|
||||
videoId,
|
||||
animeId: 11,
|
||||
lineIndex: 2,
|
||||
occurrenceCount: 3,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
partOfSpeech: 'noun',
|
||||
});
|
||||
|
||||
const all = getVocabularyStats(db, 10);
|
||||
assert.deepEqual(
|
||||
all.map((row) => row.headword).sort(),
|
||||
['猫', '走る'],
|
||||
'precondition: both parts of speech are present',
|
||||
);
|
||||
|
||||
const nounsOnly = getVocabularyStats(db, 10, ['verb']);
|
||||
assert.deepEqual(
|
||||
nounsOnly.map((row) => row.headword),
|
||||
['猫'],
|
||||
'excluded part of speech is filtered out even though it ranks higher',
|
||||
);
|
||||
assert.equal(nounsOnly[0]?.animeCount, 1, 'surviving rows keep their anime count');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -115,29 +115,18 @@ export function getVocabularyStats(
|
||||
const whereClause = hasExclude
|
||||
? `WHERE (part_of_speech IS NULL OR part_of_speech NOT IN (${placeholders}))`
|
||||
: '';
|
||||
// The page is selected before the join so `animeCount` is only computed for the
|
||||
// rows being returned. Aggregating first made every request walk each word's
|
||||
// entire occurrence history — seconds of blocked event loop on a large library,
|
||||
// because only the ordering, not the aggregate, decides which rows survive.
|
||||
const stmt = db.prepare(`
|
||||
WITH page AS (
|
||||
SELECT id, headword, word, reading, part_of_speech, pos1, pos2, pos3,
|
||||
frequency, frequency_rank, first_seen, last_seen
|
||||
FROM imm_words
|
||||
${whereClause}
|
||||
ORDER BY frequency DESC, id
|
||||
LIMIT ? OFFSET ?
|
||||
)
|
||||
SELECT p.id AS wordId, p.headword, p.word, p.reading,
|
||||
p.part_of_speech AS partOfSpeech, p.pos1, p.pos2, p.pos3,
|
||||
p.frequency, p.frequency_rank AS frequencyRank,
|
||||
p.first_seen AS firstSeen, p.last_seen AS lastSeen,
|
||||
SELECT w.id AS wordId, w.headword, w.word, w.reading,
|
||||
w.part_of_speech AS partOfSpeech, w.pos1, w.pos2, w.pos3,
|
||||
w.frequency, w.frequency_rank AS frequencyRank,
|
||||
w.first_seen AS firstSeen, w.last_seen AS lastSeen,
|
||||
COUNT(DISTINCT sl.anime_id) AS animeCount
|
||||
FROM page p
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = p.id
|
||||
FROM imm_words w
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id AND sl.anime_id IS NOT NULL
|
||||
GROUP BY p.id
|
||||
ORDER BY p.frequency DESC, p.id
|
||||
${whereClause ? whereClause.replace('part_of_speech', 'w.part_of_speech') : ''}
|
||||
GROUP BY w.id
|
||||
ORDER BY w.frequency DESC LIMIT ? OFFSET ?
|
||||
`);
|
||||
const visibleRows: VocabularyStatsRow[] = [];
|
||||
let offset = 0;
|
||||
|
||||
@@ -9,12 +9,14 @@ import { PartOfSpeech, type MergedToken } from '../../../types';
|
||||
import { shouldExcludeTokenFromVocabularyPersistence } from '../tokenizer/annotation-stage';
|
||||
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
|
||||
import {
|
||||
applyLexicalRemovals,
|
||||
cleanupUnusedCoverArtBlobHash,
|
||||
deleteSessionsByIds,
|
||||
findSharedCoverBlobHash,
|
||||
planLexicalRemovalsForSessions,
|
||||
planLexicalRemovalsForVideos,
|
||||
getAffectedKanjiIdsForSessions,
|
||||
getAffectedKanjiIdsForVideo,
|
||||
getAffectedWordIdsForSessions,
|
||||
getAffectedWordIdsForVideo,
|
||||
refreshLexicalAggregates,
|
||||
toDbMs,
|
||||
toDbTimestamp,
|
||||
} from './query-shared';
|
||||
@@ -230,13 +232,12 @@ export async function cleanupVocabularyStats(
|
||||
WHERE id = ?`,
|
||||
);
|
||||
const moveOccurrencesStmt = db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms)
|
||||
SELECT line_id, ?, occurrence_count, seen_ms
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
|
||||
SELECT line_id, ?, occurrence_count
|
||||
FROM imm_word_line_occurrences
|
||||
WHERE word_id = ?
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)`,
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
const deleteOccurrencesStmt = db.prepare(
|
||||
'DELETE FROM imm_word_line_occurrences WHERE word_id = ?',
|
||||
@@ -483,16 +484,14 @@ export function isVideoWatched(db: DatabaseSync, videoId: number): boolean {
|
||||
|
||||
export function deleteSession(db: DatabaseSync, sessionId: number): void {
|
||||
const sessionIds = [sessionId];
|
||||
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
// Measured inside the write lock: the plan records what the delete removes,
|
||||
// and applying a plan taken against a different snapshot would subtract the
|
||||
// wrong totals from imm_words/imm_kanji.
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
@@ -504,13 +503,14 @@ export function deleteSession(db: DatabaseSync, sessionId: number): void {
|
||||
|
||||
export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
|
||||
if (sessionIds.length === 0) return;
|
||||
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
@@ -520,83 +520,24 @@ export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an entire library entry: every episode of the anime, all of their
|
||||
* sessions and derived stats, and the anime row itself.
|
||||
*
|
||||
* Mirrors {@link deleteVideo} per episode, but batches the lexical refresh and
|
||||
* lifetime rebuild into a single transaction so a multi-episode title doesn't
|
||||
* pay for one full rebuild per episode.
|
||||
*/
|
||||
export function deleteAnime(db: DatabaseSync, animeId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const videoIds = (
|
||||
db.prepare('SELECT video_id FROM imm_videos WHERE anime_id = ?').all(animeId) as Array<{
|
||||
video_id: number;
|
||||
}>
|
||||
).map((row) => row.video_id);
|
||||
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, videoIds);
|
||||
const coverBlobHashes: string[] = [];
|
||||
const sessionIds: number[] = [];
|
||||
for (const videoId of videoIds) {
|
||||
const artRow = db
|
||||
.prepare('SELECT cover_blob_hash AS coverBlobHash FROM imm_media_art WHERE video_id = ?')
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
if (artRow?.coverBlobHash) {
|
||||
coverBlobHashes.push(artRow.coverBlobHash);
|
||||
}
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
sessionIds.push(...sessions.map((session) => session.session_id));
|
||||
}
|
||||
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
const deleteLinesStmt = db.prepare('DELETE FROM imm_subtitle_lines WHERE video_id = ?');
|
||||
const deleteDailyStmt = db.prepare('DELETE FROM imm_daily_rollups WHERE video_id = ?');
|
||||
const deleteMonthlyStmt = db.prepare('DELETE FROM imm_monthly_rollups WHERE video_id = ?');
|
||||
const deleteArtStmt = db.prepare('DELETE FROM imm_media_art WHERE video_id = ?');
|
||||
const deleteVideoStmt = db.prepare('DELETE FROM imm_videos WHERE video_id = ?');
|
||||
for (const videoId of videoIds) {
|
||||
deleteLinesStmt.run(videoId);
|
||||
deleteDailyStmt.run(videoId);
|
||||
deleteMonthlyStmt.run(videoId);
|
||||
deleteArtStmt.run(videoId);
|
||||
deleteVideoStmt.run(videoId);
|
||||
}
|
||||
for (const coverBlobHash of new Set(coverBlobHashes)) {
|
||||
cleanupUnusedCoverArtBlobHash(db, coverBlobHash);
|
||||
}
|
||||
db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?').run(animeId);
|
||||
db.prepare('DELETE FROM imm_anime WHERE anime_id = ?').run(animeId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function deleteVideo(db: DatabaseSync, videoId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const artRow = db
|
||||
.prepare(
|
||||
`
|
||||
const artRow = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT cover_blob_hash AS coverBlobHash
|
||||
FROM imm_media_art
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
)
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, [videoId]);
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
)
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
const affectedWordIds = getAffectedWordIdsForVideo(db, videoId);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForVideo(db, videoId);
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
deleteSessionsByIds(
|
||||
db,
|
||||
sessions.map((session) => session.session_id),
|
||||
@@ -607,7 +548,7 @@ export function deleteVideo(db: DatabaseSync, videoId: number): void {
|
||||
db.prepare('DELETE FROM imm_media_art WHERE video_id = ?').run(videoId);
|
||||
cleanupUnusedCoverArtBlobHash(db, artRow?.coverBlobHash ?? null);
|
||||
db.prepare('DELETE FROM imm_videos WHERE video_id = ?').run(videoId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
|
||||
@@ -203,167 +203,6 @@ export function getAffectedKanjiIdsForVideo(db: DatabaseSync, videoId: number):
|
||||
return getAffectedIdsForVideo(db, 'kanji', videoId);
|
||||
}
|
||||
|
||||
/** Per-entity totals that a pending delete is about to remove. */
|
||||
interface LexicalRemoval {
|
||||
id: number;
|
||||
removedFrequency: number;
|
||||
removedFirstSeenMs: number | null;
|
||||
removedLastSeenMs: number | null;
|
||||
}
|
||||
|
||||
/** What a pending delete removes from `imm_words` and `imm_kanji`. */
|
||||
export interface LexicalRemovalPlan {
|
||||
words: LexicalRemoval[];
|
||||
kanji: LexicalRemoval[];
|
||||
}
|
||||
|
||||
export const EMPTY_LEXICAL_REMOVAL_PLAN: LexicalRemovalPlan = { words: [], kanji: [] };
|
||||
|
||||
function collectLexicalRemovals(
|
||||
db: DatabaseSync,
|
||||
entity: LexicalEntity,
|
||||
lineScopeSql: string,
|
||||
params: number[],
|
||||
): LexicalRemoval[] {
|
||||
const table = entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
|
||||
const col = `${entity}_id`;
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT
|
||||
o.${col} AS id,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS removedFrequency,
|
||||
MIN(COALESCE(o.seen_ms, sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS removedFirstSeenMs,
|
||||
MAX(COALESCE(o.seen_ms, sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS removedLastSeenMs
|
||||
FROM imm_subtitle_lines sl
|
||||
JOIN ${table} o ON o.line_id = sl.line_id
|
||||
WHERE ${lineScopeSql}
|
||||
GROUP BY o.${col}`,
|
||||
)
|
||||
.all(...params) as LexicalRemoval[];
|
||||
}
|
||||
|
||||
function planLexicalRemovals(
|
||||
db: DatabaseSync,
|
||||
lineScopeSql: string,
|
||||
params: number[],
|
||||
): LexicalRemovalPlan {
|
||||
return {
|
||||
words: collectLexicalRemovals(db, 'word', lineScopeSql, params),
|
||||
kanji: collectLexicalRemovals(db, 'kanji', lineScopeSql, params),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Measure what deleting these sessions removes from the vocabulary tables.
|
||||
*
|
||||
* Must run before the rows are deleted. Reads only the subtitle lines in scope,
|
||||
* unlike {@link refreshLexicalAggregates}, which re-reads every occurrence of
|
||||
* every affected word across the whole library.
|
||||
*/
|
||||
export function planLexicalRemovalsForSessions(
|
||||
db: DatabaseSync,
|
||||
sessionIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
if (sessionIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
|
||||
return planLexicalRemovals(db, `sl.session_id IN (${makePlaceholders(sessionIds)})`, sessionIds);
|
||||
}
|
||||
|
||||
/** Measure what deleting these videos removes from the vocabulary tables. */
|
||||
export function planLexicalRemovalsForVideos(
|
||||
db: DatabaseSync,
|
||||
videoIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
if (videoIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
|
||||
return planLexicalRemovals(db, `sl.video_id IN (${makePlaceholders(videoIds)})`, videoIds);
|
||||
}
|
||||
|
||||
function toStoredSeenSeconds(ms: number | null): number | null {
|
||||
if (ms === null || !Number.isFinite(ms)) return null;
|
||||
return Math.floor(ms / 1000);
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a removal plan to the vocabulary aggregates.
|
||||
*
|
||||
* Frequencies are adjusted by subtraction, which is exact and touches only the
|
||||
* affected rows. `first_seen`/`last_seen` only need a rescan when the removed
|
||||
* lines held the current extreme, and rows whose frequency reaches zero are
|
||||
* verified against the surviving occurrences before deletion — so stored counts
|
||||
* that have drifted still converge on the truth instead of dropping a live row.
|
||||
*/
|
||||
export function applyLexicalRemovals(db: DatabaseSync, plan: LexicalRemovalPlan): void {
|
||||
applyRemovalsForEntity(db, 'word', plan.words);
|
||||
applyRemovalsForEntity(db, 'kanji', plan.kanji);
|
||||
}
|
||||
|
||||
function applyRemovalsForEntity(
|
||||
db: DatabaseSync,
|
||||
entity: LexicalEntity,
|
||||
removals: LexicalRemoval[],
|
||||
): void {
|
||||
if (removals.length === 0) return;
|
||||
|
||||
const entityTable = entity === 'word' ? 'imm_words' : 'imm_kanji';
|
||||
const occurrenceTable =
|
||||
entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
|
||||
const col = `${entity}_id`;
|
||||
|
||||
const selectStmt = db.prepare(
|
||||
`SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen
|
||||
FROM ${entityTable}
|
||||
WHERE id = ?`,
|
||||
);
|
||||
const updateFrequencyStmt = db.prepare(`UPDATE ${entityTable} SET frequency = ? WHERE id = ?`);
|
||||
const hasOccurrencesStmt = db.prepare(
|
||||
`SELECT 1 AS found FROM ${occurrenceTable} WHERE ${col} = ? LIMIT 1`,
|
||||
);
|
||||
const deleteStmt = db.prepare(`DELETE FROM ${entityTable} WHERE id = ?`);
|
||||
|
||||
const needsExactRefresh: number[] = [];
|
||||
|
||||
for (const removal of removals) {
|
||||
const current = selectStmt.get(removal.id) as {
|
||||
frequency: number | null;
|
||||
firstSeen: number | null;
|
||||
lastSeen: number | null;
|
||||
} | null;
|
||||
if (!current) continue;
|
||||
|
||||
const nextFrequency = (current.frequency ?? 0) - removal.removedFrequency;
|
||||
if (nextFrequency <= 0) {
|
||||
// The rows in scope are already gone by now, so anything still pointing at
|
||||
// this entity means the stored frequency was stale rather than exhausted.
|
||||
if (hasOccurrencesStmt.get(removal.id)) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
} else {
|
||||
deleteStmt.run(removal.id);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const removedFirstSeen = toStoredSeenSeconds(removal.removedFirstSeenMs);
|
||||
const removedLastSeen = toStoredSeenSeconds(removal.removedLastSeenMs);
|
||||
const firstSeenMayHaveMoved =
|
||||
current.firstSeen === null ||
|
||||
(removedFirstSeen !== null && removedFirstSeen <= current.firstSeen);
|
||||
const lastSeenMayHaveMoved =
|
||||
current.lastSeen === null ||
|
||||
(removedLastSeen !== null && removedLastSeen >= current.lastSeen);
|
||||
if (firstSeenMayHaveMoved || lastSeenMayHaveMoved) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
updateFrequencyStmt.run(nextFrequency, removal.id);
|
||||
}
|
||||
|
||||
if (entity === 'word') {
|
||||
refreshWordAggregates(db, needsExactRefresh);
|
||||
} else {
|
||||
refreshKanjiAggregates(db, needsExactRefresh);
|
||||
}
|
||||
}
|
||||
|
||||
function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
|
||||
if (wordIds.length === 0) {
|
||||
return;
|
||||
@@ -375,18 +214,11 @@ function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
|
||||
SELECT
|
||||
w.id AS wordId,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
|
||||
MIN(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS firstSeen,
|
||||
MAX(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS lastSeen
|
||||
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
|
||||
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
|
||||
FROM imm_words w
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE w.id IN (${makePlaceholders(wordIds)})
|
||||
GROUP BY w.id
|
||||
`,
|
||||
@@ -431,18 +263,11 @@ function refreshKanjiAggregates(db: DatabaseSync, kanjiIds: number[]): void {
|
||||
SELECT
|
||||
k.id AS kanjiId,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
|
||||
MIN(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS firstSeen,
|
||||
MAX(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS lastSeen
|
||||
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
|
||||
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
|
||||
FROM imm_kanji k
|
||||
LEFT JOIN imm_kanji_line_occurrences o ON o.kanji_id = k.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE k.id IN (${makePlaceholders(kanjiIds)})
|
||||
GROUP BY k.id
|
||||
`,
|
||||
|
||||
@@ -192,33 +192,6 @@ function addColumnIfMissing(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy each subtitle line's timestamp onto its word/kanji occurrence rows.
|
||||
*
|
||||
* Vocabulary aggregates used to be recomputed by joining every occurrence back
|
||||
* to `imm_subtitle_lines` just to read two timestamps, which meant one random
|
||||
* read into the widest table per occurrence — the reason deleting a session cost
|
||||
* seconds on a large library. With the timestamp stored alongside the count, the
|
||||
* covering index answers those aggregates on its own.
|
||||
*/
|
||||
function backfillLexicalOccurrenceSeenMs(db: DatabaseSync): void {
|
||||
for (const table of ['imm_word_line_occurrences', 'imm_kanji_line_occurrences']) {
|
||||
addColumnIfMissing(db, table, 'seen_ms', 'INTEGER');
|
||||
db.exec(`
|
||||
UPDATE ${table}
|
||||
SET seen_ms = (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = ${table}.line_id
|
||||
)
|
||||
WHERE seen_ms IS NULL
|
||||
`);
|
||||
}
|
||||
// Superseded by the covering (entity, seen_ms, occurrence_count, line_id) indexes.
|
||||
db.exec('DROP INDEX IF EXISTS idx_word_line_occurrences_word');
|
||||
db.exec('DROP INDEX IF EXISTS idx_kanji_line_occurrences_kanji');
|
||||
}
|
||||
|
||||
function dropColumnIfExists(db: DatabaseSync, tableName: string, columnName: string): void {
|
||||
if (hasColumn(db, tableName, columnName)) {
|
||||
db.exec(`ALTER TABLE ${tableName} DROP COLUMN ${columnName}`);
|
||||
@@ -950,7 +923,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -961,7 +933,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -1122,7 +1093,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -1133,7 +1103,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -1380,19 +1349,13 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_anime_line
|
||||
ON imm_subtitle_lines(anime_id, line_index)
|
||||
`);
|
||||
if (currentVersion?.schema_version && currentVersion.schema_version < 19) {
|
||||
backfillLexicalOccurrenceSeenMs(db);
|
||||
}
|
||||
|
||||
// Covering indexes: vocabulary aggregates (frequency, first/last seen) are
|
||||
// answered from the index alone, without reading the wide subtitle-line rows.
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word_seen
|
||||
ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id)
|
||||
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word
|
||||
ON imm_word_line_occurrences(word_id, line_id)
|
||||
`);
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji_seen
|
||||
ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id)
|
||||
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji
|
||||
ON imm_kanji_line_occurrences(kanji_id, line_id)
|
||||
`);
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_media_art_cover_blob_hash
|
||||
@@ -1512,23 +1475,21 @@ export function createTrackerPreparedStatements(db: DatabaseSync): TrackerPrepar
|
||||
`),
|
||||
wordLineOccurrenceUpsertStmt: db.prepare(`
|
||||
INSERT INTO imm_word_line_occurrences (
|
||||
line_id, word_id, occurrence_count, seen_ms
|
||||
line_id, word_id, occurrence_count
|
||||
) VALUES (
|
||||
?, ?, ?, ?
|
||||
?, ?, ?
|
||||
)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count
|
||||
`),
|
||||
kanjiLineOccurrenceUpsertStmt: db.prepare(`
|
||||
INSERT INTO imm_kanji_line_occurrences (
|
||||
line_id, kanji_id, occurrence_count, seen_ms
|
||||
line_id, kanji_id, occurrence_count
|
||||
) VALUES (
|
||||
?, ?, ?, ?
|
||||
?, ?, ?
|
||||
)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
|
||||
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_kanji_line_occurrences.seen_ms, excluded.seen_ms)
|
||||
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count
|
||||
`),
|
||||
videoAnimeIdSelectStmt: db.prepare(`
|
||||
SELECT anime_id FROM imm_videos
|
||||
@@ -1669,16 +1630,11 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
|
||||
const lineId = Number(lineResult.lastInsertRowid);
|
||||
for (const occurrence of write.wordOccurrences) {
|
||||
const wordId = incrementWordAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
|
||||
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount, currentMs);
|
||||
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount);
|
||||
}
|
||||
for (const occurrence of write.kanjiOccurrences) {
|
||||
const kanjiId = incrementKanjiAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
|
||||
stmts.kanjiLineOccurrenceUpsertStmt.run(
|
||||
lineId,
|
||||
kanjiId,
|
||||
occurrence.occurrenceCount,
|
||||
currentMs,
|
||||
);
|
||||
stmts.kanjiLineOccurrenceUpsertStmt.run(lineId, kanjiId, occurrence.occurrenceCount);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
export const SCHEMA_VERSION = 19;
|
||||
export const SCHEMA_VERSION = 18;
|
||||
export const DEFAULT_QUEUE_CAP = 1_000;
|
||||
export const DEFAULT_BATCH_SIZE = 25;
|
||||
export const DEFAULT_FLUSH_INTERVAL_MS = 500;
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { Hono } from 'hono';
|
||||
import type { ImmersionTrackerService } from './immersion-tracker-service.js';
|
||||
import { statsJson, type StatsCoverImagesRequest } from '../../types/stats-http-contract.js';
|
||||
@@ -75,31 +74,6 @@ function detectImageContentType(bytes: Uint8Array): string {
|
||||
return 'application/octet-stream';
|
||||
}
|
||||
|
||||
function ifNoneMatchHits(header: string | undefined, etag: string): boolean {
|
||||
if (!header) return false;
|
||||
if (header.trim() === '*') return true;
|
||||
return header
|
||||
.split(',')
|
||||
.map((candidate) => candidate.trim().replace(/^W\//, ''))
|
||||
.some((candidate) => candidate === etag);
|
||||
}
|
||||
|
||||
// Cover URLs are stable per anime/video, so relinking an AniList entry swaps the
|
||||
// bytes behind an unchanged URL. Long max-age would keep serving the old art from
|
||||
// the browser cache; revalidate instead and let the ETag skip the transfer.
|
||||
function coverResponse(bytes: Uint8Array<ArrayBuffer>, ifNoneMatch: string | undefined): Response {
|
||||
const etag = `"${createHash('sha256').update(bytes).digest('hex')}"`;
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Type': detectImageContentType(bytes),
|
||||
'Cache-Control': 'no-cache',
|
||||
ETag: etag,
|
||||
};
|
||||
if (ifNoneMatchHits(ifNoneMatch, etag)) {
|
||||
return new Response(null, { status: 304, headers });
|
||||
}
|
||||
return new Response(bytes, { headers });
|
||||
}
|
||||
|
||||
function createLimitedTaskRunner(maxConcurrentTasks: number): (task: () => Promise<void>) => void {
|
||||
const queue: Array<() => Promise<void>> = [];
|
||||
let activeTasks = 0;
|
||||
@@ -163,7 +137,13 @@ export function registerStatsCoverRoutes(app: Hono, tracker: ImmersionTrackerSer
|
||||
art = await tracker.getAnimeCoverArt(animeId);
|
||||
}
|
||||
if (!art?.coverBlob) return c.body(null, 404);
|
||||
return coverResponse(new Uint8Array(art.coverBlob), c.req.header('if-none-match'));
|
||||
const bytes = new Uint8Array(art.coverBlob);
|
||||
return new Response(bytes, {
|
||||
headers: {
|
||||
'Content-Type': detectImageContentType(bytes),
|
||||
'Cache-Control': 'public, max-age=86400',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
app.get('/api/stats/media/:videoId/cover', async (c) => {
|
||||
@@ -175,6 +155,12 @@ export function registerStatsCoverRoutes(app: Hono, tracker: ImmersionTrackerSer
|
||||
art = await tracker.getCoverArt(videoId);
|
||||
}
|
||||
if (!art?.coverBlob) return c.body(null, 404);
|
||||
return coverResponse(new Uint8Array(art.coverBlob), c.req.header('if-none-match'));
|
||||
const bytes = new Uint8Array(art.coverBlob);
|
||||
return new Response(bytes, {
|
||||
headers: {
|
||||
'Content-Type': detectImageContentType(bytes),
|
||||
'Cache-Control': 'public, max-age=604800',
|
||||
},
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -190,11 +190,4 @@ export function registerStatsLibraryRoutes(
|
||||
await tracker.deleteVideo(videoId);
|
||||
return c.json(statsJson('deleteVideo', { ok: true }));
|
||||
});
|
||||
|
||||
app.delete('/api/stats/anime/:animeId', async (c) => {
|
||||
const animeId = parseIntQuery(c.req.param('animeId'), 0);
|
||||
if (animeId <= 0) return c.body(null, 400);
|
||||
await tracker.deleteAnime(animeId);
|
||||
return c.json(statsJson('deleteAnime', { ok: true }));
|
||||
});
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
resolveSecondarySubtitleTextFromSidecar,
|
||||
} from '../secondary-subtitle-sidecar.js';
|
||||
import {
|
||||
applyStatsWordCardFields,
|
||||
applyStatsWordAndSentenceCardFields,
|
||||
createStatsMiningContext,
|
||||
getStatsDirectMiningAudioFieldNames,
|
||||
getStatsWordMiningAudioFieldName,
|
||||
@@ -272,7 +272,7 @@ export function registerStatsMiningRoutes(app: Hono, options?: StatsMiningRouteO
|
||||
const imageFieldName = ankiConfig.fields?.image ?? 'Picture';
|
||||
|
||||
mediaFields[sentenceFieldName] = highlightedSentence;
|
||||
applyStatsWordCardFields(mediaFields, noteInfo, ankiConfig);
|
||||
applyStatsWordAndSentenceCardFields(mediaFields, noteInfo, ankiConfig);
|
||||
|
||||
if (audioBuffer) {
|
||||
const audioFilename = `subminer_audio_${timestamp}_${noteId}.mp3`;
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
import type { MediaGenerator } from '../../../media-generator.js';
|
||||
import type { AnkiConnectConfig } from '../../../types.js';
|
||||
import { applyCardKindFlagFields } from '../../../anki-integration/card-kinds.js';
|
||||
import { resolveWordCardKind } from '../../../anki-integration/note-field-utils.js';
|
||||
import { createLogger } from '../../../logger.js';
|
||||
import type { RetimedSecondarySubtitleInput } from '../secondary-subtitle-sidecar.js';
|
||||
|
||||
@@ -96,22 +94,20 @@ export function shouldUseStatsLapisKikuCardFields(ankiConfig: AnkiConnectConfig)
|
||||
return ankiConfig.isLapis?.enabled === true || ankiConfig.isKiku?.enabled === true;
|
||||
}
|
||||
|
||||
export function applyStatsWordCardFields(
|
||||
export function applyStatsWordAndSentenceCardFields(
|
||||
fields: Record<string, string>,
|
||||
noteInfo: StatsServerNoteInfo | null,
|
||||
ankiConfig: AnkiConnectConfig,
|
||||
): void {
|
||||
if (!noteInfo) return;
|
||||
const cardKind = resolveWordCardKind(noteInfo, {
|
||||
lapisEnabled: ankiConfig.isLapis?.enabled === true,
|
||||
kikuEnabled: ankiConfig.isKiku?.enabled === true,
|
||||
wordCardKind: ankiConfig.lapisKiku?.wordCardKind,
|
||||
});
|
||||
if (!cardKind) return;
|
||||
if (!shouldUseStatsLapisKikuCardFields(ankiConfig) || !noteInfo) return;
|
||||
const wordAndSentenceFlag = resolveStatsNoteFieldName(noteInfo, 'IsWordAndSentenceCard');
|
||||
if (!wordAndSentenceFlag) return;
|
||||
|
||||
applyCardKindFlagFields(fields, cardKind, (preferredName) =>
|
||||
resolveStatsNoteFieldName(noteInfo, preferredName),
|
||||
);
|
||||
fields[wordAndSentenceFlag] = 'x';
|
||||
for (const flagName of ['IsSentenceCard', 'IsAudioCard']) {
|
||||
const resolved = resolveStatsNoteFieldName(noteInfo, flagName);
|
||||
if (resolved && resolved !== wordAndSentenceFlag) fields[resolved] = '';
|
||||
}
|
||||
}
|
||||
|
||||
export function getStatsDirectMiningAudioFieldNames(
|
||||
|
||||
@@ -1,100 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { Database } from '../immersion-tracker/sqlite';
|
||||
import { ensureSchema } from '../immersion-tracker/storage';
|
||||
import { mergeSnapshotIntoDb } from './merge';
|
||||
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
const DAY_MS = 86_400_000;
|
||||
|
||||
/**
|
||||
* Build a database holding one anime, one episode, one ended session and a
|
||||
* single word occurrence.
|
||||
*
|
||||
* `legacyOccurrences` reproduces a peer whose schema predates the denormalised
|
||||
* `seen_ms` column, so the merge has to recover the timestamp from the line.
|
||||
*/
|
||||
function buildDb(
|
||||
dir: string,
|
||||
name: string,
|
||||
options: { word: string; seenMs: number; legacyOccurrences: boolean },
|
||||
): string {
|
||||
const dbPath = path.join(dir, name);
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'key-${name}', 'Show ${name}', ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'video-${name}', 1, 'Episode 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'uuid-${name}', 1, '${options.seenMs}', '${options.seenMs + 1000}', 2, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 1, 1, 1, 0, 'line', ${options.seenMs}, ${options.seenMs});
|
||||
INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (1, '${options.word}', '${options.word}', '', 'noun', '名詞',
|
||||
${Math.floor(options.seenMs / 1000)}, ${Math.floor(options.seenMs / 1000)}, 1);
|
||||
`);
|
||||
db.exec(
|
||||
options.legacyOccurrences
|
||||
? 'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (1, 1, 1)'
|
||||
: `INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (1, 1, 1, ${options.seenMs})`,
|
||||
);
|
||||
db.close();
|
||||
return dbPath;
|
||||
}
|
||||
|
||||
function mergedOccurrences(dbPath: string): Array<{ word: string; seenMs: number | null }> {
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT w.word AS word, o.seen_ms AS seenMs
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_words w ON w.id = o.word_id
|
||||
ORDER BY w.word`,
|
||||
)
|
||||
.all() as Array<{ word: string; seenMs: number | null }>;
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
for (const legacyOccurrences of [false, true]) {
|
||||
const label = legacyOccurrences ? 'a peer predating the seen_ms column' : 'a current peer';
|
||||
|
||||
test(`sync merge carries occurrence timestamps in from ${label}`, () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-merge-occurrences-test-'));
|
||||
try {
|
||||
const local = buildDb(dir, 'local.sqlite', {
|
||||
word: '猫',
|
||||
seenMs: BASE_MS,
|
||||
legacyOccurrences: false,
|
||||
});
|
||||
const remoteSeenMs = BASE_MS + 5 * DAY_MS;
|
||||
const remote = buildDb(dir, 'remote.sqlite', {
|
||||
word: '犬',
|
||||
seenMs: remoteSeenMs,
|
||||
legacyOccurrences,
|
||||
});
|
||||
|
||||
mergeSnapshotIntoDb(local, remote);
|
||||
|
||||
const rows = mergedOccurrences(local);
|
||||
assert.equal(rows.length, 2, 'local and remote occurrences both survive the merge');
|
||||
for (const row of rows) {
|
||||
assert.ok(row.seenMs, `${row.word} must carry a timestamp so aggregates stay index-only`);
|
||||
}
|
||||
assert.equal(
|
||||
Number(rows.find((row) => row.word === '犬')?.seenMs),
|
||||
remoteSeenMs,
|
||||
"the remote line's own timestamp is preserved rather than stamped with merge time",
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -215,16 +215,12 @@ function copySubtitleLines(
|
||||
'SELECT kanji_id, occurrence_count FROM imm_kanji_line_occurrences WHERE line_id = ?',
|
||||
);
|
||||
const insertWordOccurrence = local.query(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count) VALUES (?, ?, ?)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
const insertKanjiOccurrence = local.query(
|
||||
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
|
||||
occurrence_count = occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
|
||||
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count) VALUES (?, ?, ?)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
|
||||
for (const row of rows) {
|
||||
@@ -245,20 +241,17 @@ function copySubtitleLines(
|
||||
],
|
||||
);
|
||||
summary.subtitleLinesAdded += 1;
|
||||
// Taken from the line rather than the remote occurrence row so this also
|
||||
// works when the remote database predates the seen_ms column.
|
||||
const seenMs = row.CREATED_DATE ?? row.LAST_UPDATE_DATE ?? null;
|
||||
|
||||
for (const occurrence of wordOccurrences.all(row.line_id) as SqlRow[]) {
|
||||
const localWordId = lexicon.resolveWord(Number(occurrence.word_id));
|
||||
const count = Number(occurrence.occurrence_count);
|
||||
insertWordOccurrence.run(localLineId, localWordId, count, seenMs);
|
||||
insertWordOccurrence.run(localLineId, localWordId, count);
|
||||
lexicon.addWordOccurrences(Number(occurrence.word_id), count);
|
||||
}
|
||||
for (const occurrence of kanjiOccurrences.all(row.line_id) as SqlRow[]) {
|
||||
const localKanjiId = lexicon.resolveKanji(Number(occurrence.kanji_id));
|
||||
const count = Number(occurrence.occurrence_count);
|
||||
insertKanjiOccurrence.run(localLineId, localKanjiId, count, seenMs);
|
||||
insertKanjiOccurrence.run(localLineId, localKanjiId, count);
|
||||
lexicon.addKanjiOccurrences(Number(occurrence.kanji_id), count);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -85,5 +85,5 @@ test('writeConfigExampleArtifacts creates parent directories for resolved output
|
||||
test('generateConfigExampleTemplate uses the canonical example socket path', () => {
|
||||
const template = generateConfigExampleTemplate();
|
||||
|
||||
assert.match(template, /"socketPath": "\/tmp\/subminer-socket"/);
|
||||
assert.match(template, /"socketPath": "\\\\\\\\.\\\\pipe\\\\subminer-socket"/);
|
||||
});
|
||||
|
||||
@@ -3,7 +3,7 @@ import * as path from 'path';
|
||||
import { DEFAULT_CONFIG, deepCloneConfig, generateConfigTemplate } from './config';
|
||||
import { getDefaultMpvSocketPath } from './shared/mpv-socket-path';
|
||||
|
||||
const CONFIG_EXAMPLE_PLATFORM: NodeJS.Platform = 'linux';
|
||||
const CONFIG_EXAMPLE_PLATFORM: NodeJS.Platform = 'win32';
|
||||
|
||||
export function generateConfigExampleTemplate(): string {
|
||||
const config = deepCloneConfig(DEFAULT_CONFIG);
|
||||
|
||||
+4
-11
@@ -3616,12 +3616,10 @@ const {
|
||||
);
|
||||
},
|
||||
refreshAnilistClientSecretState: () => refreshAnilistClientSecretState(),
|
||||
updateAnilistPostWatchProgress: (accessToken, title, episode, season, mediaId) =>
|
||||
updateAnilistPostWatchProgress: (accessToken, title, episode, season) =>
|
||||
updateAnilistPostWatchProgress(accessToken, title, episode, {
|
||||
rateLimiter: anilistRateLimiter,
|
||||
season,
|
||||
mediaId,
|
||||
logInfo: (message) => logger.info(message),
|
||||
}),
|
||||
markSuccess: (key) => {
|
||||
anilistUpdateQueue.markSuccess(key);
|
||||
@@ -3657,12 +3655,9 @@ const {
|
||||
hasAttemptedUpdateKey: (key) => anilistAttemptedUpdateKeys.has(key),
|
||||
processNextAnilistRetryUpdate: () => processNextAnilistRetryUpdate(),
|
||||
refreshAnilistClientSecretState: () => refreshAnilistClientSecretState(),
|
||||
enqueueRetry: (key, title, episode, season, mediaId) => {
|
||||
anilistUpdateQueue.enqueue(key, title, episode, season, mediaId);
|
||||
enqueueRetry: (key, title, episode, season) => {
|
||||
anilistUpdateQueue.enqueue(key, title, episode, season);
|
||||
},
|
||||
getCurrentMediaTitle: () => appState.currentMediaTitle,
|
||||
resolvePinnedAnilistMediaId: ({ mediaPath, mediaTitle, guess }) =>
|
||||
characterDictionaryRuntime.resolvePinnedMediaId({ mediaPath, mediaTitle, guess }),
|
||||
markRetryFailure: (key, message) => {
|
||||
anilistUpdateQueue.markFailure(key, message);
|
||||
},
|
||||
@@ -3670,12 +3665,10 @@ const {
|
||||
anilistUpdateQueue.markSuccess(key);
|
||||
},
|
||||
refreshRetryQueueState: () => anilistStateRuntime.refreshRetryQueueState(),
|
||||
updateAnilistPostWatchProgress: (accessToken, title, episode, season, mediaId) =>
|
||||
updateAnilistPostWatchProgress: (accessToken, title, episode, season) =>
|
||||
updateAnilistPostWatchProgress(accessToken, title, episode, {
|
||||
rateLimiter: anilistRateLimiter,
|
||||
season,
|
||||
mediaId,
|
||||
logInfo: (message) => logger.info(message),
|
||||
}),
|
||||
rememberAttemptedUpdateKey: (key) => {
|
||||
rememberAnilistAttemptedUpdate(key);
|
||||
|
||||
@@ -36,7 +36,6 @@ import {
|
||||
import {
|
||||
buildCharacterDictionarySeriesKey,
|
||||
createCharacterDictionaryManualSelectionStore,
|
||||
type CharacterDictionarySeriesKeyGuess,
|
||||
} from './character-dictionary-runtime/manual-selection';
|
||||
import { snapshotHasCharacterNameImages } from './character-dictionary-runtime/image-lookup';
|
||||
import { resolveJapaneseNameSplits } from './character-dictionary-runtime/name-split-resolver';
|
||||
@@ -169,11 +168,6 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
targetPath?: string,
|
||||
options?: CharacterDictionaryGenerateOptions,
|
||||
) => Promise<CharacterDictionaryBuildResult>;
|
||||
resolvePinnedMediaId: (input: {
|
||||
mediaPath: string | null;
|
||||
mediaTitle: string | null;
|
||||
guess: CharacterDictionarySeriesKeyGuess | null;
|
||||
}) => Promise<number | null>;
|
||||
} {
|
||||
const outputDir = path.join(deps.userDataPath, 'character-dictionaries');
|
||||
const sleepMs = deps.sleep ?? sleep;
|
||||
@@ -278,7 +272,7 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
: ''
|
||||
}`,
|
||||
);
|
||||
const override = await manualSelectionStore.getOverride(seriesKey, guessed.season);
|
||||
const override = await manualSelectionStore.getOverride(seriesKey);
|
||||
if (override) {
|
||||
deps.logInfo?.(
|
||||
`[dictionary] manual AniList override: ${override.mediaTitle} -> AniList ${override.mediaId}`,
|
||||
@@ -320,17 +314,7 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
};
|
||||
}
|
||||
|
||||
const resolved = await resolveAniListMediaIdFromGuess(guessed, beforeRequest, (message) =>
|
||||
deps.logInfo?.(message),
|
||||
);
|
||||
if (resolved.seasonResolved === false) {
|
||||
deps.logWarn?.(
|
||||
`[dictionary] could not find season ${resolved.requestedSeason} of "${guessed.title}"; using ${resolved.title} (AniList ${resolved.id}). Set a manual AniList match to correct it.`,
|
||||
);
|
||||
// Caching the fallback would make the wrong season stick silently for every later
|
||||
// episode, so leave it uncached and re-resolve (and re-warn) until it is corrected.
|
||||
return resolved;
|
||||
}
|
||||
const resolved = await resolveAniListMediaIdFromGuess(guessed, beforeRequest);
|
||||
writeCachedMediaResolution(outputDir, {
|
||||
seriesKey,
|
||||
mediaId: resolved.id,
|
||||
@@ -525,7 +509,7 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
? await searchAniListMediaCandidates(candidateSearchTitle, waitForAniListRequestSlot)
|
||||
: [];
|
||||
const [override, current] = await Promise.all([
|
||||
manualSelectionStore.getOverride(seriesKey, guessed.season),
|
||||
manualSelectionStore.getOverride(seriesKey),
|
||||
shouldUseExplicitSearch
|
||||
? Promise.resolve(null)
|
||||
: resolveAniListMediaIdFromGuess(guessed, waitForAniListRequestSlot)
|
||||
@@ -569,7 +553,6 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
mediaId: selected.id,
|
||||
mediaTitle: selected.title,
|
||||
staleMediaIds,
|
||||
season: guessed.season,
|
||||
});
|
||||
return {
|
||||
ok: true,
|
||||
@@ -622,14 +605,5 @@ export function createCharacterDictionaryRuntimeService(deps: CharacterDictionar
|
||||
revision,
|
||||
};
|
||||
},
|
||||
resolvePinnedMediaId: async ({ mediaPath, mediaTitle, guess }) => {
|
||||
const seriesKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: deps.resolveMediaPathForJimaku(mediaPath),
|
||||
mediaTitle,
|
||||
guess,
|
||||
});
|
||||
const override = await manualSelectionStore.getOverride(seriesKey, guess?.season ?? null);
|
||||
return override?.mediaId ?? null;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import type { AnilistMediaGuess } from '../../core/services/anilist/anilist-updater';
|
||||
import { resolveAnilistSeasonMedia } from '../../core/services/anilist/season-resolver';
|
||||
import { ANILIST_GRAPHQL_URL } from './constants';
|
||||
import type {
|
||||
AniListMediaCandidate,
|
||||
@@ -74,6 +73,57 @@ type AniListCharacterPageResponse = {
|
||||
} | null;
|
||||
};
|
||||
|
||||
function normalizeTitle(value: string): string {
|
||||
return value.trim().toLowerCase().replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
function pickAniListSearchResult(
|
||||
title: string,
|
||||
episode: number | null,
|
||||
media: Array<{
|
||||
id: number;
|
||||
episodes?: number | null;
|
||||
title?: {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
native?: string | null;
|
||||
};
|
||||
}>,
|
||||
): ResolvedAniListMedia | null {
|
||||
if (media.length === 0) return null;
|
||||
|
||||
const episodeFiltered =
|
||||
episode && episode > 0
|
||||
? media.filter((entry) => {
|
||||
const totalEpisodes = entry.episodes;
|
||||
return (
|
||||
typeof totalEpisodes !== 'number' || totalEpisodes <= 0 || episode <= totalEpisodes
|
||||
);
|
||||
})
|
||||
: media;
|
||||
const candidates = episodeFiltered.length > 0 ? episodeFiltered : media;
|
||||
const normalizedTitle = normalizeTitle(title);
|
||||
|
||||
const exact = candidates.find((entry) => {
|
||||
const titles = [entry.title?.english, entry.title?.romaji, entry.title?.native]
|
||||
.filter((value): value is string => typeof value === 'string')
|
||||
.map((value) => normalizeTitle(value));
|
||||
return titles.includes(normalizedTitle);
|
||||
});
|
||||
const selected = exact ?? candidates[0] ?? media[0];
|
||||
if (!selected) return null;
|
||||
|
||||
const selectedTitle =
|
||||
selected.title?.english?.trim() ||
|
||||
selected.title?.romaji?.trim() ||
|
||||
selected.title?.native?.trim() ||
|
||||
title.trim();
|
||||
return {
|
||||
id: selected.id,
|
||||
title: selectedTitle,
|
||||
};
|
||||
}
|
||||
|
||||
function toAniListMediaCandidate(
|
||||
entry: {
|
||||
id: number;
|
||||
@@ -192,24 +242,35 @@ function inferImageExt(contentType: string | null, bytes: Buffer): string {
|
||||
export async function resolveAniListMediaIdFromGuess(
|
||||
guess: AnilistMediaGuess,
|
||||
beforeRequest?: () => Promise<void>,
|
||||
logInfo?: (message: string) => void,
|
||||
): Promise<ResolvedAniListMedia> {
|
||||
const resolution = await resolveAnilistSeasonMedia(
|
||||
{ title: guess.title, season: guess.season, episode: guess.episode },
|
||||
const data = await fetchAniList<AniListSearchResponse>(
|
||||
`
|
||||
query($search: String!) {
|
||||
Page(perPage: 10) {
|
||||
media(search: $search, type: ANIME, sort: [SEARCH_MATCH, POPULARITY_DESC]) {
|
||||
id
|
||||
episodes
|
||||
title {
|
||||
romaji
|
||||
english
|
||||
native
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`,
|
||||
{
|
||||
execute: (query, variables) => fetchAniList(query, variables, beforeRequest),
|
||||
logInfo,
|
||||
search: guess.title,
|
||||
},
|
||||
beforeRequest,
|
||||
);
|
||||
if (!resolution) {
|
||||
|
||||
const media = data.Page?.media ?? [];
|
||||
const resolved = pickAniListSearchResult(guess.title, guess.episode, media);
|
||||
if (!resolved) {
|
||||
throw new Error(`No AniList media match found for "${guess.title}".`);
|
||||
}
|
||||
return {
|
||||
id: resolution.id,
|
||||
title: resolution.title,
|
||||
seasonResolved: resolution.seasonResolved,
|
||||
requestedSeason: resolution.requestedSeason,
|
||||
};
|
||||
return resolved;
|
||||
}
|
||||
|
||||
export async function searchAniListMediaCandidates(
|
||||
|
||||
@@ -160,73 +160,3 @@ test('getManualSelectionSnapshot hydrates override episode count from searched c
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('resolvePinnedMediaId returns the manual override for the current season', async () => {
|
||||
const userDataPath = makeTempDir();
|
||||
const seasonThreePath = '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S03E01.mkv';
|
||||
const guess = {
|
||||
title: 'My Teen Romantic Comedy SNAFU',
|
||||
year: 2013,
|
||||
season: 3,
|
||||
episode: 1,
|
||||
source: 'guessit' as const,
|
||||
};
|
||||
|
||||
const runtime = createCharacterDictionaryRuntimeService({
|
||||
userDataPath,
|
||||
getCurrentMediaPath: () => seasonThreePath,
|
||||
getCurrentMediaTitle: () => null,
|
||||
resolveMediaPathForJimaku: (mediaPath) => mediaPath,
|
||||
guessAnilistMediaInfo: async () => guess,
|
||||
now: () => 1_700_000_000_000,
|
||||
});
|
||||
|
||||
assert.equal(
|
||||
await runtime.resolvePinnedMediaId({
|
||||
mediaPath: seasonThreePath,
|
||||
mediaTitle: null,
|
||||
guess,
|
||||
}),
|
||||
null,
|
||||
);
|
||||
|
||||
const overridesPath = path.join(userDataPath, 'character-dictionaries', 'anilist-overrides.json');
|
||||
fs.mkdirSync(path.dirname(overridesPath), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
overridesPath,
|
||||
JSON.stringify({
|
||||
overrides: [
|
||||
{
|
||||
seriesKey: buildCharacterDictionarySeriesKey({
|
||||
mediaPath: seasonThreePath,
|
||||
mediaTitle: null,
|
||||
guess,
|
||||
}),
|
||||
mediaId: 108489,
|
||||
mediaTitle: 'My Teen Romantic Comedy SNAFU Climax!',
|
||||
staleMediaIds: [],
|
||||
},
|
||||
],
|
||||
}),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
await runtime.resolvePinnedMediaId({
|
||||
mediaPath: seasonThreePath,
|
||||
mediaTitle: null,
|
||||
guess,
|
||||
}),
|
||||
108489,
|
||||
);
|
||||
|
||||
// The season 1 file in the same folder must not inherit the season 3 override.
|
||||
assert.equal(
|
||||
await runtime.resolvePinnedMediaId({
|
||||
mediaPath: '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S01E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { ...guess, season: 1 },
|
||||
}),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -312,131 +312,3 @@ test('manual selection store keeps overrides separate for different season direc
|
||||
assert.notEqual(secondSeasonKey, firstSeasonKey);
|
||||
assert.equal(await store.getOverride(secondSeasonKey), null);
|
||||
});
|
||||
|
||||
test('buildCharacterDictionarySeriesKey tags seasons past the first', () => {
|
||||
const base = {
|
||||
title: 'My Teen Romantic Comedy SNAFU',
|
||||
year: 2013,
|
||||
episode: 1,
|
||||
source: 'guessit' as const,
|
||||
};
|
||||
const seasonOne = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S01E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { ...base, season: 1 },
|
||||
});
|
||||
const seasonThree = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S03E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { ...base, season: 3 },
|
||||
});
|
||||
|
||||
// Season 1 keeps the pre-existing key shape so cached snapshots stay valid.
|
||||
assert.equal(seasonOne, 'anime-oregairu--my-teen-romantic-comedy-snafu-2013');
|
||||
assert.equal(seasonThree, 'anime-oregairu--my-teen-romantic-comedy-snafu-s3-2013');
|
||||
});
|
||||
|
||||
test('manual selection store keeps seasons apart inside one flat directory', async () => {
|
||||
const userDataPath = makeTempDir();
|
||||
const store = createCharacterDictionaryManualSelectionStore({ userDataPath });
|
||||
const base = {
|
||||
title: 'My Teen Romantic Comedy SNAFU',
|
||||
year: 2013,
|
||||
episode: 1,
|
||||
source: 'guessit' as const,
|
||||
};
|
||||
const seasonOneKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S01E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { ...base, season: 1 },
|
||||
});
|
||||
const seasonThreeKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Oregairu/My Teen Romantic Comedy SNAFU (2013) - S03E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { ...base, season: 3 },
|
||||
});
|
||||
|
||||
await store.setOverride({
|
||||
seriesKey: seasonOneKey,
|
||||
mediaId: 14813,
|
||||
mediaTitle: 'My Teen Romantic Comedy SNAFU',
|
||||
staleMediaIds: [],
|
||||
});
|
||||
|
||||
// Same directory, different season: the season 1 override must not leak across.
|
||||
assert.equal(await store.getOverride(seasonThreeKey), null);
|
||||
|
||||
await store.setOverride({
|
||||
seriesKey: seasonThreeKey,
|
||||
mediaId: 108489,
|
||||
mediaTitle: 'My Teen Romantic Comedy SNAFU Climax!',
|
||||
staleMediaIds: [],
|
||||
});
|
||||
|
||||
assert.equal((await store.getOverride(seasonOneKey))?.mediaId, 14813);
|
||||
assert.equal((await store.getOverride(seasonThreeKey))?.mediaId, 108489);
|
||||
});
|
||||
|
||||
test('override scope uses the recorded season, not season-like text in the title', async () => {
|
||||
const userDataPath = makeTempDir();
|
||||
const store = createCharacterDictionaryManualSelectionStore({ userDataPath });
|
||||
// A title that itself normalizes to a trailing "-s2" would be misparsed as season 2.
|
||||
const trickyTitleKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Mixed/Some Show S2 - S01E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { title: 'Some Show S2', season: 1, episode: 1, source: 'guessit' },
|
||||
});
|
||||
const realSeasonTwoKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: '/anime/Mixed/Other Show - S02E01.mkv',
|
||||
mediaTitle: null,
|
||||
guess: { title: 'Other Show', season: 2, episode: 1, source: 'guessit' },
|
||||
});
|
||||
|
||||
await store.setOverride({
|
||||
seriesKey: trickyTitleKey,
|
||||
mediaId: 111,
|
||||
mediaTitle: 'Some Show S2',
|
||||
staleMediaIds: [],
|
||||
season: 1,
|
||||
});
|
||||
await store.setOverride({
|
||||
seriesKey: realSeasonTwoKey,
|
||||
mediaId: 222,
|
||||
mediaTitle: 'Other Show 2',
|
||||
staleMediaIds: [],
|
||||
season: 2,
|
||||
});
|
||||
|
||||
// Same directory, genuinely different seasons: neither override may replace the other.
|
||||
assert.equal((await store.getOverride(trickyTitleKey, 1))?.mediaId, 111);
|
||||
assert.equal((await store.getOverride(realSeasonTwoKey, 2))?.mediaId, 222);
|
||||
});
|
||||
|
||||
test('override records without a stored season still resolve via the key', async () => {
|
||||
const userDataPath = makeTempDir();
|
||||
const store = createCharacterDictionaryManualSelectionStore({ userDataPath });
|
||||
const legacyKey = buildCharacterDictionarySeriesKey({
|
||||
mediaPath: REZERO_EP1,
|
||||
mediaTitle: null,
|
||||
guess: {
|
||||
title: 'Re ZERO, Starting Life in Another World',
|
||||
year: 2016,
|
||||
season: 1,
|
||||
episode: 1,
|
||||
source: 'guessit',
|
||||
},
|
||||
});
|
||||
const overridesPath = path.join(userDataPath, 'character-dictionaries', 'anilist-overrides.json');
|
||||
fs.mkdirSync(path.dirname(overridesPath), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
overridesPath,
|
||||
JSON.stringify({
|
||||
overrides: [
|
||||
{ seriesKey: legacyKey, mediaId: 21355, mediaTitle: 'Re:ZERO', staleMediaIds: [] },
|
||||
],
|
||||
}),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
assert.equal((await store.getOverride(legacyKey, 1))?.mediaId, 21355);
|
||||
});
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user