mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-07-28 04:49:49 -07:00
Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
fb212cd575
|
|||
|
998e4b52af
|
|||
|
765df15048
|
|||
|
c1db917210
|
|||
|
f0e1cd6548
|
|||
|
a173b108d1
|
|||
|
a39ca2cbac
|
|||
|
8a8a700ccb
|
@@ -2,4 +2,3 @@ type: added
|
||||
area: overlay
|
||||
|
||||
- Known-word subtitle highlights can now be colored by Anki card maturity (new, learning, young, mature) like asbplayer. Enable with `ankiConnect.knownWords.maturityEnabled`; the mature interval threshold (`matureThresholdDays`, default 21) and the four tier colors (`subtitleStyle.knownWordMaturityColors`) are configurable, and a runtime option toggles it in-session. The session help color legend shows the four tier colors while maturity highlighting is on.
|
||||
- Tiers follow Anki's own card counts: the interval tiers exclude cards in the learning/relearning queue, so a lapsed card shows the learning color instead of young (its interval is reset to at least 1 day, which previously made the learning tier unreachable). A note with a mature card alongside a relearning card still shows mature.
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Stats reported 0 known words for every session after the known-word cache gained maturity tiers. The stats server carried its own copy of the cache parser that only recognized versions up to 3, so the new v4 file was read as "no cache" rather than as a format it should understand.
|
||||
- The cache format, its parser, and the derived known-word set now live in one module that both the cache manager and the stats server read, and the version dispatch ends in an exhaustive check so a future format bump fails the build instead of silently reporting zero. A cache that exists but does not parse now logs a warning rather than passing for an empty one.
|
||||
@@ -2,4 +2,3 @@ type: added
|
||||
area: launcher
|
||||
|
||||
- After a watch-history episode ends or mpv closes, the fzf or rofi launcher returns to that series with options to play the previous episode, rewatch, play the next episode, select another episode, or quit SubMiner. Previous and Next continue across season directories.
|
||||
- The action menu shown right after picking a series from `subminer -H` now also offers the previous episode, matching the menu shown after playback.
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
type: fixed
|
||||
area: launcher
|
||||
|
||||
- Rofi menu prompts now keep a space between the prompt text and the input field instead of running into the search placeholder.
|
||||
@@ -1,5 +0,0 @@
|
||||
type: added
|
||||
area: stats
|
||||
|
||||
- Added a "Delete Entry" action to the stats Library detail view. It removes the whole title in one step — every episode, session, subtitle line, rollup, cover art and vocabulary counts derived from them — and the title disappears from the Library grid. Previously only individual episodes and sessions could be deleted, so a mistakenly created entry had to be cleared episode by episode and still lingered in the library.
|
||||
- Deletion progress is now shown app-wide instead of only on Overview and Sessions. Every delete (session, session group, episode, and library entry) drives a sweeping bar at the top of the window plus a bottom-right status toast, and both stay visible across tab switches, detail views, and the in-app stats overlay window. The per-tab indicators previously went blank the moment you left the tab that started the delete.
|
||||
@@ -1,7 +0,0 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Made deleting stats data much faster and stopped it stalling playback. Every delete used to recompute each affected word's totals by re-reading that word's whole occurrence history across the library, which meant one random read into the largest table per occurrence. On a 960-session library that cost 3.8s to delete a single session and over a minute to clear a 12-episode title, and because the stats server runs in the app process when the app owns it, that time was spent blocking mpv.
|
||||
- Word and kanji occurrences now store the subtitle line's timestamp alongside the count, so those totals are answered from a covering index instead of the subtitle-line table, and a delete only subtracts what it actually removed rather than recomputing untouched entries. Measured on the same 960-session library: deleting one session 3832ms to 190ms, a ten-session day group 6453ms to 433ms, one episode 5437ms to 263ms, and a whole 12-episode title 60556ms to 621ms.
|
||||
- Opening the Vocabulary tab no longer stalls either. It was computing "seen in N titles" for every word in the library before ordering by frequency and keeping the top 100, so the whole occurrence history was walked to produce one page. It now picks the page first and counts only those rows: 2136ms to 62ms on the same library.
|
||||
- The first launch after upgrading migrates the existing stats database in place (about 3s for a 288k-line library) and it grows roughly 20% from the added index. Vocabulary dates recorded before the upgrade keep working while the migration runs.
|
||||
@@ -1,7 +0,0 @@
|
||||
type: internal
|
||||
area: tokenizer
|
||||
|
||||
- Added `verify-known-word-highlights:electron` script: tokenizes a real subtitle file through the app's Yomitan/MeCab pipeline with the live known-word cache, prints each line in the configured tier colors, and summarizes the tier counts so highlighting can be checked outside of playback.
|
||||
- Added `--audit`, which re-derives every highlighted tier from live Anki card data (`notesInfo` + `cardsInfo` intervals) and reports each token whose rendered tier disagrees, catching both stale cache entries and tier-classification bugs.
|
||||
- Added `--profile-copy` so the check can run while SubMiner is open (Electron locks the Yomitan userData dir), plus `--refresh`, `--limit`, `--json`, and `--quiet`.
|
||||
- Added `KnownWordCacheManager.getKnownWordMatchNoteIds`, exposing the note ids behind a known-word match so an audit can trace a rendered tier back to the exact Anki notes.
|
||||
@@ -57,8 +57,6 @@ Jellyfin stream URLs are normalized to stable item links before stats titles are
|
||||
|
||||
When YouTube channel metadata is available, the Library tab groups videos by creator/channel and treats each tracked video as an episode-like entry inside that channel section.
|
||||
|
||||
Open a title and use **Delete Entry** in its header to remove a mistakenly tracked show outright. This deletes every episode of that title along with their sessions, subtitle lines, rollups and cover art, drops the words and kanji that were only seen there, and removes the card from the Library grid. Individual episodes and sessions can still be deleted on their own from the episode list and session rows. Entry deletion is refused while that title is the one currently playing.
|
||||
|
||||

|
||||
|
||||
#### Trends
|
||||
|
||||
@@ -73,7 +73,6 @@ subminer -R -H # rofi history browser
|
||||
|
||||
The first menu lists every locally watched series, most recently watched first, using the parsed media title (e.g. the anime title) when available and the directory name otherwise. Selecting a series opens an action menu:
|
||||
|
||||
- **Previous episode**: plays the episode before the last watched one and continues into the previous season directory when the season starts
|
||||
- **Replay last watched**: replays the most recently watched episode
|
||||
- **Next episode**: plays the episode after the last watched one and continues into the next season directory when the season ends
|
||||
- **Browse episodes**: lists the video files in the series directory in episode order, using the same fzf/rofi episode picker as directory browsing; if the series has multiple season directories, a season menu appears first
|
||||
|
||||
@@ -49,38 +49,26 @@ Instead of one color for every known word, maturity highlighting tints each know
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. During the known-word cache refresh, SubMiner classifies each note with Anki search filters (`prop:ivl`, `is:learn`) - no extra card data is downloaded.
|
||||
2. Each note gets the tier of its **most mature** card: `mature` (in review, interval ≥ threshold), `young` (in review, interval below the threshold), `learning` (in the learning or relearning queue), or `new` (never studied). The buckets are disjoint, matching Anki's own card counts: a lapsed card in relearning counts as `learning`, not `young`, even though its interval is ≥ 1 day. A note with a mature card plus a relearning card still shows `mature`.
|
||||
1. During the known-word cache refresh, SubMiner classifies each note with Anki search filters (`prop:ivl`, `is:learn`, `is:new`) - no extra card data is downloaded.
|
||||
2. Each note gets the tier of its **most mature** card: `mature` (interval ≥ threshold), `young` (in review below the threshold), `learning` (in (re)learning), or `new` (never reviewed).
|
||||
3. A word matched by several notes takes the most mature tier among them, with the same reading-aware matching as regular known-word highlighting.
|
||||
4. Known tokens render in the tier color instead of `subtitleStyle.knownWordColor`; if tier data is missing for a match, the token falls back to the single known-word color.
|
||||
|
||||
**Key settings:**
|
||||
|
||||
| Option | Default | Description |
|
||||
| ------------------------------------------------ | --------- | --------------------------------------------------------------------- |
|
||||
| `ankiConnect.knownWords.maturityEnabled` | `false` | Color known words by card maturity (requires known-word highlighting) |
|
||||
| `ankiConnect.knownWords.matureThresholdDays` | `21` | Card interval in days at which a word counts as mature |
|
||||
| `subtitleStyle.knownWordMaturityColors.new` | `#ee99a0` | Tier color for never-reviewed cards |
|
||||
| `subtitleStyle.knownWordMaturityColors.learning` | `#b7bdf8` | Tier color for cards in the learning/relearning queue |
|
||||
| `subtitleStyle.knownWordMaturityColors.young` | `#91d7e3` | Tier color for young review cards |
|
||||
| `subtitleStyle.knownWordMaturityColors.mature` | `#a6da95` | Tier color for mature cards |
|
||||
| Option | Default | Description |
|
||||
| -------------------------------------------- | --------- | ---------------------------------------------------------- |
|
||||
| `ankiConnect.knownWords.maturityEnabled` | `false` | Color known words by card maturity (requires known-word highlighting) |
|
||||
| `ankiConnect.knownWords.matureThresholdDays` | `21` | Card interval in days at which a word counts as mature |
|
||||
| `subtitleStyle.knownWordMaturityColors.new` | `#ee99a0` | Tier color for never-reviewed cards |
|
||||
| `subtitleStyle.knownWordMaturityColors.learning` | `#b7bdf8` | Tier color for (re)learning cards |
|
||||
| `subtitleStyle.knownWordMaturityColors.young` | `#91d7e3` | Tier color for young review cards |
|
||||
| `subtitleStyle.knownWordMaturityColors.mature` | `#a6da95` | Tier color for mature cards |
|
||||
|
||||
Changing `maturityEnabled` or the threshold triggers a full known-word cache refresh so tiers are refetched, as does upgrading to a build that revises the tier rules.
|
||||
|
||||
How often the `learning` color appears depends on your deck preset: with no relearning steps configured, a lapsed card returns straight to review and shows `young` instead.
|
||||
Changing `maturityEnabled` or the threshold triggers a full known-word cache refresh so tiers are refetched.
|
||||
|
||||
While maturity highlighting is on, the session help color legend replaces its single "Known words" swatch with one row per tier (new, learning, young, mature).
|
||||
|
||||
**Checking the colors you actually see:**
|
||||
|
||||
Tiers are only as fresh as the last known-word cache refresh (`ankiConnect.knownWords.refreshMinutes`), so a card that crosses the mature threshold mid-day keeps its old color until the next refresh. To check a whole episode offline, run the verifier against its subtitle file:
|
||||
|
||||
```sh
|
||||
bun run verify-known-word-highlights:electron -- --input /path/to/episode.ja.srt --audit
|
||||
```
|
||||
|
||||
It tokenizes every cue through the real Yomitan/MeCab pipeline with your live known-word cache, prints each line in your configured tier colors, and summarizes the tier counts. `--audit` re-derives each highlighted tier from live Anki card data (`notesInfo` + `cardsInfo` intervals) and lists any token whose color disagrees, with the note ids and intervals behind it. Electron locks the Yomitan profile, so quit SubMiner first or pass `--profile-copy` to run against a scratch copy. Other useful flags: `--refresh` (refresh the cache first), `--limit <n>`, `--quiet`, `--json`.
|
||||
|
||||
## Character-Name Highlighting
|
||||
|
||||
Character-name matches are built from the active merged SubMiner character dictionary, which auto-syncs character data from AniList for your recently-watched titles. When the current AniList media ID is known, SubMiner ignores loaded entries from other titles for subtitle name matching and inline portraits. Matching names are highlighted in subtitles and become available for hover-driven Yomitan character profiles - portraits, roles, voice actors, and biographical detail.
|
||||
|
||||
@@ -7,7 +7,6 @@ import {
|
||||
collectVideos,
|
||||
findRofiTheme,
|
||||
formatPickerLaunchError,
|
||||
formatRofiPrompt,
|
||||
showFzfMenu,
|
||||
showRofiMenu,
|
||||
} from '../picker.js';
|
||||
@@ -68,34 +67,6 @@ export function buildHistorySessionActions(
|
||||
return actions;
|
||||
}
|
||||
|
||||
export function buildHistoryEntryActions(
|
||||
lastWatchedPath: string | null,
|
||||
previousEpisodePath: string | null,
|
||||
nextEpisodePath: string | null,
|
||||
): HistorySessionMenuAction[] {
|
||||
const actions: HistorySessionMenuAction[] = [];
|
||||
if (previousEpisodePath) {
|
||||
actions.push({
|
||||
kind: 'previous',
|
||||
label: `Previous episode: ${path.basename(previousEpisodePath)}`,
|
||||
});
|
||||
}
|
||||
if (lastWatchedPath) {
|
||||
actions.push({
|
||||
kind: 'replay',
|
||||
label: `Replay last watched: ${path.basename(lastWatchedPath)}`,
|
||||
});
|
||||
}
|
||||
if (nextEpisodePath) {
|
||||
actions.push({ kind: 'next', label: `Next episode: ${path.basename(nextEpisodePath)}` });
|
||||
}
|
||||
actions.push(
|
||||
{ kind: 'browse', label: 'Browse episodes' },
|
||||
{ kind: 'quit', label: 'Quit SubMiner' },
|
||||
);
|
||||
return actions;
|
||||
}
|
||||
|
||||
interface HistoryPlaybackLoopDeps {
|
||||
play: (videoPath: string) => Promise<void>;
|
||||
pickPostPlaybackAction: (input: {
|
||||
@@ -200,16 +171,7 @@ function showRofiIndexMenu(
|
||||
themePath: string | null,
|
||||
icons: Array<string | null> = [],
|
||||
): number {
|
||||
const rofiArgs = [
|
||||
'-dmenu',
|
||||
'-i',
|
||||
'-matching',
|
||||
'fuzzy',
|
||||
'-format',
|
||||
'i',
|
||||
'-p',
|
||||
formatRofiPrompt(prompt),
|
||||
];
|
||||
const rofiArgs = ['-dmenu', '-i', '-matching', 'fuzzy', '-format', 'i', '-p', prompt];
|
||||
const hasIcons = icons.some(Boolean);
|
||||
if (hasIcons) rofiArgs.push('-show-icons');
|
||||
if (themePath) {
|
||||
@@ -370,14 +332,17 @@ export async function runHistoryCommand(
|
||||
|
||||
const lastPath = path.resolve(entry.lastWatched.sourcePath);
|
||||
const lastExists = fs.existsSync(lastPath);
|
||||
const previousEpisode = findPreviousEpisode(lastPath);
|
||||
const nextEpisode = findNextEpisode(lastPath);
|
||||
|
||||
const actions = buildHistoryEntryActions(
|
||||
lastExists ? lastPath : null,
|
||||
previousEpisode,
|
||||
nextEpisode,
|
||||
);
|
||||
const actions: HistorySessionMenuAction[] = [];
|
||||
if (lastExists) {
|
||||
actions.push({ kind: 'replay', label: `Replay last watched: ${path.basename(lastPath)}` });
|
||||
}
|
||||
if (nextEpisode) {
|
||||
actions.push({ kind: 'next', label: `Next episode: ${path.basename(nextEpisode)}` });
|
||||
}
|
||||
actions.push({ kind: 'browse', label: 'Browse episodes' });
|
||||
actions.push({ kind: 'quit', label: 'Quit SubMiner' });
|
||||
|
||||
const entryIcon = seriesIcons[seriesIdx] ?? null;
|
||||
const actionIdx = pickIndex(
|
||||
@@ -392,14 +357,13 @@ export async function runHistoryCommand(
|
||||
switch (actions[actionIdx]!.kind) {
|
||||
case 'replay':
|
||||
return { entry, videoPath: lastPath, themePath, entryIcon };
|
||||
case 'previous':
|
||||
return previousEpisode ? { entry, videoPath: previousEpisode, themePath, entryIcon } : null;
|
||||
case 'next':
|
||||
return nextEpisode ? { entry, videoPath: nextEpisode, themePath, entryIcon } : null;
|
||||
case 'browse': {
|
||||
const videoPath = browseEpisodes(entry, context, themePath);
|
||||
return videoPath ? { entry, videoPath, themePath, entryIcon } : null;
|
||||
}
|
||||
case 'previous':
|
||||
case 'quit':
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -1,11 +1,7 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import path from 'node:path';
|
||||
import {
|
||||
buildHistoryEntryActions,
|
||||
buildHistorySessionActions,
|
||||
runHistoryPlaybackLoop,
|
||||
} from './history-command.js';
|
||||
import { buildHistorySessionActions, runHistoryPlaybackLoop } from './history-command.js';
|
||||
import type { HistorySeriesEntry } from '../history.js';
|
||||
|
||||
type HistoryLoop = (
|
||||
@@ -190,37 +186,6 @@ test('history show menu omits previous and next when the just-played episode has
|
||||
]);
|
||||
});
|
||||
|
||||
test('history entry menu offers previous, replay, and next before playback starts', () => {
|
||||
assert.equal(
|
||||
typeof buildHistoryEntryActions,
|
||||
'function',
|
||||
'history entry actions not implemented',
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
buildHistoryEntryActions(
|
||||
'/shows/test-show/episode-03.mkv',
|
||||
'/shows/test-show/episode-02.mkv',
|
||||
'/shows/test-show/episode-04.mkv',
|
||||
),
|
||||
[
|
||||
{ kind: 'previous', label: 'Previous episode: episode-02.mkv' },
|
||||
{ kind: 'replay', label: 'Replay last watched: episode-03.mkv' },
|
||||
{ kind: 'next', label: 'Next episode: episode-04.mkv' },
|
||||
{ kind: 'browse', label: 'Browse episodes' },
|
||||
{ kind: 'quit', label: 'Quit SubMiner' },
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test('history entry menu omits replay when the last watched file is gone', () => {
|
||||
assert.deepEqual(buildHistoryEntryActions(null, null, '/shows/test-show/episode-04.mkv'), [
|
||||
{ kind: 'next', label: 'Next episode: episode-04.mkv' },
|
||||
{ kind: 'browse', label: 'Browse episodes' },
|
||||
{ kind: 'quit', label: 'Quit SubMiner' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('history playback loop selects previous based on the just-played path, then re-derives previous from the new current episode', async () => {
|
||||
assert.equal(
|
||||
typeof runHistoryPlaybackLoop,
|
||||
|
||||
+1
-16
@@ -3,22 +3,7 @@ import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import os from 'node:os';
|
||||
import { findRofiTheme, formatRofiPrompt } from './picker';
|
||||
|
||||
// ── formatRofiPrompt: spacing between prompt and input field ──────────────────
|
||||
|
||||
test('formatRofiPrompt appends a single trailing space', () => {
|
||||
assert.equal(formatRofiPrompt('Select Video'), 'Select Video ');
|
||||
});
|
||||
|
||||
test('formatRofiPrompt collapses existing trailing whitespace to one space', () => {
|
||||
assert.equal(formatRofiPrompt('Watch History '), 'Watch History ');
|
||||
});
|
||||
|
||||
test('formatRofiPrompt leaves an empty prompt empty', () => {
|
||||
assert.equal(formatRofiPrompt(''), '');
|
||||
assert.equal(formatRofiPrompt(' '), '');
|
||||
});
|
||||
import { findRofiTheme } from './picker';
|
||||
|
||||
// ── findRofiTheme: Linux packaged path discovery ──────────────────────────────
|
||||
|
||||
|
||||
+4
-13
@@ -17,22 +17,13 @@ export function escapeShellSingle(value: string): string {
|
||||
return `'${value.replace(/'/g, `'\\''`)}'`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Rofi renders the prompt flush against the input field, so keep exactly one
|
||||
* trailing space to separate them.
|
||||
*/
|
||||
export function formatRofiPrompt(prompt: string): string {
|
||||
const trimmed = prompt.trimEnd();
|
||||
return trimmed ? `${trimmed} ` : '';
|
||||
}
|
||||
|
||||
export function showRofiFlatMenu(
|
||||
items: string[],
|
||||
prompt: string,
|
||||
initialQuery = '',
|
||||
themePath: string | null = null,
|
||||
): string {
|
||||
const args = ['-dmenu', '-i', '-matching', 'fuzzy', '-p', formatRofiPrompt(prompt)];
|
||||
const args = ['-dmenu', '-i', '-matching', 'fuzzy', '-p', prompt];
|
||||
if (themePath) {
|
||||
args.push('-theme', themePath);
|
||||
} else {
|
||||
@@ -119,7 +110,7 @@ export async function promptOptionalJellyfinSearch(
|
||||
themePath: string | null = null,
|
||||
): Promise<string> {
|
||||
if (useRofi && commandExists('rofi')) {
|
||||
const rofiArgs = ['-dmenu', '-i', '-p', formatRofiPrompt('Jellyfin Search (optional)')];
|
||||
const rofiArgs = ['-dmenu', '-i', '-p', 'Jellyfin Search (optional)'];
|
||||
if (themePath) {
|
||||
rofiArgs.push('-theme', themePath);
|
||||
} else {
|
||||
@@ -166,7 +157,7 @@ function showRofiIconMenu(
|
||||
themePath: string | null = null,
|
||||
): number {
|
||||
if (entries.length === 0) return -1;
|
||||
const rofiArgs = ['-dmenu', '-i', '-show-icons', '-format', 'i', '-p', formatRofiPrompt(prompt)];
|
||||
const rofiArgs = ['-dmenu', '-i', '-show-icons', '-format', 'i', '-p', prompt];
|
||||
if (initialQuery) rofiArgs.push('-filter', initialQuery);
|
||||
if (themePath) {
|
||||
rofiArgs.push('-theme', themePath);
|
||||
@@ -400,7 +391,7 @@ export function showRofiMenu(
|
||||
'-dmenu',
|
||||
'-i',
|
||||
'-p',
|
||||
formatRofiPrompt('Select Video'),
|
||||
'Select Video ',
|
||||
'-show-icons',
|
||||
'-theme-str',
|
||||
'configuration { font: "Noto Sans CJK JP Regular 8";}',
|
||||
|
||||
@@ -185,7 +185,6 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -194,7 +193,6 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -315,8 +313,8 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
CREATE INDEX idx_subtitle_lines_session_line ON imm_subtitle_lines(session_id, line_index);
|
||||
CREATE INDEX idx_subtitle_lines_video_line ON imm_subtitle_lines(video_id, line_index);
|
||||
CREATE INDEX idx_subtitle_lines_anime_line ON imm_subtitle_lines(anime_id, line_index);
|
||||
CREATE INDEX idx_word_line_occurrences_word_seen ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id);
|
||||
CREATE INDEX idx_kanji_line_occurrences_kanji_seen ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id);
|
||||
CREATE INDEX idx_word_line_occurrences_word ON imm_word_line_occurrences(word_id, line_id);
|
||||
CREATE INDEX idx_kanji_line_occurrences_kanji ON imm_kanji_line_occurrences(kanji_id, line_id);
|
||||
CREATE INDEX idx_media_art_cover_blob_hash ON imm_media_art(cover_blob_hash);
|
||||
CREATE INDEX idx_media_art_anilist_id ON imm_media_art(anilist_id);
|
||||
CREATE INDEX idx_media_art_cover_url ON imm_media_art(cover_url);
|
||||
|
||||
@@ -13,7 +13,6 @@
|
||||
"get-frequency:electron": "bun run build:yomitan && bun build scripts/get_frequency.ts --format=cjs --target=node --outfile dist/scripts/get_frequency.js --external electron && env -u ELECTRON_RUN_AS_NODE electron dist/scripts/get_frequency.js --pretty --color-top-x 10000 --yomitan-user-data ~/.config/SubMiner --colorized-line",
|
||||
"test-yomitan-parser": "bun run scripts/test-yomitan-parser.ts",
|
||||
"test-yomitan-parser:electron": "bun run build:yomitan && bun build scripts/test-yomitan-parser.ts --format=cjs --target=node --outfile dist/scripts/test-yomitan-parser.js --external electron && env -u ELECTRON_RUN_AS_NODE electron dist/scripts/test-yomitan-parser.js",
|
||||
"verify-known-word-highlights:electron": "bun run build:yomitan && bun build scripts/verify-known-word-highlights.ts --format=cjs --target=node --outfile dist/scripts/verify-known-word-highlights.js --packages=external && env -u ELECTRON_RUN_AS_NODE electron dist/scripts/verify-known-word-highlights.js",
|
||||
"record-tokenizer-fixture:electron": "bun run build:yomitan && bun build scripts/record-tokenizer-fixture.ts --format=cjs --target=node --outfile dist/scripts/record-tokenizer-fixture.js --external electron && env -u ELECTRON_RUN_AS_NODE electron dist/scripts/record-tokenizer-fixture.js",
|
||||
"compare-yomitan-api:electron": "bun run build:yomitan && bun build scripts/compare-yomitan-api.ts --format=cjs --target=node --outfile dist/scripts/compare-yomitan-api.js --external electron && env -u ELECTRON_RUN_AS_NODE electron dist/scripts/compare-yomitan-api.js",
|
||||
"build:yomitan": "bun scripts/build-yomitan.mjs",
|
||||
|
||||
@@ -1,593 +0,0 @@
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import process from 'node:process';
|
||||
|
||||
import {
|
||||
KnownWordCacheManager,
|
||||
getKnownWordCacheLifecycleConfig,
|
||||
} from '../src/anki-integration/known-word-cache.js';
|
||||
import { getMatureIntervalThresholdDays } from '../src/anki-integration/known-word-maturity.js';
|
||||
import { resolveConfigDir } from '../src/config/path-resolution.js';
|
||||
import { ConfigService } from '../src/config/service.js';
|
||||
import { parseSubtitleCues } from '../src/core/services/subtitle-cue-parser.js';
|
||||
import { createTokenizerDepsRuntime, tokenizeSubtitle } from '../src/core/services/tokenizer.js';
|
||||
import {
|
||||
resolveCompleteTokenReading,
|
||||
resolveKnownWordReadingForMatch,
|
||||
resolveKnownWordText,
|
||||
} from '../src/core/services/tokenizer/annotation-stage.js';
|
||||
import { MecabTokenizer } from '../src/mecab-tokenizer.js';
|
||||
import type { MergedToken } from '../src/types.js';
|
||||
import type { KnownWordMaturityTier } from '../src/types/subtitle.js';
|
||||
import {
|
||||
createYomitanRuntimeStateWithSearch,
|
||||
destroyParserWindow,
|
||||
loadElectronModule,
|
||||
withTimeout,
|
||||
type YomitanRuntimeState,
|
||||
} from './yomitan-script-runtime.js';
|
||||
|
||||
interface CliOptions {
|
||||
input: string;
|
||||
configDir?: string;
|
||||
yomitanUserDataPath?: string;
|
||||
yomitanExtensionPath?: string;
|
||||
limit: number;
|
||||
audit: boolean;
|
||||
refresh: boolean;
|
||||
json: boolean;
|
||||
quiet: boolean;
|
||||
profileCopy: boolean;
|
||||
}
|
||||
|
||||
type TierOrFallback = KnownWordMaturityTier | 'known-no-tier';
|
||||
|
||||
interface TokenReport {
|
||||
cueIndex: number;
|
||||
startTime: number;
|
||||
surface: string;
|
||||
headword: string;
|
||||
reading: string;
|
||||
tier: TierOrFallback;
|
||||
noteIds: number[];
|
||||
}
|
||||
|
||||
interface AuditMismatch extends TokenReport {
|
||||
liveTier: KnownWordMaturityTier | 'no-notes';
|
||||
intervals: number[];
|
||||
}
|
||||
|
||||
const TIERS: readonly KnownWordMaturityTier[] = ['new', 'learning', 'young', 'mature'];
|
||||
const FALLBACK_TIER_COLORS: Record<KnownWordMaturityTier, string> = {
|
||||
new: '#ee99a0',
|
||||
learning: '#b7bdf8',
|
||||
young: '#91d7e3',
|
||||
mature: '#a6da95',
|
||||
};
|
||||
|
||||
function parseCliArgs(argv: string[]): CliOptions {
|
||||
const options: CliOptions = {
|
||||
input: '',
|
||||
limit: 0,
|
||||
audit: false,
|
||||
refresh: false,
|
||||
json: false,
|
||||
quiet: false,
|
||||
profileCopy: false,
|
||||
};
|
||||
const rest: string[] = [];
|
||||
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
const arg = argv[i]!;
|
||||
const takeValue = (flag: string): string => {
|
||||
const next = argv[i + 1];
|
||||
if (!next) {
|
||||
throw new Error(`Missing value for ${flag}`);
|
||||
}
|
||||
i += 1;
|
||||
return next;
|
||||
};
|
||||
|
||||
if (arg === '--help' || arg === '-h') {
|
||||
process.stdout.write(`${usage()}\n`);
|
||||
process.exit(0);
|
||||
} else if (arg === '--input') {
|
||||
options.input = takeValue(arg);
|
||||
} else if (arg === '--config-dir') {
|
||||
options.configDir = takeValue(arg);
|
||||
} else if (arg === '--yomitan-user-data') {
|
||||
options.yomitanUserDataPath = takeValue(arg);
|
||||
} else if (arg === '--yomitan-extension-path') {
|
||||
options.yomitanExtensionPath = takeValue(arg);
|
||||
} else if (arg === '--limit') {
|
||||
options.limit = Math.max(0, Number.parseInt(takeValue(arg), 10) || 0);
|
||||
} else if (arg === '--audit') {
|
||||
options.audit = true;
|
||||
} else if (arg === '--refresh') {
|
||||
options.refresh = true;
|
||||
} else if (arg === '--json') {
|
||||
options.json = true;
|
||||
} else if (arg === '--quiet') {
|
||||
options.quiet = true;
|
||||
} else if (arg === '--profile-copy') {
|
||||
options.profileCopy = true;
|
||||
} else if (arg === '--') {
|
||||
// `bun run <script> -- --flag ...` forwards the separator too.
|
||||
continue;
|
||||
} else if (arg.startsWith('--')) {
|
||||
throw new Error(`Unknown flag: ${arg}`);
|
||||
} else {
|
||||
rest.push(arg);
|
||||
}
|
||||
}
|
||||
|
||||
if (!options.input && rest.length > 0) {
|
||||
options.input = rest.join(' ');
|
||||
}
|
||||
if (!options.input) {
|
||||
throw new Error(`No subtitle file given.\n${usage()}`);
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
function usage(): string {
|
||||
return [
|
||||
'Usage: verify-known-word-highlights <subtitle.srt|.ass> [flags]',
|
||||
'',
|
||||
' --limit <n> Only check the first n cues (default: all)',
|
||||
' --audit Re-derive every highlighted tier from live Anki card data',
|
||||
' --refresh Force a known-word cache refresh before checking',
|
||||
' --json Emit a machine-readable report',
|
||||
' --quiet Skip the per-line colored dump',
|
||||
' --profile-copy Copy the Yomitan profile to a scratch dir so this can run',
|
||||
' while SubMiner is open (Electron locks the userData dir)',
|
||||
' --config-dir <dir> SubMiner config dir (default: auto-detected)',
|
||||
' --yomitan-user-data <dir> Electron userData dir holding the Yomitan profile',
|
||||
' --yomitan-extension-path <dir>',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
const ANSI_RESET = '\u001b[0m';
|
||||
|
||||
function colorize(text: string, hex: string): string {
|
||||
const normalized = hex.trim().replace(/^#/, '');
|
||||
const expanded =
|
||||
normalized.length === 3
|
||||
? normalized
|
||||
.split('')
|
||||
.map((char) => `${char}${char}`)
|
||||
.join('')
|
||||
: normalized;
|
||||
if (!/^[0-9a-fA-F]{6}$/.test(expanded)) {
|
||||
return text;
|
||||
}
|
||||
const r = Number.parseInt(expanded.slice(0, 2), 16);
|
||||
const g = Number.parseInt(expanded.slice(2, 4), 16);
|
||||
const b = Number.parseInt(expanded.slice(4, 6), 16);
|
||||
return `\u001b[38;2;${r};${g};${b}m${text}${ANSI_RESET}`;
|
||||
}
|
||||
|
||||
function formatTimestamp(seconds: number): string {
|
||||
const total = Math.max(0, Math.floor(seconds));
|
||||
const mm = String(Math.floor(total / 60)).padStart(2, '0');
|
||||
const ss = String(total % 60).padStart(2, '0');
|
||||
return `${mm}:${ss}`;
|
||||
}
|
||||
|
||||
// The user's real config dir is copied into a scratch dir so this read-only
|
||||
// check can never rewrite config.jsonc (ConfigService migrates on load) or the
|
||||
// live known-word cache (a --refresh persists tier data).
|
||||
function createScratchState(configDir: string): { dir: string; cachePath: string } {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-highlight-verify-'));
|
||||
for (const fileName of ['config.jsonc', 'config.json']) {
|
||||
const source = path.join(configDir, fileName);
|
||||
if (fs.existsSync(source)) {
|
||||
fs.copyFileSync(source, path.join(dir, fileName));
|
||||
}
|
||||
}
|
||||
const cachePath = path.join(dir, 'known-words-cache.json');
|
||||
const liveCachePath = path.join(configDir, 'known-words-cache.json');
|
||||
if (fs.existsSync(liveCachePath)) {
|
||||
fs.copyFileSync(liveCachePath, cachePath);
|
||||
}
|
||||
return { dir, cachePath };
|
||||
}
|
||||
|
||||
// Electron locks a userData dir, so the Yomitan profile can't be shared with a
|
||||
// running SubMiner. Copying the dictionary-bearing parts of the profile lets
|
||||
// this check run mid-session (IndexedDB alone is often over 1 GB).
|
||||
const YOMITAN_PROFILE_DIRS = [
|
||||
'extensions',
|
||||
'IndexedDB',
|
||||
'Local Extension Settings',
|
||||
'Local Storage',
|
||||
];
|
||||
|
||||
function copyYomitanProfile(sourceUserDataPath: string): string {
|
||||
const target = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-highlight-profile-'));
|
||||
for (const name of YOMITAN_PROFILE_DIRS) {
|
||||
const source = path.join(sourceUserDataPath, name);
|
||||
if (fs.existsSync(source)) {
|
||||
fs.cpSync(source, path.join(target, name), { recursive: true });
|
||||
}
|
||||
}
|
||||
return target;
|
||||
}
|
||||
|
||||
function readPersistedCacheScope(cachePath: string): string | null {
|
||||
try {
|
||||
const parsed = JSON.parse(fs.readFileSync(cachePath, 'utf-8')) as { scope?: unknown };
|
||||
return typeof parsed.scope === 'string' ? parsed.scope : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const ANKI_REQUEST_TIMEOUT_MS = 30_000;
|
||||
|
||||
function createAnkiClient(url: string) {
|
||||
const request = async (action: string, params: unknown): Promise<unknown> => {
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ action, version: 6, params }),
|
||||
signal: AbortSignal.timeout(ANKI_REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`AnkiConnect ${action}: HTTP ${response.status} ${response.statusText}`);
|
||||
}
|
||||
const payload = (await response.json()) as { result: unknown; error: string | null };
|
||||
if (payload.error) {
|
||||
throw new Error(`AnkiConnect ${action}: ${payload.error}`);
|
||||
}
|
||||
return payload.result;
|
||||
};
|
||||
|
||||
return {
|
||||
request,
|
||||
findNotes: (query: string) => request('findNotes', { query }),
|
||||
notesInfo: (noteIds: number[]) => request('notesInfo', { notes: noteIds }),
|
||||
};
|
||||
}
|
||||
|
||||
function resolveTokenMatch(
|
||||
token: MergedToken,
|
||||
cache: KnownWordCacheManager,
|
||||
matchMode: 'surface' | 'headword',
|
||||
): { tier: KnownWordMaturityTier | null; noteIds: Set<number> } {
|
||||
const matchText = resolveKnownWordText(token.surface, token.headword, matchMode);
|
||||
const matchReading = resolveKnownWordReadingForMatch(token, matchMode);
|
||||
const primaryTier = matchText ? cache.getKnownWordTier(matchText, matchReading) : null;
|
||||
if (primaryTier) {
|
||||
return { tier: primaryTier, noteIds: cache.getKnownWordMatchNoteIds(matchText, matchReading) };
|
||||
}
|
||||
|
||||
const fallbackReading = resolveCompleteTokenReading(token);
|
||||
if (!fallbackReading || fallbackReading === matchText.trim()) {
|
||||
return {
|
||||
tier: null,
|
||||
noteIds: matchText ? cache.getKnownWordMatchNoteIds(matchText, matchReading) : new Set(),
|
||||
};
|
||||
}
|
||||
const fallbackOptions = { allowReadingOnlyMatch: false } as const;
|
||||
return {
|
||||
tier: cache.getKnownWordTier(fallbackReading, undefined, fallbackOptions),
|
||||
noteIds: cache.getKnownWordMatchNoteIds(fallbackReading, undefined, fallbackOptions),
|
||||
};
|
||||
}
|
||||
|
||||
// Ground truth straight from card data, independent of the Anki search filters
|
||||
// the cache refresh uses (prop:ivl / is:learn).
|
||||
function classifyCardsIntoTier(
|
||||
cards: Array<{ interval: number; queue: number; type: number }>,
|
||||
thresholdDays: number,
|
||||
): KnownWordMaturityTier {
|
||||
if (cards.some((card) => card.interval >= thresholdDays)) return 'mature';
|
||||
if (cards.some((card) => card.interval >= 1)) return 'young';
|
||||
if (
|
||||
cards.some((card) => card.type === 1 || card.type === 3 || card.queue === 1 || card.queue === 4)
|
||||
)
|
||||
return 'learning';
|
||||
return 'new';
|
||||
}
|
||||
|
||||
async function auditTokens(
|
||||
reports: TokenReport[],
|
||||
client: ReturnType<typeof createAnkiClient>,
|
||||
thresholdDays: number,
|
||||
): Promise<{ mismatches: AuditMismatch[]; auditedNotes: number }> {
|
||||
const noteIds = [...new Set(reports.flatMap((report) => report.noteIds))];
|
||||
const cardIdsByNote = new Map<number, number[]>();
|
||||
for (let i = 0; i < noteIds.length; i += 500) {
|
||||
const infos = (await client.notesInfo(noteIds.slice(i, i + 500))) as Array<{
|
||||
noteId: number;
|
||||
cards?: number[];
|
||||
}>;
|
||||
for (const info of infos) {
|
||||
cardIdsByNote.set(info.noteId, info.cards ?? []);
|
||||
}
|
||||
}
|
||||
|
||||
const allCardIds = [...cardIdsByNote.values()].flat();
|
||||
const cardById = new Map<number, { interval: number; queue: number; type: number }>();
|
||||
for (let i = 0; i < allCardIds.length; i += 500) {
|
||||
const infos = (await client.request('cardsInfo', {
|
||||
cards: allCardIds.slice(i, i + 500),
|
||||
})) as Array<{ cardId: number; interval: number; queue: number; type: number }>;
|
||||
for (const info of infos) {
|
||||
cardById.set(info.cardId, {
|
||||
interval: info.interval,
|
||||
queue: info.queue,
|
||||
type: info.type,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const mismatches: AuditMismatch[] = [];
|
||||
for (const report of reports) {
|
||||
const cards = report.noteIds
|
||||
.flatMap((noteId) => cardIdsByNote.get(noteId) ?? [])
|
||||
.map((cardId) => cardById.get(cardId))
|
||||
.filter((card): card is { interval: number; queue: number; type: number } => Boolean(card));
|
||||
const liveTier = cards.length === 0 ? 'no-notes' : classifyCardsIntoTier(cards, thresholdDays);
|
||||
if (liveTier !== report.tier) {
|
||||
mismatches.push({
|
||||
...report,
|
||||
liveTier,
|
||||
intervals: cards.map((card) => card.interval),
|
||||
});
|
||||
}
|
||||
}
|
||||
return { mismatches, auditedNotes: noteIds.length };
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const args = parseCliArgs(process.argv.slice(2));
|
||||
let electronModule: typeof import('electron') | null = null;
|
||||
let yomitanState: YomitanRuntimeState | null = null;
|
||||
let scratchDir: string | null = null;
|
||||
let profileCopyDir: string | null = null;
|
||||
|
||||
try {
|
||||
const configDir =
|
||||
args.configDir ??
|
||||
resolveConfigDir({
|
||||
homeDir: os.homedir(),
|
||||
xdgConfigHome: process.env.XDG_CONFIG_HOME,
|
||||
existsSync: fs.existsSync,
|
||||
});
|
||||
const scratch = createScratchState(configDir);
|
||||
scratchDir = scratch.dir;
|
||||
const config = new ConfigService(scratch.dir).getConfig();
|
||||
const ankiConfig = config.ankiConnect;
|
||||
const matchMode = ankiConfig.knownWords?.matchMode === 'surface' ? 'surface' : 'headword';
|
||||
const thresholdDays = getMatureIntervalThresholdDays(ankiConfig);
|
||||
const client = createAnkiClient(ankiConfig.url);
|
||||
const cacheScopeKey = getKnownWordCacheLifecycleConfig(ankiConfig);
|
||||
|
||||
const cache = new KnownWordCacheManager({
|
||||
client: { findNotes: (query) => client.findNotes(query), notesInfo: client.notesInfo },
|
||||
getConfig: () => ankiConfig,
|
||||
knownWordCacheStatePath: scratch.cachePath,
|
||||
showStatusNotification: () => {},
|
||||
});
|
||||
// A cache whose persisted scope key no longer matches the config is
|
||||
// discarded on load, so every token would come back unknown.
|
||||
if (!args.refresh && readPersistedCacheScope(scratch.cachePath) !== cacheScopeKey) {
|
||||
process.stderr.write(
|
||||
'warning: the persisted known-word cache was built under different settings and will be ' +
|
||||
'ignored (the app refetches it on its next refresh). Re-run with --refresh to fetch tiers now.\n',
|
||||
);
|
||||
}
|
||||
// startLifecycle loads the persisted cache; the refresh timer it arms is
|
||||
// cleared before the event loop can run it.
|
||||
cache.startLifecycle();
|
||||
cache.stopLifecycle();
|
||||
if (args.refresh) {
|
||||
await cache.refresh(true);
|
||||
}
|
||||
|
||||
const cues = parseSubtitleCues(fs.readFileSync(args.input, 'utf-8'), args.input);
|
||||
const selectedCues = args.limit > 0 ? cues.slice(0, args.limit) : cues;
|
||||
|
||||
const mecabTokenizer = new MecabTokenizer();
|
||||
if (!(await mecabTokenizer.checkAvailability())) {
|
||||
throw new Error('MeCab is not available; tokenization would not match the overlay.');
|
||||
}
|
||||
electronModule = await loadElectronModule();
|
||||
const userDataPath = args.profileCopy
|
||||
? copyYomitanProfile(args.yomitanUserDataPath ?? configDir)
|
||||
: (args.yomitanUserDataPath ?? configDir);
|
||||
profileCopyDir = args.profileCopy ? userDataPath : null;
|
||||
if (electronModule?.app && typeof electronModule.app.setPath === 'function') {
|
||||
electronModule.app.setPath('userData', userDataPath);
|
||||
}
|
||||
yomitanState = await createYomitanRuntimeStateWithSearch(
|
||||
userDataPath,
|
||||
args.yomitanExtensionPath,
|
||||
);
|
||||
if (!yomitanState.available) {
|
||||
throw new Error(`Yomitan tokenizer unavailable: ${yomitanState.note ?? 'unknown reason'}`);
|
||||
}
|
||||
|
||||
const deps = createTokenizerDepsRuntime({
|
||||
getYomitanExt: () => yomitanState!.yomitanExt as never,
|
||||
getYomitanSession: () => yomitanState!.yomitanSession as never,
|
||||
getYomitanParserWindow: () => yomitanState!.parserWindow as never,
|
||||
setYomitanParserWindow: (window) => {
|
||||
yomitanState!.parserWindow = window;
|
||||
},
|
||||
getYomitanParserReadyPromise: () => yomitanState!.parserReadyPromise as never,
|
||||
setYomitanParserReadyPromise: (promise) => {
|
||||
yomitanState!.parserReadyPromise = promise;
|
||||
},
|
||||
getYomitanParserInitPromise: () => yomitanState!.parserInitPromise as never,
|
||||
setYomitanParserInitPromise: (promise) => {
|
||||
yomitanState!.parserInitPromise = promise;
|
||||
},
|
||||
isKnownWord: (text, reading, options) => cache.isKnownWord(text, reading, options),
|
||||
getKnownWordTier: (text, reading, options) => cache.getKnownWordTier(text, reading, options),
|
||||
getKnownWordMatchMode: () => matchMode,
|
||||
getKnownWordsEnabled: () => true,
|
||||
// Other annotation layers are off so every colored token below is a
|
||||
// known-word decision, not an N+1/frequency/name override.
|
||||
getNPlusOneEnabled: () => false,
|
||||
getNameMatchEnabled: () => false,
|
||||
getJlptEnabled: () => false,
|
||||
getFrequencyDictionaryEnabled: () => false,
|
||||
getJlptLevel: () => null,
|
||||
getMecabTokenizer: () => ({ tokenize: (text: string) => mecabTokenizer.tokenize(text) }),
|
||||
});
|
||||
|
||||
const styleColors = {
|
||||
...FALLBACK_TIER_COLORS,
|
||||
...(config.subtitleStyle?.knownWordMaturityColors ?? {}),
|
||||
} as Record<KnownWordMaturityTier, string>;
|
||||
const knownWordColor = config.subtitleStyle?.knownWordColor ?? '#a6da95';
|
||||
|
||||
const reports: TokenReport[] = [];
|
||||
const tierCounts: Record<string, number> = {};
|
||||
let knownTokens = 0;
|
||||
let totalTokens = 0;
|
||||
const lines: string[] = [];
|
||||
|
||||
for (const [cueIndex, cue] of selectedCues.entries()) {
|
||||
const { text, tokens } = await withTimeout(
|
||||
tokenizeSubtitle(cue.text, deps),
|
||||
20_000,
|
||||
`Tokenizer (cue ${cueIndex + 1})`,
|
||||
);
|
||||
if (!tokens || tokens.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let cursor = 0;
|
||||
let rendered = '';
|
||||
const ordered = [...tokens].sort((a, b) => (a.startPos ?? 0) - (b.startPos ?? 0));
|
||||
for (const token of ordered) {
|
||||
totalTokens += 1;
|
||||
const start = Math.min(Math.max(0, token.startPos ?? 0), text.length);
|
||||
const end = Math.min(Math.max(start, token.endPos ?? start), text.length);
|
||||
if (start > cursor) {
|
||||
rendered += text.slice(cursor, start);
|
||||
}
|
||||
const surfaceText = text.slice(start, end);
|
||||
cursor = end;
|
||||
|
||||
if (!token.isKnown) {
|
||||
rendered += surfaceText;
|
||||
continue;
|
||||
}
|
||||
knownTokens += 1;
|
||||
const tier: TierOrFallback = token.knownMaturity ?? 'known-no-tier';
|
||||
tierCounts[tier] = (tierCounts[tier] ?? 0) + 1;
|
||||
rendered += colorize(
|
||||
surfaceText,
|
||||
tier === 'known-no-tier' ? knownWordColor : styleColors[tier],
|
||||
);
|
||||
reports.push({
|
||||
cueIndex,
|
||||
startTime: cue.startTime,
|
||||
surface: token.surface,
|
||||
headword: token.headword,
|
||||
reading: token.reading,
|
||||
tier,
|
||||
noteIds: [...resolveTokenMatch(token, cache, matchMode).noteIds],
|
||||
});
|
||||
}
|
||||
rendered += text.slice(cursor);
|
||||
lines.push(`${formatTimestamp(cue.startTime)} ${rendered}`);
|
||||
}
|
||||
|
||||
if (totalTokens === 0 && selectedCues.length > 0) {
|
||||
throw new Error(
|
||||
'Yomitan returned no tokens. SubMiner is probably running and holding the Electron ' +
|
||||
'profile lock - quit it, or re-run with --profile-copy.',
|
||||
);
|
||||
}
|
||||
|
||||
let audit: { mismatches: AuditMismatch[]; auditedNotes: number } | null = null;
|
||||
if (args.audit) {
|
||||
audit = await auditTokens(reports, client, thresholdDays);
|
||||
}
|
||||
|
||||
if (args.json) {
|
||||
process.stdout.write(
|
||||
`${JSON.stringify(
|
||||
{
|
||||
input: args.input,
|
||||
cues: selectedCues.length,
|
||||
totalTokens,
|
||||
knownTokens,
|
||||
tierCounts,
|
||||
matureThresholdDays: thresholdDays,
|
||||
matchMode,
|
||||
tokens: reports,
|
||||
audit,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!args.quiet) {
|
||||
process.stdout.write(`${lines.join('\n')}\n\n`);
|
||||
}
|
||||
process.stdout.write(
|
||||
[
|
||||
`file : ${args.input}`,
|
||||
`cues checked : ${selectedCues.length} of ${cues.length}`,
|
||||
`tokens : ${totalTokens} (${knownTokens} known, ${totalTokens - knownTokens} unknown)`,
|
||||
`match mode : ${matchMode} mature threshold: ${thresholdDays}d`,
|
||||
'',
|
||||
'known-token tiers:',
|
||||
...TIERS.map(
|
||||
(tier) =>
|
||||
` ${colorize(tier.padEnd(9), styleColors[tier])} ${String(tierCounts[tier] ?? 0).padStart(5)}` +
|
||||
` ${styleColors[tier]}`,
|
||||
),
|
||||
` ${colorize('no tier'.padEnd(9), knownWordColor)} ${String(tierCounts['known-no-tier'] ?? 0).padStart(5)} ${knownWordColor} (falls back to knownWordColor)`,
|
||||
'',
|
||||
].join('\n'),
|
||||
);
|
||||
|
||||
if (audit) {
|
||||
process.stdout.write(
|
||||
`audit: ${reports.length - audit.mismatches.length}/${reports.length} highlighted tokens agree with live Anki card data ` +
|
||||
`(${audit.auditedNotes} notes)\n`,
|
||||
);
|
||||
for (const mismatch of audit.mismatches.slice(0, 40)) {
|
||||
process.stdout.write(
|
||||
` ${formatTimestamp(mismatch.startTime)} ${mismatch.surface} (${mismatch.headword}) ` +
|
||||
`shown=${mismatch.tier} live=${mismatch.liveTier} ivl=[${mismatch.intervals.join(', ')}] ` +
|
||||
`notes=[${mismatch.noteIds.join(', ')}]\n`,
|
||||
);
|
||||
}
|
||||
if (audit.mismatches.length > 40) {
|
||||
process.stdout.write(` ... ${audit.mismatches.length - 40} more\n`);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
destroyParserWindow(yomitanState?.parserWindow ?? null);
|
||||
for (const dir of [scratchDir, profileCopyDir]) {
|
||||
if (dir) {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
if (electronModule?.app) {
|
||||
electronModule.app.quit();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main()
|
||||
.then(() => {
|
||||
process.exit(0);
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error(`Error: ${(error as Error).message}`);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,157 +0,0 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
import { resolveYomitanExtensionPath as resolveBuiltYomitanExtensionPath } from '../src/core/services/yomitan-extension-paths.js';
|
||||
|
||||
// Yomitan bootstrap for CLI scripts that need the app's real tokenizer. Mirrors
|
||||
// what scripts/get_frequency.ts does inline; new scripts should import this.
|
||||
export interface YomitanRuntimeState {
|
||||
yomitanExt: unknown | null;
|
||||
yomitanSession: unknown | null;
|
||||
parserWindow: unknown | null;
|
||||
parserReadyPromise: Promise<void> | null;
|
||||
parserInitPromise: Promise<boolean> | null;
|
||||
available: boolean;
|
||||
note?: string;
|
||||
}
|
||||
|
||||
export function withTimeout<T>(promise: Promise<T>, timeoutMs: number, label: string): Promise<T> {
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const timer = setTimeout(() => {
|
||||
reject(new Error(`${label} timed out after ${timeoutMs}ms`));
|
||||
}, timeoutMs);
|
||||
promise
|
||||
.then((value) => {
|
||||
clearTimeout(timer);
|
||||
resolve(value);
|
||||
})
|
||||
.catch((error) => {
|
||||
clearTimeout(timer);
|
||||
reject(error);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export function destroyParserWindow(window: unknown): void {
|
||||
if (!window || typeof window !== 'object') {
|
||||
return;
|
||||
}
|
||||
const candidate = window as { isDestroyed?: () => boolean; destroy?: () => void };
|
||||
if (typeof candidate.isDestroyed !== 'function' || typeof candidate.destroy !== 'function') {
|
||||
return;
|
||||
}
|
||||
if (!candidate.isDestroyed()) {
|
||||
candidate.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadElectronModule(): Promise<typeof import('electron') | null> {
|
||||
try {
|
||||
const electronImport = await import('electron');
|
||||
return (electronImport.default ?? electronImport) as typeof import('electron');
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function createYomitanRuntimeState(
|
||||
userDataPath: string,
|
||||
extensionPath?: string,
|
||||
): Promise<YomitanRuntimeState> {
|
||||
const state: YomitanRuntimeState = {
|
||||
yomitanExt: null,
|
||||
yomitanSession: null,
|
||||
parserWindow: null,
|
||||
parserReadyPromise: null,
|
||||
parserInitPromise: null,
|
||||
available: false,
|
||||
};
|
||||
|
||||
const electronImport = await loadElectronModule();
|
||||
if (
|
||||
!electronImport ||
|
||||
!electronImport.app ||
|
||||
typeof electronImport.app.whenReady !== 'function' ||
|
||||
!electronImport.session
|
||||
) {
|
||||
state.note = electronImport
|
||||
? 'electron runtime not available in this process'
|
||||
: 'electron import failed';
|
||||
return state;
|
||||
}
|
||||
|
||||
try {
|
||||
await electronImport.app.whenReady();
|
||||
const loadYomitanExtension = (await import('../src/core/services/yomitan-extension-loader.js'))
|
||||
.loadYomitanExtension as (options: {
|
||||
userDataPath: string;
|
||||
extensionPath?: string;
|
||||
getYomitanParserWindow: () => unknown;
|
||||
setYomitanParserWindow: (window: unknown) => void;
|
||||
setYomitanParserReadyPromise: (promise: Promise<void> | null) => void;
|
||||
setYomitanParserInitPromise: (promise: Promise<boolean> | null) => void;
|
||||
setYomitanExtension: (extension: unknown) => void;
|
||||
setYomitanSession: (session: unknown) => void;
|
||||
}) => Promise<unknown>;
|
||||
|
||||
const extension = await loadYomitanExtension({
|
||||
userDataPath,
|
||||
extensionPath,
|
||||
getYomitanParserWindow: () => state.parserWindow,
|
||||
setYomitanParserWindow: (window) => {
|
||||
state.parserWindow = window;
|
||||
},
|
||||
setYomitanParserReadyPromise: (promise) => {
|
||||
state.parserReadyPromise = promise;
|
||||
},
|
||||
setYomitanParserInitPromise: (promise) => {
|
||||
state.parserInitPromise = promise;
|
||||
},
|
||||
setYomitanExtension: (loaded) => {
|
||||
state.yomitanExt = loaded;
|
||||
},
|
||||
setYomitanSession: (nextSession) => {
|
||||
state.yomitanSession = nextSession;
|
||||
},
|
||||
});
|
||||
|
||||
if (!extension) {
|
||||
state.note = 'yomitan extension is not available';
|
||||
return state;
|
||||
}
|
||||
|
||||
state.yomitanExt = extension;
|
||||
state.available = true;
|
||||
return state;
|
||||
} catch (error) {
|
||||
state.note = error instanceof Error ? error.message : 'failed to initialize yomitan extension';
|
||||
return state;
|
||||
}
|
||||
}
|
||||
|
||||
export async function createYomitanRuntimeStateWithSearch(
|
||||
userDataPath: string,
|
||||
extensionPath?: string,
|
||||
): Promise<YomitanRuntimeState> {
|
||||
const resolvedExtensionPath = resolveBuiltYomitanExtensionPath({
|
||||
explicitPath: extensionPath,
|
||||
cwd: process.cwd(),
|
||||
});
|
||||
|
||||
if (resolvedExtensionPath) {
|
||||
try {
|
||||
if (fs.existsSync(path.join(resolvedExtensionPath, 'manifest.json'))) {
|
||||
const state = await createYomitanRuntimeState(userDataPath, resolvedExtensionPath);
|
||||
if (!state.available && !state.note) {
|
||||
state.note = `Failed to load yomitan extension at ${resolvedExtensionPath}`;
|
||||
}
|
||||
return state;
|
||||
}
|
||||
} catch {
|
||||
// fall through to the unconstrained loader below
|
||||
}
|
||||
}
|
||||
|
||||
// No usable manifest at the resolved path, so let the loader search on its own.
|
||||
return createYomitanRuntimeState(userDataPath);
|
||||
}
|
||||
@@ -1,76 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
import {
|
||||
KnownWordCacheState,
|
||||
knownWordsFromState,
|
||||
parseKnownWordCacheState,
|
||||
} from './known-word-cache-format';
|
||||
|
||||
function parseOrThrow(value: unknown): KnownWordCacheState {
|
||||
const parsed = parseKnownWordCacheState(value);
|
||||
assert.ok(parsed, 'expected the payload to parse');
|
||||
return parsed;
|
||||
}
|
||||
|
||||
const BASE = { refreshedAtMs: 1, scope: 'deck:test' };
|
||||
|
||||
test('known-word cache format reads words from every version the union covers', () => {
|
||||
assert.deepEqual(
|
||||
knownWordsFromState(parseOrThrow({ ...BASE, version: 1, words: ['する'] })),
|
||||
new Set(['する']),
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
knownWordsFromState(
|
||||
parseOrThrow({ ...BASE, version: 2, words: ['する'], notes: { '1': ['する'] } }),
|
||||
),
|
||||
new Set(['する']),
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
knownWordsFromState(
|
||||
parseOrThrow({
|
||||
...BASE,
|
||||
version: 3,
|
||||
notes: { '1': [{ word: 'する', reading: 'する' }], '2': [{ word: '猫', reading: null }] },
|
||||
}),
|
||||
),
|
||||
new Set(['する', '猫']),
|
||||
);
|
||||
|
||||
// v4 only adds `tiers`; the words a reader sees must not change.
|
||||
assert.deepEqual(
|
||||
knownWordsFromState(
|
||||
parseOrThrow({
|
||||
...BASE,
|
||||
version: 4,
|
||||
notes: { '1': [{ word: 'する', reading: 'する' }], '2': [{ word: '猫', reading: null }] },
|
||||
tiers: { '1': 'mature', '2': 'young' },
|
||||
}),
|
||||
),
|
||||
new Set(['する', '猫']),
|
||||
);
|
||||
});
|
||||
|
||||
test('known-word cache format rejects payloads that are not a known cache state', () => {
|
||||
const notes = { '1': [{ word: 'する', reading: null }] };
|
||||
|
||||
assert.equal(parseKnownWordCacheState(null), null);
|
||||
assert.equal(parseKnownWordCacheState('{}'), null);
|
||||
|
||||
// An unknown version must not be read as an empty cache.
|
||||
assert.equal(parseKnownWordCacheState({ ...BASE, version: 99, notes }), null);
|
||||
|
||||
assert.equal(
|
||||
parseKnownWordCacheState({ version: 4, scope: 'deck:test', notes, tiers: {} }),
|
||||
null,
|
||||
);
|
||||
assert.equal(parseKnownWordCacheState({ version: 4, refreshedAtMs: 1, notes, tiers: {} }), null);
|
||||
|
||||
// v4 without its tiers map is a v3 payload mislabelled as v4.
|
||||
assert.equal(parseKnownWordCacheState({ ...BASE, version: 4, notes }), null);
|
||||
|
||||
// v3 carries entry objects, not the bare strings v2 used.
|
||||
assert.equal(parseKnownWordCacheState({ ...BASE, version: 3, notes: { '1': ['する'] } }), null);
|
||||
});
|
||||
@@ -1,141 +0,0 @@
|
||||
// On-disk shape of the known-word cache, plus the only parser for it.
|
||||
//
|
||||
// Two processes read this file: the cache manager (which rebuilds its indexes
|
||||
// from it) and the stats server (which counts known words). They used to carry
|
||||
// separate hand-written parsers, so bumping the format to v4 for maturity tiers
|
||||
// left the stats server silently reporting zero known words. Everything that
|
||||
// touches the format now goes through here, and the version dispatch below ends
|
||||
// in assertNever so adding a V5 to the union fails the build at every consumer
|
||||
// instead of degrading to an empty result at runtime.
|
||||
|
||||
import type { KnownWordMaturityTier } from '../types/subtitle';
|
||||
import type { KnownWordEntry } from './known-word-entries';
|
||||
|
||||
export interface KnownWordCacheStateV1 {
|
||||
readonly version: 1;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly words: string[];
|
||||
}
|
||||
|
||||
export interface KnownWordCacheStateV2 {
|
||||
readonly version: 2;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly words: string[];
|
||||
readonly notes: Record<string, string[]>;
|
||||
}
|
||||
|
||||
export interface KnownWordCacheStateV3 {
|
||||
readonly version: 3;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly notes: Record<string, KnownWordEntry[]>;
|
||||
}
|
||||
|
||||
export interface KnownWordCacheStateV4 {
|
||||
readonly version: 4;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly notes: Record<string, KnownWordEntry[]>;
|
||||
readonly tiers: Record<string, KnownWordMaturityTier>;
|
||||
}
|
||||
|
||||
export type KnownWordCacheState =
|
||||
| KnownWordCacheStateV1
|
||||
| KnownWordCacheStateV2
|
||||
| KnownWordCacheStateV3
|
||||
| KnownWordCacheStateV4;
|
||||
|
||||
// Version written by persistKnownWordCacheState. Readers accept every version
|
||||
// in the union above; only the writer pins one.
|
||||
export type CurrentKnownWordCacheState = KnownWordCacheStateV4;
|
||||
|
||||
// Exported so every consumer that switches on `version` can close its dispatch
|
||||
// the same way: a new member of the union becomes a type error at each call
|
||||
// site rather than a case that silently falls through.
|
||||
export function assertNever(value: never): never {
|
||||
throw new Error(`Unhandled known-word cache state: ${JSON.stringify(value)}`);
|
||||
}
|
||||
|
||||
function isEntryRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function isKnownWordEntry(entry: unknown): boolean {
|
||||
if (!isEntryRecord(entry)) return false;
|
||||
const candidate = entry as Partial<KnownWordEntry>;
|
||||
return (
|
||||
typeof candidate.word === 'string' &&
|
||||
(candidate.reading === null || typeof candidate.reading === 'string')
|
||||
);
|
||||
}
|
||||
|
||||
// Returns the narrowed state, or null when the payload is not a cache state we
|
||||
// recognize. Per-entry values that are merely unusable (an unknown maturity
|
||||
// tier, a non-numeric note id) are dropped by callers at load time rather than
|
||||
// rejecting the whole file.
|
||||
export function parseKnownWordCacheState(value: unknown): KnownWordCacheState | null {
|
||||
if (!isEntryRecord(value)) return null;
|
||||
const candidate = value;
|
||||
if (
|
||||
candidate.version !== 1 &&
|
||||
candidate.version !== 2 &&
|
||||
candidate.version !== 3 &&
|
||||
candidate.version !== 4
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
if (typeof candidate.refreshedAtMs !== 'number') return null;
|
||||
if (typeof candidate.scope !== 'string') return null;
|
||||
|
||||
if (candidate.version === 1 || candidate.version === 2) {
|
||||
if (!Array.isArray(candidate.words)) return null;
|
||||
if (!candidate.words.every((entry: unknown) => typeof entry === 'string')) return null;
|
||||
}
|
||||
|
||||
if (candidate.version === 4) {
|
||||
// Per-tier values are sanitized entry-by-entry at load time.
|
||||
if (!isEntryRecord(candidate.tiers)) return null;
|
||||
}
|
||||
|
||||
if (candidate.version === 2 || candidate.version === 3 || candidate.version === 4) {
|
||||
if (!isEntryRecord(candidate.notes)) return null;
|
||||
const isValidNoteEntry =
|
||||
candidate.version === 2
|
||||
? (entry: unknown): boolean => typeof entry === 'string'
|
||||
: isKnownWordEntry;
|
||||
if (
|
||||
!Object.values(candidate.notes).every(
|
||||
(noteEntries) => Array.isArray(noteEntries) && noteEntries.every(isValidNoteEntry),
|
||||
)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
return candidate as unknown as KnownWordCacheState;
|
||||
}
|
||||
|
||||
// Every word the cache considers known, flattened across notes. Consumers that
|
||||
// only need membership (the stats server) use this instead of walking the
|
||||
// version-specific layout themselves.
|
||||
export function knownWordsFromState(state: KnownWordCacheState): Set<string> {
|
||||
switch (state.version) {
|
||||
case 1:
|
||||
case 2:
|
||||
return new Set(state.words);
|
||||
case 3:
|
||||
case 4: {
|
||||
const words = new Set<string>();
|
||||
for (const entries of Object.values(state.notes)) {
|
||||
for (const entry of entries) {
|
||||
if (entry.word) words.add(entry.word);
|
||||
}
|
||||
}
|
||||
return words;
|
||||
}
|
||||
default:
|
||||
return assertNever(state);
|
||||
}
|
||||
}
|
||||
@@ -69,18 +69,6 @@ function createMaturityHarness(config: AnkiConnectConfig): {
|
||||
};
|
||||
}
|
||||
|
||||
// The four queries a maturity refresh issues, in one place so a query-string
|
||||
// change lands in a single spot.
|
||||
function setTierQueries(
|
||||
clientState: { findNotesByQuery: Map<string, number[]> },
|
||||
tiers: { all: number[]; mature: number[]; young: number[]; learning: number[] },
|
||||
): void {
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', tiers.all);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21 -is:learn', tiers.mature);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn', tiers.young);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', tiers.learning);
|
||||
}
|
||||
|
||||
function maturityConfig(overrides: Partial<AnkiConnectConfig> = {}): AnkiConnectConfig {
|
||||
return {
|
||||
deck: 'Mining',
|
||||
@@ -110,7 +98,7 @@ test('lifecycle config key is unchanged when maturity is disabled', () => {
|
||||
};
|
||||
assert.equal(
|
||||
getKnownWordCacheLifecycleConfig(enabled),
|
||||
'{"refreshMinutes":60,"scope":"all","fieldsWord":"","maturity":21,"maturityRules":2}',
|
||||
'{"refreshMinutes":60,"scope":"all","fieldsWord":"","maturity":21}',
|
||||
);
|
||||
|
||||
const customThreshold: AnkiConnectConfig = {
|
||||
@@ -123,19 +111,7 @@ test('lifecycle config key is unchanged when maturity is disabled', () => {
|
||||
};
|
||||
assert.equal(
|
||||
getKnownWordCacheLifecycleConfig(customThreshold),
|
||||
'{"refreshMinutes":60,"scope":"all","fieldsWord":"","maturity":30,"maturityRules":2}',
|
||||
);
|
||||
});
|
||||
|
||||
test('a cache built under the old tier rules is invalidated', () => {
|
||||
const config: AnkiConnectConfig = {
|
||||
knownWords: { highlightEnabled: true, maturityEnabled: true, refreshMinutes: 60 },
|
||||
};
|
||||
// v1 rules put lapsed cards in young because the interval queries did not
|
||||
// exclude is:learn; those persisted tiers must not be served under v2.
|
||||
assert.notEqual(
|
||||
getKnownWordCacheLifecycleConfig(config),
|
||||
'{"refreshMinutes":60,"scope":"all","fieldsWord":"","maturity":21}',
|
||||
'{"refreshMinutes":60,"scope":"all","fieldsWord":"","maturity":30}',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -143,7 +119,10 @@ test('refresh fetches tier sets and getKnownWordTier classifies notes', async ()
|
||||
const { manager, calls, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2, 3, 4], mature: [1], young: [2], learning: [3] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2, 3, 4]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', [2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [3]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '犬' } } },
|
||||
@@ -171,7 +150,10 @@ test('a note with cards in several tiers counts as its most mature card', async
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1], mature: [1], young: [1], learning: [1] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [1]);
|
||||
clientState.notesInfoResult = [{ noteId: 1, fields: { Word: { value: '猫' } } }];
|
||||
|
||||
await manager.refresh(true);
|
||||
@@ -186,7 +168,10 @@ test('a word matched by several notes takes the most mature note tier', async ()
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [], young: [2], learning: [1] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', [2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [1]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '猫' } } },
|
||||
@@ -204,7 +189,10 @@ test('tiers are reading-aware for words with several readings', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [1], young: [], learning: [2] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [2]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '床' }, Reading: { value: 'とこ' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '床' }, Reading: { value: 'ゆか' } } },
|
||||
@@ -227,7 +215,10 @@ test('reading-only fallback resolves tiers unless opted out', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1], mature: [1], young: [], learning: [] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', []);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '警告' }, Reading: { value: 'けいこく' } } },
|
||||
];
|
||||
@@ -274,7 +265,7 @@ test('refresh preserves known-word cache when maturity lookup fails', async () =
|
||||
infoLogs.push(args.map((value) => String(value)).join(' '));
|
||||
};
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1]);
|
||||
clientState.failedQueries.add('deck:"Mining" prop:ivl>=21 -is:learn');
|
||||
clientState.failedQueries.add('deck:"Mining" prop:ivl>=21');
|
||||
clientState.notesInfoResult = [{ noteId: 1, fields: { Word: { value: '猫' } } }];
|
||||
|
||||
await manager.refresh(true);
|
||||
@@ -302,7 +293,10 @@ test('tiers persist to v4 state and reload without refetching', async () => {
|
||||
|
||||
try {
|
||||
Date.now = () => 120_000;
|
||||
setTierQueries(clientState, { all: [1, 2], mature: [1], young: [], learning: [2] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [1, 2]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [1]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [2]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '犬' } } },
|
||||
@@ -353,7 +347,10 @@ test('appendFromNoteInfo preserves an existing maturity tier', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [7, 8], mature: [7], young: [], learning: [8] });
|
||||
clientState.findNotesByQuery.set('deck:"Mining"', [7, 8]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=21', [7]);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" prop:ivl>=1 prop:ivl<21', []);
|
||||
clientState.findNotesByQuery.set('deck:"Mining" is:learn', [8]);
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 7, fields: { Word: { value: '猫' } } },
|
||||
{ noteId: 8, fields: { Word: { value: '犬' } } },
|
||||
@@ -375,37 +372,3 @@ test('appendFromNoteInfo preserves an existing maturity tier', async () => {
|
||||
cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('getKnownWordMatchNoteIds reports the notes behind a tier', async () => {
|
||||
const { manager, clientState, cleanup } = createMaturityHarness(maturityConfig());
|
||||
|
||||
try {
|
||||
setTierQueries(clientState, { all: [1, 2, 3], mature: [1], young: [2], learning: [3] });
|
||||
clientState.notesInfoResult = [
|
||||
{ noteId: 1, fields: { Word: { value: '床' }, Reading: { value: 'とこ' } } },
|
||||
{ noteId: 2, fields: { Word: { value: '床' }, Reading: { value: 'ゆか' } } },
|
||||
{ noteId: 3, fields: { Word: { value: '警告' }, Reading: { value: 'けいこく' } } },
|
||||
];
|
||||
|
||||
await manager.refresh(true);
|
||||
|
||||
// Same matching rules as getKnownWordTier, so an audit can re-derive the
|
||||
// rendered tier from the exact notes that produced it.
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('床', 'とこ')], [1]);
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('床', 'ゆか')], [2]);
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('床')].sort(), [1, 2]);
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('床', 'しょう')], []);
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('けいこく')], [3]);
|
||||
assert.deepEqual(
|
||||
[
|
||||
...manager.getKnownWordMatchNoteIds('けいこく', undefined, {
|
||||
allowReadingOnlyMatch: false,
|
||||
}),
|
||||
],
|
||||
[],
|
||||
);
|
||||
assert.deepEqual([...manager.getKnownWordMatchNoteIds('馬')], []);
|
||||
} finally {
|
||||
cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -7,7 +7,6 @@ import { AnkiConnectConfig } from '../types/anki';
|
||||
import type { KnownWordMaturityTier } from '../types/subtitle';
|
||||
import { createLogger } from '../logger';
|
||||
import {
|
||||
KNOWN_WORD_MATURITY_RULES_VERSION,
|
||||
classifyKnownWordNoteTier,
|
||||
fetchKnownWordMaturityTierSets,
|
||||
getKnownWordMaturityEnabled,
|
||||
@@ -15,11 +14,6 @@ import {
|
||||
maxKnownWordMaturityTier,
|
||||
sanitizeKnownWordMaturityTier,
|
||||
} from './known-word-maturity';
|
||||
import {
|
||||
CurrentKnownWordCacheState,
|
||||
assertNever,
|
||||
parseKnownWordCacheState,
|
||||
} from './known-word-cache-format';
|
||||
import {
|
||||
DEFAULT_KNOWN_WORD_READING_FIELDS,
|
||||
KnownWordEntry,
|
||||
@@ -82,14 +76,10 @@ export function getKnownWordCacheLifecycleConfig(config: AnkiConnectConfig): str
|
||||
scope: getKnownWordCacheScopeForConfig(config),
|
||||
fieldsWord: trimToNonEmptyString(config.fields?.word) ?? '',
|
||||
};
|
||||
// The maturity fields are only added while enabled so persisted caches from
|
||||
// The maturity field is only added while enabled so persisted caches from
|
||||
// before the feature existed (or with it off) keep their identity.
|
||||
// maturityRules is the classification-rule version: bump it whenever the tier
|
||||
// queries change meaning so existing caches refetch instead of serving tiers
|
||||
// computed under the old rules.
|
||||
if (getKnownWordMaturityEnabled(config)) {
|
||||
payload.maturity = getMatureIntervalThresholdDays(config);
|
||||
payload.maturityRules = KNOWN_WORD_MATURITY_RULES_VERSION;
|
||||
}
|
||||
return JSON.stringify(payload);
|
||||
}
|
||||
@@ -99,6 +89,42 @@ export interface KnownWordCacheNoteInfo {
|
||||
fields: Record<string, { value: string }>;
|
||||
}
|
||||
|
||||
interface KnownWordCacheStateV1 {
|
||||
readonly version: 1;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly words: string[];
|
||||
}
|
||||
|
||||
interface KnownWordCacheStateV2 {
|
||||
readonly version: 2;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly words: string[];
|
||||
readonly notes: Record<string, string[]>;
|
||||
}
|
||||
|
||||
interface KnownWordCacheStateV3 {
|
||||
readonly version: 3;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly notes: Record<string, KnownWordEntry[]>;
|
||||
}
|
||||
|
||||
interface KnownWordCacheStateV4 {
|
||||
readonly version: 4;
|
||||
readonly refreshedAtMs: number;
|
||||
readonly scope: string;
|
||||
readonly notes: Record<string, KnownWordEntry[]>;
|
||||
readonly tiers: Record<string, KnownWordMaturityTier>;
|
||||
}
|
||||
|
||||
type KnownWordCacheState =
|
||||
| KnownWordCacheStateV1
|
||||
| KnownWordCacheStateV2
|
||||
| KnownWordCacheStateV3
|
||||
| KnownWordCacheStateV4;
|
||||
|
||||
const NO_READING_KEY = '';
|
||||
|
||||
interface KnownWordCacheClient {
|
||||
@@ -200,56 +226,43 @@ export class KnownWordCacheManager {
|
||||
return null;
|
||||
}
|
||||
|
||||
return this.maxTierForNotes(null, this.getKnownWordMatchNoteIds(text, reading, options));
|
||||
}
|
||||
|
||||
// Note ids a known-word lookup matches, using the same matching rules as
|
||||
// getKnownWordTier. Exposed for diagnostics (see
|
||||
// scripts/verify-known-word-highlights.ts), which audits a rendered tier
|
||||
// against the live card data of the notes that produced it.
|
||||
getKnownWordMatchNoteIds(
|
||||
text: string,
|
||||
reading?: string,
|
||||
options?: { allowReadingOnlyMatch?: boolean },
|
||||
): Set<number> {
|
||||
const matches = new Set<number>();
|
||||
const normalized = this.normalizeKnownWordForLookup(text);
|
||||
if (normalized.length === 0) {
|
||||
return matches;
|
||||
return null;
|
||||
}
|
||||
|
||||
const knownReadings = this.wordReadingNoteIds.get(normalized);
|
||||
if (knownReadings && knownReadings.size > 0) {
|
||||
const normalizedReading =
|
||||
typeof reading === 'string' ? normalizeKnownReadingForLookup(reading) : '';
|
||||
let tier: KnownWordMaturityTier | null = null;
|
||||
if (normalizedReading.length === 0) {
|
||||
for (const noteIds of knownReadings.values()) {
|
||||
for (const noteId of noteIds) {
|
||||
matches.add(noteId);
|
||||
}
|
||||
tier = this.maxTierForNotes(tier, noteIds);
|
||||
}
|
||||
return matches;
|
||||
return tier;
|
||||
}
|
||||
for (const key of [NO_READING_KEY, normalizedReading]) {
|
||||
for (const noteId of knownReadings.get(key) ?? []) {
|
||||
matches.add(noteId);
|
||||
}
|
||||
const noReadingNotes = knownReadings.get(NO_READING_KEY);
|
||||
if (noReadingNotes) {
|
||||
tier = this.maxTierForNotes(tier, noReadingNotes);
|
||||
}
|
||||
return matches;
|
||||
const exactReadingNotes = knownReadings.get(normalizedReading);
|
||||
if (exactReadingNotes) {
|
||||
tier = this.maxTierForNotes(tier, exactReadingNotes);
|
||||
}
|
||||
return tier;
|
||||
}
|
||||
|
||||
if (options?.allowReadingOnlyMatch === false) {
|
||||
return matches;
|
||||
return null;
|
||||
}
|
||||
|
||||
const hiragana = convertKatakanaToHiragana(normalized);
|
||||
if ([...hiragana].length === 1) {
|
||||
return matches;
|
||||
return null;
|
||||
}
|
||||
for (const noteId of this.readingNoteIds.get(hiragana) ?? []) {
|
||||
matches.add(noteId);
|
||||
}
|
||||
return matches;
|
||||
const readingNotes = this.readingNoteIds.get(hiragana);
|
||||
return readingNotes ? this.maxTierForNotes(null, readingNotes) : null;
|
||||
}
|
||||
|
||||
private maxTierForNotes(
|
||||
@@ -811,8 +824,8 @@ export class KnownWordCacheManager {
|
||||
return;
|
||||
}
|
||||
|
||||
const parsed = parseKnownWordCacheState(JSON.parse(raw) as unknown);
|
||||
if (!parsed) {
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (!this.isKnownWordCacheStateValid(parsed)) {
|
||||
this.clearInMemoryState();
|
||||
this.knownWordsStateKey = this.getKnownWordCacheStateKey();
|
||||
return;
|
||||
@@ -825,63 +838,57 @@ export class KnownWordCacheManager {
|
||||
}
|
||||
|
||||
this.clearInMemoryState();
|
||||
switch (parsed.version) {
|
||||
case 1:
|
||||
// v1 has no per-note snapshots to convert; refetch from Anki.
|
||||
this.knownWordsStateKey = this.getKnownWordCacheStateKey();
|
||||
return;
|
||||
case 2:
|
||||
// Older states have no readings; load them reading-less (fail-open,
|
||||
// matching the old behavior) but leave the cache marked stale so the
|
||||
// next refresh upgrades entries with readings from Anki.
|
||||
for (const [noteIdKey, words] of Object.entries(parsed.notes)) {
|
||||
if (parsed.version === 3 || parsed.version === 4) {
|
||||
for (const [noteIdKey, entries] of Object.entries(parsed.notes)) {
|
||||
const noteId = Number.parseInt(noteIdKey, 10);
|
||||
if (!Number.isInteger(noteId) || noteId <= 0) {
|
||||
continue;
|
||||
}
|
||||
const normalizedEntries = normalizeKnownWordEntryList(entries);
|
||||
if (normalizedEntries.length === 0) {
|
||||
continue;
|
||||
}
|
||||
this.noteEntriesById.set(noteId, normalizedEntries);
|
||||
this.addEntriesToIndexes(noteId, normalizedEntries);
|
||||
}
|
||||
if (parsed.version === 4) {
|
||||
for (const [noteIdKey, tier] of Object.entries(parsed.tiers)) {
|
||||
const noteId = Number.parseInt(noteIdKey, 10);
|
||||
if (!Number.isInteger(noteId) || noteId <= 0) {
|
||||
continue;
|
||||
}
|
||||
const normalizedEntries = normalizeKnownWordEntryList(
|
||||
words.map((word) => ({
|
||||
word: this.normalizeKnownWordForLookup(word),
|
||||
reading: null,
|
||||
})),
|
||||
);
|
||||
if (normalizedEntries.length === 0) {
|
||||
continue;
|
||||
}
|
||||
this.noteEntriesById.set(noteId, normalizedEntries);
|
||||
this.addEntriesToIndexes(noteId, normalizedEntries);
|
||||
}
|
||||
this.knownWordsStateKey = parsed.scope;
|
||||
return;
|
||||
case 3:
|
||||
case 4:
|
||||
for (const [noteIdKey, entries] of Object.entries(parsed.notes)) {
|
||||
const noteId = Number.parseInt(noteIdKey, 10);
|
||||
if (!Number.isInteger(noteId) || noteId <= 0) {
|
||||
continue;
|
||||
}
|
||||
const normalizedEntries = normalizeKnownWordEntryList(entries);
|
||||
if (normalizedEntries.length === 0) {
|
||||
continue;
|
||||
}
|
||||
this.noteEntriesById.set(noteId, normalizedEntries);
|
||||
this.addEntriesToIndexes(noteId, normalizedEntries);
|
||||
}
|
||||
if (parsed.version === 4) {
|
||||
for (const [noteIdKey, tier] of Object.entries(parsed.tiers)) {
|
||||
const noteId = Number.parseInt(noteIdKey, 10);
|
||||
const sanitizedTier = sanitizeKnownWordMaturityTier(tier);
|
||||
if (sanitizedTier && this.noteEntriesById.has(noteId)) {
|
||||
this.noteTierById.set(noteId, sanitizedTier);
|
||||
}
|
||||
const sanitizedTier = sanitizeKnownWordMaturityTier(tier);
|
||||
if (sanitizedTier && this.noteEntriesById.has(noteId)) {
|
||||
this.noteTierById.set(noteId, sanitizedTier);
|
||||
}
|
||||
}
|
||||
this.knownWordsLastRefreshedAtMs = parsed.refreshedAtMs;
|
||||
this.knownWordsStateKey = parsed.scope;
|
||||
return;
|
||||
default:
|
||||
assertNever(parsed);
|
||||
}
|
||||
this.knownWordsLastRefreshedAtMs = parsed.refreshedAtMs;
|
||||
this.knownWordsStateKey = parsed.scope;
|
||||
return;
|
||||
}
|
||||
|
||||
if (parsed.version === 2) {
|
||||
// Older states have no readings; load them reading-less (fail-open,
|
||||
// matching the old behavior) but leave the cache marked stale so the
|
||||
// next refresh upgrades entries with readings from Anki.
|
||||
for (const [noteIdKey, words] of Object.entries(parsed.notes)) {
|
||||
const noteId = Number.parseInt(noteIdKey, 10);
|
||||
if (!Number.isInteger(noteId) || noteId <= 0) {
|
||||
continue;
|
||||
}
|
||||
const normalizedEntries = normalizeKnownWordEntryList(
|
||||
words.map((word) => ({ word: this.normalizeKnownWordForLookup(word), reading: null })),
|
||||
);
|
||||
if (normalizedEntries.length === 0) {
|
||||
continue;
|
||||
}
|
||||
this.noteEntriesById.set(noteId, normalizedEntries);
|
||||
this.addEntriesToIndexes(noteId, normalizedEntries);
|
||||
}
|
||||
this.knownWordsStateKey = parsed.scope;
|
||||
return;
|
||||
}
|
||||
|
||||
// v1 has no per-note snapshots to convert; refetch from Anki.
|
||||
this.knownWordsStateKey = this.getKnownWordCacheStateKey();
|
||||
} catch (error) {
|
||||
log.warn('Failed to load known-word cache state:', (error as Error).message);
|
||||
this.clearInMemoryState();
|
||||
@@ -903,7 +910,7 @@ export class KnownWordCacheManager {
|
||||
}
|
||||
}
|
||||
|
||||
const state: CurrentKnownWordCacheState = {
|
||||
const state: KnownWordCacheStateV4 = {
|
||||
version: 4,
|
||||
refreshedAtMs: this.knownWordsLastRefreshedAtMs,
|
||||
scope: this.knownWordsStateKey,
|
||||
@@ -916,6 +923,63 @@ export class KnownWordCacheManager {
|
||||
}
|
||||
}
|
||||
|
||||
private isKnownWordCacheStateValid(value: unknown): value is KnownWordCacheState {
|
||||
if (typeof value !== 'object' || value === null) return false;
|
||||
const candidate = value as Record<string, unknown>;
|
||||
if (
|
||||
candidate.version !== 1 &&
|
||||
candidate.version !== 2 &&
|
||||
candidate.version !== 3 &&
|
||||
candidate.version !== 4
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (typeof candidate.refreshedAtMs !== 'number') return false;
|
||||
if (typeof candidate.scope !== 'string') return false;
|
||||
if (candidate.version === 1 || candidate.version === 2) {
|
||||
if (!Array.isArray(candidate.words)) return false;
|
||||
if (!candidate.words.every((entry: unknown) => typeof entry === 'string')) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (candidate.version === 4) {
|
||||
// Per-tier values are sanitized entry-by-entry at load time.
|
||||
if (
|
||||
typeof candidate.tiers !== 'object' ||
|
||||
candidate.tiers === null ||
|
||||
Array.isArray(candidate.tiers)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (candidate.version === 2 || candidate.version === 3 || candidate.version === 4) {
|
||||
if (
|
||||
typeof candidate.notes !== 'object' ||
|
||||
candidate.notes === null ||
|
||||
Array.isArray(candidate.notes)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
const isValidNoteEntry =
|
||||
candidate.version === 2
|
||||
? (entry: unknown): boolean => typeof entry === 'string'
|
||||
: (entry: unknown): boolean =>
|
||||
typeof entry === 'object' &&
|
||||
entry !== null &&
|
||||
typeof (entry as KnownWordEntry).word === 'string' &&
|
||||
((entry as KnownWordEntry).reading === null ||
|
||||
typeof (entry as KnownWordEntry).reading === 'string');
|
||||
if (
|
||||
!Object.values(candidate.notes as Record<string, unknown>).every(
|
||||
(noteEntries) => Array.isArray(noteEntries) && noteEntries.every(isValidNoteEntry),
|
||||
)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private extractKnownWordEntriesFromNoteInfo(
|
||||
noteInfo: KnownWordCacheNoteInfo,
|
||||
preferredFields = this.getConfiguredFields(),
|
||||
|
||||
@@ -46,26 +46,15 @@ test('mature threshold falls back to default for invalid values', () => {
|
||||
|
||||
test('tier queries append Anki search props to a deck scope query', () => {
|
||||
const queries = buildKnownWordMaturityTierQueries('deck:"Mining"', 21);
|
||||
assert.equal(queries.mature, 'deck:"Mining" prop:ivl>=21 -is:learn');
|
||||
assert.equal(queries.young, 'deck:"Mining" prop:ivl>=1 prop:ivl<21 -is:learn');
|
||||
assert.equal(queries.learning, 'deck:"Mining" is:learn');
|
||||
});
|
||||
|
||||
test('interval tiers exclude (re)learning cards so the buckets stay disjoint', () => {
|
||||
const queries = buildKnownWordMaturityTierQueries('deck:"Mining"', 21);
|
||||
// A lapsed card keeps an interval of at least the lapse minInt (>= 1), so
|
||||
// without the exclusion the young query would claim every relearning card
|
||||
// and the learning tier could only ever match brand-new cards mid-step.
|
||||
for (const intervalQuery of [queries.mature, queries.young]) {
|
||||
assert.ok(intervalQuery.includes('-is:learn'));
|
||||
}
|
||||
assert.equal(queries.mature, 'deck:"Mining" prop:ivl>=21');
|
||||
assert.equal(queries.young, 'deck:"Mining" prop:ivl>=1 prop:ivl<21');
|
||||
assert.equal(queries.learning, 'deck:"Mining" is:learn');
|
||||
});
|
||||
|
||||
test('tier queries with an empty scope query have no leading space', () => {
|
||||
const queries = buildKnownWordMaturityTierQueries('', 30);
|
||||
assert.equal(queries.mature, 'prop:ivl>=30 -is:learn');
|
||||
assert.equal(queries.young, 'prop:ivl>=1 prop:ivl<30 -is:learn');
|
||||
assert.equal(queries.mature, 'prop:ivl>=30');
|
||||
assert.equal(queries.young, 'prop:ivl>=1 prop:ivl<30');
|
||||
assert.equal(queries.learning, 'is:learn');
|
||||
});
|
||||
|
||||
|
||||
@@ -3,10 +3,6 @@ import type { KnownWordMaturityTier } from '../types/subtitle';
|
||||
|
||||
export const DEFAULT_MATURE_INTERVAL_THRESHOLD_DAYS = 21;
|
||||
|
||||
// Version of the tier classification rules; part of the known-word cache
|
||||
// identity so a rule change invalidates caches built under the old rules.
|
||||
export const KNOWN_WORD_MATURITY_RULES_VERSION = 2;
|
||||
|
||||
// Ascending maturity; index order backs tier comparison.
|
||||
const TIER_ORDER: readonly KnownWordMaturityTier[] = ['new', 'learning', 'young', 'mature'];
|
||||
|
||||
@@ -41,20 +37,14 @@ export function getMatureIntervalThresholdDays(config: AnkiConnectConfig): numbe
|
||||
// Anki search props classify notes server-side: a note matches a tier query
|
||||
// when ANY of its cards matches, which implements most-mature-card-wins for
|
||||
// free once tiers are checked in mature > young > learning order.
|
||||
//
|
||||
// The interval tiers exclude is:learn so the per-card buckets stay disjoint and
|
||||
// match Anki's own card counts, where (re)learning is its own bucket rather
|
||||
// than part of young/mature. Without the exclusion a lapsed card - whose
|
||||
// interval is reset to at least lapse minInt, so >= 1 - is caught by the young
|
||||
// query first and the learning tier becomes unreachable in practice.
|
||||
export function buildKnownWordMaturityTierQueries(
|
||||
scopeQuery: string,
|
||||
thresholdDays: number,
|
||||
): KnownWordMaturityTierQueries {
|
||||
const prefix = scopeQuery.trim().length > 0 ? `${scopeQuery.trim()} ` : '';
|
||||
return {
|
||||
mature: `${prefix}prop:ivl>=${thresholdDays} -is:learn`,
|
||||
young: `${prefix}prop:ivl>=1 prop:ivl<${thresholdDays} -is:learn`,
|
||||
mature: `${prefix}prop:ivl>=${thresholdDays}`,
|
||||
young: `${prefix}prop:ivl>=1 prop:ivl<${thresholdDays}`,
|
||||
learning: `${prefix}is:learn`,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -497,8 +497,6 @@ describe('stats server API routes', () => {
|
||||
cachePath,
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
refreshedAtMs: 1,
|
||||
scope: 'deck:test',
|
||||
words: ['する'],
|
||||
}),
|
||||
);
|
||||
@@ -563,69 +561,6 @@ describe('stats server API routes', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('GET /api/stats/sessions enriches known-word metrics from a v4 maturity cache', async () => {
|
||||
await withTempDir(async (dir) => {
|
||||
const cachePath = path.join(dir, 'known-words.json');
|
||||
fs.writeFileSync(
|
||||
cachePath,
|
||||
JSON.stringify({
|
||||
version: 4,
|
||||
refreshedAtMs: 1,
|
||||
scope: 'deck:test',
|
||||
notes: {
|
||||
'101': [{ word: 'する', reading: 'する' }],
|
||||
'102': [{ word: '猫', reading: null }],
|
||||
},
|
||||
tiers: { '101': 'mature', '102': 'young' },
|
||||
}),
|
||||
);
|
||||
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getSessionWordsByLine: async (sessionId: number) =>
|
||||
sessionId === 1
|
||||
? [
|
||||
{ lineIndex: 1, headword: 'する', occurrenceCount: 2 },
|
||||
{ lineIndex: 2, headword: '未知', occurrenceCount: 1 },
|
||||
]
|
||||
: [],
|
||||
}),
|
||||
{ knownWordCachePath: cachePath },
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/sessions?limit=5');
|
||||
assert.equal(res.status, 200);
|
||||
const body = await res.json();
|
||||
const first = body[0];
|
||||
assert.equal(first.knownWordsSeen, 2);
|
||||
assert.equal(first.knownWordRate, 66.7);
|
||||
});
|
||||
});
|
||||
|
||||
it('GET /api/stats/sessions reports no known words when the cache format is unrecognized', async () => {
|
||||
await withTempDir(async (dir) => {
|
||||
const cachePath = path.join(dir, 'known-words.json');
|
||||
fs.writeFileSync(
|
||||
cachePath,
|
||||
JSON.stringify({ version: 99, refreshedAtMs: 1, scope: 'deck:test', notes: {} }),
|
||||
);
|
||||
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getSessionWordsByLine: async () => [
|
||||
{ lineIndex: 1, headword: 'する', occurrenceCount: 2 },
|
||||
],
|
||||
}),
|
||||
{ knownWordCachePath: cachePath },
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/sessions?limit=5');
|
||||
assert.equal(res.status, 200);
|
||||
const body = await res.json();
|
||||
assert.equal(body[0].knownWordsSeen, 0);
|
||||
});
|
||||
});
|
||||
|
||||
it('GET /api/stats/sessions/:id/events forwards event type filters to the tracker', async () => {
|
||||
let seenSessionId = 0;
|
||||
let seenLimit = 0;
|
||||
@@ -674,8 +609,6 @@ describe('stats server API routes', () => {
|
||||
cachePath,
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
refreshedAtMs: 1,
|
||||
scope: 'deck:test',
|
||||
words: ['知る', '猫'],
|
||||
}),
|
||||
);
|
||||
@@ -2873,39 +2806,6 @@ Aligned English subtitle
|
||||
assert.equal(deleteCalls, 0);
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/:animeId deletes the whole library entry', async () => {
|
||||
let deletedAnimeId: number | null = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
deleteAnime: async (animeId: number) => {
|
||||
deletedAnimeId = animeId;
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/7', { method: 'DELETE' });
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(deletedAnimeId, 7);
|
||||
assert.deepEqual(await res.json(), { ok: true });
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/:animeId rejects non-positive anime ids', async () => {
|
||||
let deleteCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
deleteAnime: async () => {
|
||||
deleteCalls += 1;
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/0', { method: 'DELETE' });
|
||||
|
||||
assert.equal(res.status, 400);
|
||||
assert.equal(deleteCalls, 0);
|
||||
});
|
||||
|
||||
it('POST /api/stats/anki/browse returns 400 for missing noteId', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/anki/browse', { method: 'POST' });
|
||||
|
||||
@@ -1466,190 +1466,6 @@ test('deleteVideo ignores the currently active video and keeps new writes flusha
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Attach the derived rows a real session would produce to every recorded line
|
||||
* of `animeId`: word/kanji entries and their occurrences, monthly rollups, and
|
||||
* cover art backed by a shared blob.
|
||||
*/
|
||||
function seedDerivedAnimeData(db: DatabaseSync, animeId: number): void {
|
||||
const lines = db
|
||||
.prepare(
|
||||
'SELECT line_id AS lineId, CREATED_DATE AS seenMs FROM imm_subtitle_lines WHERE anime_id = ?',
|
||||
)
|
||||
.all(animeId) as Array<{ lineId: number; seenMs: number }>;
|
||||
assert.ok(lines.length > 0, 'expected recorded subtitle lines to decorate');
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (9001, '天気', '天気', 'てんき', 'noun', '名詞', 0, 0, 0)`,
|
||||
).run();
|
||||
db.prepare(
|
||||
`INSERT INTO imm_kanji(id, kanji, first_seen, last_seen, frequency) VALUES (9101, '気', 0, 0, 0)`,
|
||||
).run();
|
||||
|
||||
const insertWordOccurrence = db.prepare(
|
||||
'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, 9001, 1, ?)',
|
||||
);
|
||||
const insertKanjiOccurrence = db.prepare(
|
||||
'INSERT INTO imm_kanji_line_occurrences(line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, 9101, 1, ?)',
|
||||
);
|
||||
for (const line of lines) {
|
||||
insertWordOccurrence.run(line.lineId, line.seenMs);
|
||||
insertKanjiOccurrence.run(line.lineId, line.seenMs);
|
||||
}
|
||||
db.prepare(
|
||||
`UPDATE imm_words SET frequency = ?, first_seen = ?, last_seen = ? WHERE id = 9001`,
|
||||
).run(lines.length, 0, 0);
|
||||
|
||||
const videoIds = (
|
||||
db
|
||||
.prepare('SELECT video_id AS videoId FROM imm_videos WHERE anime_id = ?')
|
||||
.all(animeId) as Array<{ videoId: number }>
|
||||
).map((row) => row.videoId);
|
||||
const insertMonthlyRollup = db.prepare(
|
||||
`INSERT INTO imm_monthly_rollups(rollup_month, video_id, total_sessions, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (202401, ?, 1, '0', '0')`,
|
||||
);
|
||||
const insertArt = db.prepare(
|
||||
`INSERT INTO imm_media_art(video_id, anilist_id, cover_url, cover_blob, cover_blob_hash, fetched_at_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, 4242, 'https://example.test/cover.jpg', NULL, 'deadbeef', '0', '0', '0')`,
|
||||
);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_cover_art_blobs(blob_hash, cover_blob, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES ('deadbeef', X'FFD8FFD9', '0', '0')`,
|
||||
).run();
|
||||
for (const videoId of videoIds) {
|
||||
insertMonthlyRollup.run(videoId);
|
||||
insertArt.run(videoId);
|
||||
}
|
||||
}
|
||||
|
||||
test('deleteAnime removes every episode, session and library row for the title', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
|
||||
for (const episode of ['S02E05', 'S02E06']) {
|
||||
tracker = new Ctor({ dbPath });
|
||||
tracker.handleMediaChange(`/tmp/Little Witch Academia ${episode}.mkv`, `Episode ${episode}`);
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
tracker.recordSubtitleLine('今日は晴れです', 0, 1.2);
|
||||
tracker.recordCardsMined(1);
|
||||
tracker.destroy();
|
||||
tracker = null;
|
||||
}
|
||||
|
||||
tracker = new Ctor({ dbPath });
|
||||
const privateApi = tracker as unknown as { db: DatabaseSync };
|
||||
const animeId = (
|
||||
privateApi.db.prepare('SELECT anime_id FROM imm_anime LIMIT 1').get() as {
|
||||
anime_id: number;
|
||||
} | null
|
||||
)?.anime_id;
|
||||
assert.ok(animeId);
|
||||
|
||||
// The tokenizer does not run in this harness, so attach vocabulary, kanji,
|
||||
// rollups and cover art to the recorded lines by hand. Without them the
|
||||
// "everything is gone" assertions below would pass against empty tables.
|
||||
seedDerivedAnimeData(privateApi.db, animeId);
|
||||
|
||||
const countOf = (sql: string): number =>
|
||||
(privateApi.db.prepare(sql).get() as { total: number }).total;
|
||||
for (const table of [
|
||||
'imm_words',
|
||||
'imm_kanji',
|
||||
'imm_word_line_occurrences',
|
||||
'imm_kanji_line_occurrences',
|
||||
'imm_daily_rollups',
|
||||
'imm_monthly_rollups',
|
||||
'imm_media_art',
|
||||
'imm_cover_art_blobs',
|
||||
]) {
|
||||
assert.ok(
|
||||
countOf(`SELECT COUNT(*) AS total FROM ${table}`) > 0,
|
||||
`precondition: ${table} should hold rows before the delete`,
|
||||
);
|
||||
}
|
||||
|
||||
const libraryBefore = await tracker.getAnimeLibrary();
|
||||
assert.equal(libraryBefore.length, 1);
|
||||
assert.equal(libraryBefore[0]?.episodeCount, 2);
|
||||
|
||||
await tracker.deleteAnime(animeId);
|
||||
|
||||
const libraryAfter = await tracker.getAnimeLibrary();
|
||||
assert.equal(libraryAfter.length, 0);
|
||||
|
||||
for (const table of [
|
||||
'imm_anime',
|
||||
'imm_lifetime_anime',
|
||||
'imm_videos',
|
||||
'imm_sessions',
|
||||
'imm_subtitle_lines',
|
||||
'imm_daily_rollups',
|
||||
'imm_monthly_rollups',
|
||||
'imm_lifetime_media',
|
||||
'imm_words',
|
||||
'imm_kanji',
|
||||
'imm_word_line_occurrences',
|
||||
'imm_kanji_line_occurrences',
|
||||
'imm_media_art',
|
||||
'imm_cover_art_blobs',
|
||||
]) {
|
||||
assert.equal(
|
||||
countOf(`SELECT COUNT(*) AS total FROM ${table}`),
|
||||
0,
|
||||
`${table} should be empty after deleting the only title`,
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleteAnime ignores the title of the currently active session', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath });
|
||||
tracker.handleMediaChange('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
|
||||
const privateApi = tracker as unknown as {
|
||||
db: DatabaseSync;
|
||||
sessionState: { sessionId: number; videoId: number } | null;
|
||||
};
|
||||
const videoId = privateApi.sessionState?.videoId;
|
||||
assert.ok(videoId);
|
||||
const animeId = (
|
||||
privateApi.db.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?').get(videoId) as {
|
||||
anime_id: number | null;
|
||||
} | null
|
||||
)?.anime_id;
|
||||
assert.ok(animeId);
|
||||
|
||||
await tracker.deleteAnime(animeId);
|
||||
|
||||
const animeCountRow = privateApi.db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_anime WHERE anime_id = ?')
|
||||
.get(animeId) as { total: number };
|
||||
const videoCountRow = privateApi.db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_videos WHERE video_id = ?')
|
||||
.get(videoId) as { total: number };
|
||||
|
||||
assert.equal(animeCountRow.total, 1);
|
||||
assert.equal(videoCountRow.total, 1);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('handleMediaChange links parsed anime metadata on the active video row', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
@@ -83,7 +83,6 @@ import {
|
||||
} from './immersion-tracker/query-library';
|
||||
import {
|
||||
cleanupVocabularyStats,
|
||||
deleteAnime as deleteAnimeQuery,
|
||||
deleteSession as deleteSessionQuery,
|
||||
deleteSessions as deleteSessionsQuery,
|
||||
deleteVideo as deleteVideoQuery,
|
||||
@@ -734,28 +733,6 @@ export class ImmersionTrackerService {
|
||||
deleteVideoQuery(this.db, videoId);
|
||||
}
|
||||
|
||||
async deleteAnime(animeId: number): Promise<void> {
|
||||
// The active video's anime link is assigned asynchronously after the title
|
||||
// is parsed, so a guard reading imm_videos too early sees a null and lets
|
||||
// the delete through — then the late update recreates the anime row.
|
||||
const pendingVideoId = this.sessionState?.videoId;
|
||||
if (pendingVideoId !== undefined) {
|
||||
await this.pendingAnimeMetadataUpdates.get(pendingVideoId);
|
||||
}
|
||||
|
||||
const activeVideoId = this.sessionState?.videoId;
|
||||
if (activeVideoId !== undefined) {
|
||||
const activeAnime = this.db
|
||||
.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?')
|
||||
.get(activeVideoId) as { anime_id: number | null } | null;
|
||||
if (activeAnime?.anime_id === animeId) {
|
||||
this.logger.warn(`Ignoring delete request for active immersion anime ${animeId}`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
deleteAnimeQuery(this.db, animeId);
|
||||
}
|
||||
|
||||
async reassignAnimeAnilist(
|
||||
animeId: number,
|
||||
info: {
|
||||
|
||||
@@ -1,331 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database } from '../sqlite.js';
|
||||
import type { DatabaseSync } from '../sqlite.js';
|
||||
import { ensureSchema } from '../storage.js';
|
||||
import { deleteSession, deleteSessions, deleteVideo } from '../query-maintenance.js';
|
||||
|
||||
const DAY_MS = 86_400_000;
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
|
||||
function makeDbPath(): string {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-lexical-removal-test-'));
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) return;
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* Seed two episodes of one anime, each with one ended session.
|
||||
*
|
||||
* `lines` places a word occurrence on a specific day so tests can control which
|
||||
* session holds a word's first/last occurrence.
|
||||
*/
|
||||
function seed(
|
||||
db: DatabaseSync,
|
||||
lines: Array<{ session: 1 | 2; wordId: number; dayOffset: number; count?: number }>,
|
||||
options: { legacyRows?: boolean } = {},
|
||||
): void {
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'show', 'Show', ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'v1', 1, 'Ep 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS}),
|
||||
(2, 'v2', 1, 'Ep 2', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 's1', 1, '${BASE_MS}', '${BASE_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS}),
|
||||
(2, 's2', 2, '${BASE_MS + DAY_MS}', '${BASE_MS + DAY_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS});
|
||||
`);
|
||||
|
||||
const insertLine = db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, 1, ?, ?, ?, ?)`,
|
||||
);
|
||||
const insertWord = db.prepare(
|
||||
`INSERT OR IGNORE INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (?, ?, ?, '', 'noun', '名詞', 0, 0, 0)`,
|
||||
);
|
||||
// `legacyRows` reproduces databases written before the seen_ms column existed,
|
||||
// where the timestamp has to be read back off the subtitle line.
|
||||
const insertOccurrence = options.legacyRows
|
||||
? db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (?, ?, ?)`,
|
||||
)
|
||||
: db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)`,
|
||||
);
|
||||
|
||||
let lineId = 0;
|
||||
for (const line of lines) {
|
||||
lineId += 1;
|
||||
const seenMs = BASE_MS + line.dayOffset * DAY_MS;
|
||||
insertLine.run(lineId, line.session, line.session, lineId, `line ${lineId}`, seenMs, seenMs);
|
||||
insertWord.run(line.wordId, `語${line.wordId}`, `語${line.wordId}`);
|
||||
if (options.legacyRows) {
|
||||
insertOccurrence.run(lineId, line.wordId, line.count ?? 1);
|
||||
} else {
|
||||
insertOccurrence.run(lineId, line.wordId, line.count ?? 1, seenMs);
|
||||
}
|
||||
}
|
||||
|
||||
// Match what the tracker maintains: aggregates derived from the occurrences.
|
||||
db.exec(`
|
||||
UPDATE imm_words SET
|
||||
frequency = (
|
||||
SELECT COALESCE(SUM(o.occurrence_count), 0)
|
||||
FROM imm_word_line_occurrences o WHERE o.word_id = imm_words.id
|
||||
),
|
||||
first_seen = (
|
||||
SELECT MIN(sl.CREATED_DATE) / 1000
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE o.word_id = imm_words.id
|
||||
),
|
||||
last_seen = (
|
||||
SELECT MAX(sl.LAST_UPDATE_DATE) / 1000
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE o.word_id = imm_words.id
|
||||
)
|
||||
`);
|
||||
}
|
||||
|
||||
function createDb(
|
||||
lines: Parameters<typeof seed>[1],
|
||||
options: Parameters<typeof seed>[2] = {},
|
||||
): { db: DatabaseSync; dbPath: string } {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
seed(db, lines, options);
|
||||
return { db, dbPath };
|
||||
}
|
||||
|
||||
function readWord(
|
||||
db: DatabaseSync,
|
||||
wordId: number,
|
||||
): { frequency: number; firstSeen: number; lastSeen: number } | null {
|
||||
return (
|
||||
(db
|
||||
.prepare(
|
||||
'SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen FROM imm_words WHERE id = ?',
|
||||
)
|
||||
.get(wordId) as { frequency: number; firstSeen: number; lastSeen: number } | null) ?? null
|
||||
);
|
||||
}
|
||||
|
||||
test('deleting a session subtracts only the occurrences it removed', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 10, dayOffset: 0, count: 3 },
|
||||
{ session: 2, wordId: 10, dayOffset: 1, count: 4 },
|
||||
]);
|
||||
|
||||
try {
|
||||
assert.equal(readWord(db, 10)?.frequency, 7);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 10)?.frequency, 4, 'only session 1 occurrences are subtracted');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the session that held a word first drops the word entirely', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 11, dayOffset: 0 },
|
||||
{ session: 2, wordId: 12, dayOffset: 1 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSession(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 11), null, 'word seen only in the deleted session is removed');
|
||||
assert.ok(readWord(db, 12), 'word seen elsewhere survives');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the earliest session moves first_seen forward to the surviving line', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 13, dayOffset: 0 },
|
||||
{ session: 2, wordId: 13, dayOffset: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
assert.equal(readWord(db, 13)?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 13);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(
|
||||
word?.firstSeen,
|
||||
Math.floor((BASE_MS + 5 * DAY_MS) / 1000),
|
||||
'first_seen advances to the remaining occurrence',
|
||||
);
|
||||
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 5 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting the latest session moves last_seen back to the surviving line', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 14, dayOffset: 0 },
|
||||
{ session: 2, wordId: 14, dayOffset: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSession(db, 2);
|
||||
|
||||
const word = readWord(db, 14);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(word?.lastSeen, Math.floor(BASE_MS / 1000), 'last_seen falls back to session 1');
|
||||
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting an interior occurrence leaves the surrounding extremes untouched', () => {
|
||||
// Session 2 carries the middle occurrence; sessions bracket it in time.
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 15, dayOffset: 0 },
|
||||
{ session: 2, wordId: 15, dayOffset: 3 },
|
||||
{ session: 1, wordId: 15, dayOffset: 9 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteSessions(db, [2]);
|
||||
|
||||
const word = readWord(db, 15);
|
||||
assert.equal(word?.frequency, 2);
|
||||
assert.equal(word?.firstSeen, Math.floor(BASE_MS / 1000));
|
||||
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 9 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a stored frequency that has drifted low is repaired instead of dropping a live word', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 16, dayOffset: 0, count: 5 },
|
||||
{ session: 2, wordId: 16, dayOffset: 4, count: 5 },
|
||||
]);
|
||||
|
||||
try {
|
||||
// Simulate drift: the stored total is lower than the occurrences justify, so
|
||||
// naive subtraction would take the word to zero while rows still reference it.
|
||||
db.prepare('UPDATE imm_words SET frequency = 5 WHERE id = ?').run(16);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 16);
|
||||
assert.ok(word, 'word with surviving occurrences is not deleted');
|
||||
assert.equal(word?.frequency, 5, 'frequency is recomputed from the surviving occurrences');
|
||||
assert.equal(word?.firstSeen, Math.floor((BASE_MS + 4 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting a video subtracts every occurrence carried by its lines', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 17, dayOffset: 0, count: 2 },
|
||||
{ session: 1, wordId: 17, dayOffset: 1, count: 3 },
|
||||
{ session: 2, wordId: 17, dayOffset: 2, count: 4 },
|
||||
{ session: 2, wordId: 18, dayOffset: 2, count: 1 },
|
||||
]);
|
||||
|
||||
try {
|
||||
deleteVideo(db, 1);
|
||||
|
||||
assert.equal(readWord(db, 17)?.frequency, 4, 'both lines from video 1 are subtracted');
|
||||
assert.equal(readWord(db, 18)?.frequency, 1, 'untouched video keeps its counts');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('occurrence rows written before the seen_ms column still resolve their dates', () => {
|
||||
const { db, dbPath } = createDb(
|
||||
[
|
||||
{ session: 1, wordId: 20, dayOffset: 0 },
|
||||
{ session: 2, wordId: 20, dayOffset: 6 },
|
||||
],
|
||||
{ legacyRows: true },
|
||||
);
|
||||
|
||||
try {
|
||||
assert.equal(
|
||||
(
|
||||
db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_word_line_occurrences WHERE seen_ms IS NULL')
|
||||
.get() as { total: number }
|
||||
).total,
|
||||
2,
|
||||
'precondition: rows carry no denormalised timestamp',
|
||||
);
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 20);
|
||||
assert.equal(word?.frequency, 1);
|
||||
assert.equal(
|
||||
word?.firstSeen,
|
||||
Math.floor((BASE_MS + 6 * DAY_MS) / 1000),
|
||||
'falls back to the subtitle line timestamp',
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('upgrading an older database backfills seen_ms from the subtitle lines', () => {
|
||||
const { db, dbPath } = createDb(
|
||||
[
|
||||
{ session: 1, wordId: 21, dayOffset: 0 },
|
||||
{ session: 2, wordId: 21, dayOffset: 2 },
|
||||
],
|
||||
{ legacyRows: true },
|
||||
);
|
||||
|
||||
try {
|
||||
// Re-run ensureSchema the way a pre-19 database would be opened.
|
||||
db.exec('DELETE FROM imm_schema_version');
|
||||
db.exec(`INSERT INTO imm_schema_version(schema_version, applied_at_ms) VALUES (18, '0')`);
|
||||
ensureSchema(db);
|
||||
|
||||
const rows = db
|
||||
.prepare('SELECT line_id AS lineId, seen_ms AS seenMs FROM imm_word_line_occurrences')
|
||||
.all() as Array<{ lineId: number; seenMs: number | null }>;
|
||||
assert.equal(rows.length, 2);
|
||||
for (const row of rows) {
|
||||
assert.ok(row.seenMs, `line ${row.lineId} should have a backfilled timestamp`);
|
||||
}
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.seenMs).sort((a, b) => Number(a) - Number(b)),
|
||||
[BASE_MS, BASE_MS + 2 * DAY_MS],
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
@@ -4982,145 +4982,3 @@ test('getTrendsDashboard librarySummary is empty when no rollups exist', () => {
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('getVocabularyStats counts the distinct anime each word appeared in', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const startedAtMs = 1_700_000_000_000;
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs}),
|
||||
(22, 'vocab-b', 'Vocab B', ${startedAtMs}, ${startedAtMs});
|
||||
`);
|
||||
const videoIds = [1, 2].map((n) =>
|
||||
getOrCreateVideoRecord(db, `local:/tmp/vocab-anime-${n}.mkv`, {
|
||||
canonicalTitle: `Vocab Anime ${n}`,
|
||||
sourcePath: `/tmp/vocab-anime-${n}.mkv`,
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
}),
|
||||
);
|
||||
const sessionIds = videoIds.map(
|
||||
(videoId) => startSessionRecord(db, videoId, startedAtMs).sessionId,
|
||||
);
|
||||
|
||||
// 猫 spans both titles, twice in the first, so the count has to deduplicate
|
||||
// by anime rather than just tallying occurrence rows. 犬 stays in one title.
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 1,
|
||||
occurrenceCount: 5,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 2,
|
||||
occurrenceCount: 1,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[1]!,
|
||||
videoId: videoIds[1]!,
|
||||
animeId: 22,
|
||||
lineIndex: 3,
|
||||
occurrenceCount: 4,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId: sessionIds[0]!,
|
||||
videoId: videoIds[0]!,
|
||||
animeId: 11,
|
||||
lineIndex: 4,
|
||||
occurrenceCount: 2,
|
||||
startedAtMs,
|
||||
word: '犬',
|
||||
reading: 'いぬ',
|
||||
});
|
||||
|
||||
const rows = getVocabularyStats(db, 10);
|
||||
const cat = rows.find((row) => row.headword === '猫');
|
||||
const dog = rows.find((row) => row.headword === '犬');
|
||||
|
||||
assert.equal(cat?.animeCount, 2, '猫 was seen in two anime across three lines');
|
||||
assert.equal(dog?.animeCount, 1, '犬 was seen in one');
|
||||
assert.equal(cat?.frequency, 10, 'frequency still aggregates across both titles');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('getVocabularyStats still applies part-of-speech exclusions', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const startedAtMs = 1_700_000_000_000;
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (11, 'vocab-a', 'Vocab A', ${startedAtMs}, ${startedAtMs});
|
||||
`);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/vocab-exclude.mkv', {
|
||||
canonicalTitle: 'Vocab Exclude',
|
||||
sourcePath: '/tmp/vocab-exclude.mkv',
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
|
||||
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId,
|
||||
videoId,
|
||||
animeId: 11,
|
||||
lineIndex: 1,
|
||||
occurrenceCount: 9,
|
||||
startedAtMs,
|
||||
word: '走る',
|
||||
reading: 'はしる',
|
||||
partOfSpeech: 'verb',
|
||||
});
|
||||
insertFilteredWordOccurrence(db, {
|
||||
sessionId,
|
||||
videoId,
|
||||
animeId: 11,
|
||||
lineIndex: 2,
|
||||
occurrenceCount: 3,
|
||||
startedAtMs,
|
||||
word: '猫',
|
||||
reading: 'ねこ',
|
||||
partOfSpeech: 'noun',
|
||||
});
|
||||
|
||||
const all = getVocabularyStats(db, 10);
|
||||
assert.deepEqual(
|
||||
all.map((row) => row.headword).sort(),
|
||||
['猫', '走る'],
|
||||
'precondition: both parts of speech are present',
|
||||
);
|
||||
|
||||
const nounsOnly = getVocabularyStats(db, 10, ['verb']);
|
||||
assert.deepEqual(
|
||||
nounsOnly.map((row) => row.headword),
|
||||
['猫'],
|
||||
'excluded part of speech is filtered out even though it ranks higher',
|
||||
);
|
||||
assert.equal(nounsOnly[0]?.animeCount, 1, 'surviving rows keep their anime count');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -115,29 +115,18 @@ export function getVocabularyStats(
|
||||
const whereClause = hasExclude
|
||||
? `WHERE (part_of_speech IS NULL OR part_of_speech NOT IN (${placeholders}))`
|
||||
: '';
|
||||
// The page is selected before the join so `animeCount` is only computed for the
|
||||
// rows being returned. Aggregating first made every request walk each word's
|
||||
// entire occurrence history — seconds of blocked event loop on a large library,
|
||||
// because only the ordering, not the aggregate, decides which rows survive.
|
||||
const stmt = db.prepare(`
|
||||
WITH page AS (
|
||||
SELECT id, headword, word, reading, part_of_speech, pos1, pos2, pos3,
|
||||
frequency, frequency_rank, first_seen, last_seen
|
||||
FROM imm_words
|
||||
${whereClause}
|
||||
ORDER BY frequency DESC, id
|
||||
LIMIT ? OFFSET ?
|
||||
)
|
||||
SELECT p.id AS wordId, p.headword, p.word, p.reading,
|
||||
p.part_of_speech AS partOfSpeech, p.pos1, p.pos2, p.pos3,
|
||||
p.frequency, p.frequency_rank AS frequencyRank,
|
||||
p.first_seen AS firstSeen, p.last_seen AS lastSeen,
|
||||
SELECT w.id AS wordId, w.headword, w.word, w.reading,
|
||||
w.part_of_speech AS partOfSpeech, w.pos1, w.pos2, w.pos3,
|
||||
w.frequency, w.frequency_rank AS frequencyRank,
|
||||
w.first_seen AS firstSeen, w.last_seen AS lastSeen,
|
||||
COUNT(DISTINCT sl.anime_id) AS animeCount
|
||||
FROM page p
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = p.id
|
||||
FROM imm_words w
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id AND sl.anime_id IS NOT NULL
|
||||
GROUP BY p.id
|
||||
ORDER BY p.frequency DESC, p.id
|
||||
${whereClause ? whereClause.replace('part_of_speech', 'w.part_of_speech') : ''}
|
||||
GROUP BY w.id
|
||||
ORDER BY w.frequency DESC LIMIT ? OFFSET ?
|
||||
`);
|
||||
const visibleRows: VocabularyStatsRow[] = [];
|
||||
let offset = 0;
|
||||
|
||||
@@ -9,12 +9,14 @@ import { PartOfSpeech, type MergedToken } from '../../../types';
|
||||
import { shouldExcludeTokenFromVocabularyPersistence } from '../tokenizer/annotation-stage';
|
||||
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
|
||||
import {
|
||||
applyLexicalRemovals,
|
||||
cleanupUnusedCoverArtBlobHash,
|
||||
deleteSessionsByIds,
|
||||
findSharedCoverBlobHash,
|
||||
planLexicalRemovalsForSessions,
|
||||
planLexicalRemovalsForVideos,
|
||||
getAffectedKanjiIdsForSessions,
|
||||
getAffectedKanjiIdsForVideo,
|
||||
getAffectedWordIdsForSessions,
|
||||
getAffectedWordIdsForVideo,
|
||||
refreshLexicalAggregates,
|
||||
toDbMs,
|
||||
toDbTimestamp,
|
||||
} from './query-shared';
|
||||
@@ -230,13 +232,12 @@ export async function cleanupVocabularyStats(
|
||||
WHERE id = ?`,
|
||||
);
|
||||
const moveOccurrencesStmt = db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms)
|
||||
SELECT line_id, ?, occurrence_count, seen_ms
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
|
||||
SELECT line_id, ?, occurrence_count
|
||||
FROM imm_word_line_occurrences
|
||||
WHERE word_id = ?
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)`,
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
const deleteOccurrencesStmt = db.prepare(
|
||||
'DELETE FROM imm_word_line_occurrences WHERE word_id = ?',
|
||||
@@ -483,16 +484,14 @@ export function isVideoWatched(db: DatabaseSync, videoId: number): boolean {
|
||||
|
||||
export function deleteSession(db: DatabaseSync, sessionId: number): void {
|
||||
const sessionIds = [sessionId];
|
||||
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
// Measured inside the write lock: the plan records what the delete removes,
|
||||
// and applying a plan taken against a different snapshot would subtract the
|
||||
// wrong totals from imm_words/imm_kanji.
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
@@ -504,13 +503,14 @@ export function deleteSession(db: DatabaseSync, sessionId: number): void {
|
||||
|
||||
export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
|
||||
if (sessionIds.length === 0) return;
|
||||
const affectedWordIds = getAffectedWordIdsForSessions(db, sessionIds);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
@@ -520,83 +520,24 @@ export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an entire library entry: every episode of the anime, all of their
|
||||
* sessions and derived stats, and the anime row itself.
|
||||
*
|
||||
* Mirrors {@link deleteVideo} per episode, but batches the lexical refresh and
|
||||
* lifetime rebuild into a single transaction so a multi-episode title doesn't
|
||||
* pay for one full rebuild per episode.
|
||||
*/
|
||||
export function deleteAnime(db: DatabaseSync, animeId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const videoIds = (
|
||||
db.prepare('SELECT video_id FROM imm_videos WHERE anime_id = ?').all(animeId) as Array<{
|
||||
video_id: number;
|
||||
}>
|
||||
).map((row) => row.video_id);
|
||||
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, videoIds);
|
||||
const coverBlobHashes: string[] = [];
|
||||
const sessionIds: number[] = [];
|
||||
for (const videoId of videoIds) {
|
||||
const artRow = db
|
||||
.prepare('SELECT cover_blob_hash AS coverBlobHash FROM imm_media_art WHERE video_id = ?')
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
if (artRow?.coverBlobHash) {
|
||||
coverBlobHashes.push(artRow.coverBlobHash);
|
||||
}
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
sessionIds.push(...sessions.map((session) => session.session_id));
|
||||
}
|
||||
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
const deleteLinesStmt = db.prepare('DELETE FROM imm_subtitle_lines WHERE video_id = ?');
|
||||
const deleteDailyStmt = db.prepare('DELETE FROM imm_daily_rollups WHERE video_id = ?');
|
||||
const deleteMonthlyStmt = db.prepare('DELETE FROM imm_monthly_rollups WHERE video_id = ?');
|
||||
const deleteArtStmt = db.prepare('DELETE FROM imm_media_art WHERE video_id = ?');
|
||||
const deleteVideoStmt = db.prepare('DELETE FROM imm_videos WHERE video_id = ?');
|
||||
for (const videoId of videoIds) {
|
||||
deleteLinesStmt.run(videoId);
|
||||
deleteDailyStmt.run(videoId);
|
||||
deleteMonthlyStmt.run(videoId);
|
||||
deleteArtStmt.run(videoId);
|
||||
deleteVideoStmt.run(videoId);
|
||||
}
|
||||
for (const coverBlobHash of new Set(coverBlobHashes)) {
|
||||
cleanupUnusedCoverArtBlobHash(db, coverBlobHash);
|
||||
}
|
||||
db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?').run(animeId);
|
||||
db.prepare('DELETE FROM imm_anime WHERE anime_id = ?').run(animeId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function deleteVideo(db: DatabaseSync, videoId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const artRow = db
|
||||
.prepare(
|
||||
`
|
||||
const artRow = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT cover_blob_hash AS coverBlobHash
|
||||
FROM imm_media_art
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
)
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, [videoId]);
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
)
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
const affectedWordIds = getAffectedWordIdsForVideo(db, videoId);
|
||||
const affectedKanjiIds = getAffectedKanjiIdsForVideo(db, videoId);
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
deleteSessionsByIds(
|
||||
db,
|
||||
sessions.map((session) => session.session_id),
|
||||
@@ -607,7 +548,7 @@ export function deleteVideo(db: DatabaseSync, videoId: number): void {
|
||||
db.prepare('DELETE FROM imm_media_art WHERE video_id = ?').run(videoId);
|
||||
cleanupUnusedCoverArtBlobHash(db, artRow?.coverBlobHash ?? null);
|
||||
db.prepare('DELETE FROM imm_videos WHERE video_id = ?').run(videoId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
refreshLexicalAggregates(db, affectedWordIds, affectedKanjiIds);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
|
||||
@@ -203,167 +203,6 @@ export function getAffectedKanjiIdsForVideo(db: DatabaseSync, videoId: number):
|
||||
return getAffectedIdsForVideo(db, 'kanji', videoId);
|
||||
}
|
||||
|
||||
/** Per-entity totals that a pending delete is about to remove. */
|
||||
interface LexicalRemoval {
|
||||
id: number;
|
||||
removedFrequency: number;
|
||||
removedFirstSeenMs: number | null;
|
||||
removedLastSeenMs: number | null;
|
||||
}
|
||||
|
||||
/** What a pending delete removes from `imm_words` and `imm_kanji`. */
|
||||
export interface LexicalRemovalPlan {
|
||||
words: LexicalRemoval[];
|
||||
kanji: LexicalRemoval[];
|
||||
}
|
||||
|
||||
export const EMPTY_LEXICAL_REMOVAL_PLAN: LexicalRemovalPlan = { words: [], kanji: [] };
|
||||
|
||||
function collectLexicalRemovals(
|
||||
db: DatabaseSync,
|
||||
entity: LexicalEntity,
|
||||
lineScopeSql: string,
|
||||
params: number[],
|
||||
): LexicalRemoval[] {
|
||||
const table = entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
|
||||
const col = `${entity}_id`;
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT
|
||||
o.${col} AS id,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS removedFrequency,
|
||||
MIN(COALESCE(o.seen_ms, sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS removedFirstSeenMs,
|
||||
MAX(COALESCE(o.seen_ms, sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS removedLastSeenMs
|
||||
FROM imm_subtitle_lines sl
|
||||
JOIN ${table} o ON o.line_id = sl.line_id
|
||||
WHERE ${lineScopeSql}
|
||||
GROUP BY o.${col}`,
|
||||
)
|
||||
.all(...params) as LexicalRemoval[];
|
||||
}
|
||||
|
||||
function planLexicalRemovals(
|
||||
db: DatabaseSync,
|
||||
lineScopeSql: string,
|
||||
params: number[],
|
||||
): LexicalRemovalPlan {
|
||||
return {
|
||||
words: collectLexicalRemovals(db, 'word', lineScopeSql, params),
|
||||
kanji: collectLexicalRemovals(db, 'kanji', lineScopeSql, params),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Measure what deleting these sessions removes from the vocabulary tables.
|
||||
*
|
||||
* Must run before the rows are deleted. Reads only the subtitle lines in scope,
|
||||
* unlike {@link refreshLexicalAggregates}, which re-reads every occurrence of
|
||||
* every affected word across the whole library.
|
||||
*/
|
||||
export function planLexicalRemovalsForSessions(
|
||||
db: DatabaseSync,
|
||||
sessionIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
if (sessionIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
|
||||
return planLexicalRemovals(db, `sl.session_id IN (${makePlaceholders(sessionIds)})`, sessionIds);
|
||||
}
|
||||
|
||||
/** Measure what deleting these videos removes from the vocabulary tables. */
|
||||
export function planLexicalRemovalsForVideos(
|
||||
db: DatabaseSync,
|
||||
videoIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
if (videoIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
|
||||
return planLexicalRemovals(db, `sl.video_id IN (${makePlaceholders(videoIds)})`, videoIds);
|
||||
}
|
||||
|
||||
function toStoredSeenSeconds(ms: number | null): number | null {
|
||||
if (ms === null || !Number.isFinite(ms)) return null;
|
||||
return Math.floor(ms / 1000);
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a removal plan to the vocabulary aggregates.
|
||||
*
|
||||
* Frequencies are adjusted by subtraction, which is exact and touches only the
|
||||
* affected rows. `first_seen`/`last_seen` only need a rescan when the removed
|
||||
* lines held the current extreme, and rows whose frequency reaches zero are
|
||||
* verified against the surviving occurrences before deletion — so stored counts
|
||||
* that have drifted still converge on the truth instead of dropping a live row.
|
||||
*/
|
||||
export function applyLexicalRemovals(db: DatabaseSync, plan: LexicalRemovalPlan): void {
|
||||
applyRemovalsForEntity(db, 'word', plan.words);
|
||||
applyRemovalsForEntity(db, 'kanji', plan.kanji);
|
||||
}
|
||||
|
||||
function applyRemovalsForEntity(
|
||||
db: DatabaseSync,
|
||||
entity: LexicalEntity,
|
||||
removals: LexicalRemoval[],
|
||||
): void {
|
||||
if (removals.length === 0) return;
|
||||
|
||||
const entityTable = entity === 'word' ? 'imm_words' : 'imm_kanji';
|
||||
const occurrenceTable =
|
||||
entity === 'word' ? 'imm_word_line_occurrences' : 'imm_kanji_line_occurrences';
|
||||
const col = `${entity}_id`;
|
||||
|
||||
const selectStmt = db.prepare(
|
||||
`SELECT frequency, first_seen AS firstSeen, last_seen AS lastSeen
|
||||
FROM ${entityTable}
|
||||
WHERE id = ?`,
|
||||
);
|
||||
const updateFrequencyStmt = db.prepare(`UPDATE ${entityTable} SET frequency = ? WHERE id = ?`);
|
||||
const hasOccurrencesStmt = db.prepare(
|
||||
`SELECT 1 AS found FROM ${occurrenceTable} WHERE ${col} = ? LIMIT 1`,
|
||||
);
|
||||
const deleteStmt = db.prepare(`DELETE FROM ${entityTable} WHERE id = ?`);
|
||||
|
||||
const needsExactRefresh: number[] = [];
|
||||
|
||||
for (const removal of removals) {
|
||||
const current = selectStmt.get(removal.id) as {
|
||||
frequency: number | null;
|
||||
firstSeen: number | null;
|
||||
lastSeen: number | null;
|
||||
} | null;
|
||||
if (!current) continue;
|
||||
|
||||
const nextFrequency = (current.frequency ?? 0) - removal.removedFrequency;
|
||||
if (nextFrequency <= 0) {
|
||||
// The rows in scope are already gone by now, so anything still pointing at
|
||||
// this entity means the stored frequency was stale rather than exhausted.
|
||||
if (hasOccurrencesStmt.get(removal.id)) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
} else {
|
||||
deleteStmt.run(removal.id);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const removedFirstSeen = toStoredSeenSeconds(removal.removedFirstSeenMs);
|
||||
const removedLastSeen = toStoredSeenSeconds(removal.removedLastSeenMs);
|
||||
const firstSeenMayHaveMoved =
|
||||
current.firstSeen === null ||
|
||||
(removedFirstSeen !== null && removedFirstSeen <= current.firstSeen);
|
||||
const lastSeenMayHaveMoved =
|
||||
current.lastSeen === null ||
|
||||
(removedLastSeen !== null && removedLastSeen >= current.lastSeen);
|
||||
if (firstSeenMayHaveMoved || lastSeenMayHaveMoved) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
updateFrequencyStmt.run(nextFrequency, removal.id);
|
||||
}
|
||||
|
||||
if (entity === 'word') {
|
||||
refreshWordAggregates(db, needsExactRefresh);
|
||||
} else {
|
||||
refreshKanjiAggregates(db, needsExactRefresh);
|
||||
}
|
||||
}
|
||||
|
||||
function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
|
||||
if (wordIds.length === 0) {
|
||||
return;
|
||||
@@ -375,18 +214,11 @@ function refreshWordAggregates(db: DatabaseSync, wordIds: number[]): void {
|
||||
SELECT
|
||||
w.id AS wordId,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
|
||||
MIN(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS firstSeen,
|
||||
MAX(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS lastSeen
|
||||
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
|
||||
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
|
||||
FROM imm_words w
|
||||
LEFT JOIN imm_word_line_occurrences o ON o.word_id = w.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE w.id IN (${makePlaceholders(wordIds)})
|
||||
GROUP BY w.id
|
||||
`,
|
||||
@@ -431,18 +263,11 @@ function refreshKanjiAggregates(db: DatabaseSync, kanjiIds: number[]): void {
|
||||
SELECT
|
||||
k.id AS kanjiId,
|
||||
COALESCE(SUM(o.occurrence_count), 0) AS frequency,
|
||||
MIN(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS firstSeen,
|
||||
MAX(COALESCE(o.seen_ms, (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = o.line_id
|
||||
))) AS lastSeen
|
||||
MIN(COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)) AS firstSeen,
|
||||
MAX(COALESCE(sl.LAST_UPDATE_DATE, sl.CREATED_DATE)) AS lastSeen
|
||||
FROM imm_kanji k
|
||||
LEFT JOIN imm_kanji_line_occurrences o ON o.kanji_id = k.id
|
||||
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE k.id IN (${makePlaceholders(kanjiIds)})
|
||||
GROUP BY k.id
|
||||
`,
|
||||
|
||||
@@ -192,33 +192,6 @@ function addColumnIfMissing(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy each subtitle line's timestamp onto its word/kanji occurrence rows.
|
||||
*
|
||||
* Vocabulary aggregates used to be recomputed by joining every occurrence back
|
||||
* to `imm_subtitle_lines` just to read two timestamps, which meant one random
|
||||
* read into the widest table per occurrence — the reason deleting a session cost
|
||||
* seconds on a large library. With the timestamp stored alongside the count, the
|
||||
* covering index answers those aggregates on its own.
|
||||
*/
|
||||
function backfillLexicalOccurrenceSeenMs(db: DatabaseSync): void {
|
||||
for (const table of ['imm_word_line_occurrences', 'imm_kanji_line_occurrences']) {
|
||||
addColumnIfMissing(db, table, 'seen_ms', 'INTEGER');
|
||||
db.exec(`
|
||||
UPDATE ${table}
|
||||
SET seen_ms = (
|
||||
SELECT COALESCE(sl.CREATED_DATE, sl.LAST_UPDATE_DATE)
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.line_id = ${table}.line_id
|
||||
)
|
||||
WHERE seen_ms IS NULL
|
||||
`);
|
||||
}
|
||||
// Superseded by the covering (entity, seen_ms, occurrence_count, line_id) indexes.
|
||||
db.exec('DROP INDEX IF EXISTS idx_word_line_occurrences_word');
|
||||
db.exec('DROP INDEX IF EXISTS idx_kanji_line_occurrences_kanji');
|
||||
}
|
||||
|
||||
function dropColumnIfExists(db: DatabaseSync, tableName: string, columnName: string): void {
|
||||
if (hasColumn(db, tableName, columnName)) {
|
||||
db.exec(`ALTER TABLE ${tableName} DROP COLUMN ${columnName}`);
|
||||
@@ -950,7 +923,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -961,7 +933,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -1122,7 +1093,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
word_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, word_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
|
||||
@@ -1133,7 +1103,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
line_id INTEGER NOT NULL,
|
||||
kanji_id INTEGER NOT NULL,
|
||||
occurrence_count INTEGER NOT NULL,
|
||||
seen_ms INTEGER,
|
||||
PRIMARY KEY(line_id, kanji_id),
|
||||
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
|
||||
@@ -1380,19 +1349,13 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_anime_line
|
||||
ON imm_subtitle_lines(anime_id, line_index)
|
||||
`);
|
||||
if (currentVersion?.schema_version && currentVersion.schema_version < 19) {
|
||||
backfillLexicalOccurrenceSeenMs(db);
|
||||
}
|
||||
|
||||
// Covering indexes: vocabulary aggregates (frequency, first/last seen) are
|
||||
// answered from the index alone, without reading the wide subtitle-line rows.
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word_seen
|
||||
ON imm_word_line_occurrences(word_id, seen_ms, occurrence_count, line_id)
|
||||
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word
|
||||
ON imm_word_line_occurrences(word_id, line_id)
|
||||
`);
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji_seen
|
||||
ON imm_kanji_line_occurrences(kanji_id, seen_ms, occurrence_count, line_id)
|
||||
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji
|
||||
ON imm_kanji_line_occurrences(kanji_id, line_id)
|
||||
`);
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_media_art_cover_blob_hash
|
||||
@@ -1512,23 +1475,21 @@ export function createTrackerPreparedStatements(db: DatabaseSync): TrackerPrepar
|
||||
`),
|
||||
wordLineOccurrenceUpsertStmt: db.prepare(`
|
||||
INSERT INTO imm_word_line_occurrences (
|
||||
line_id, word_id, occurrence_count, seen_ms
|
||||
line_id, word_id, occurrence_count
|
||||
) VALUES (
|
||||
?, ?, ?, ?
|
||||
?, ?, ?
|
||||
)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_word_line_occurrences.seen_ms, excluded.seen_ms)
|
||||
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count
|
||||
`),
|
||||
kanjiLineOccurrenceUpsertStmt: db.prepare(`
|
||||
INSERT INTO imm_kanji_line_occurrences (
|
||||
line_id, kanji_id, occurrence_count, seen_ms
|
||||
line_id, kanji_id, occurrence_count
|
||||
) VALUES (
|
||||
?, ?, ?, ?
|
||||
?, ?, ?
|
||||
)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
|
||||
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(imm_kanji_line_occurrences.seen_ms, excluded.seen_ms)
|
||||
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count
|
||||
`),
|
||||
videoAnimeIdSelectStmt: db.prepare(`
|
||||
SELECT anime_id FROM imm_videos
|
||||
@@ -1669,16 +1630,11 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
|
||||
const lineId = Number(lineResult.lastInsertRowid);
|
||||
for (const occurrence of write.wordOccurrences) {
|
||||
const wordId = incrementWordAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
|
||||
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount, currentMs);
|
||||
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount);
|
||||
}
|
||||
for (const occurrence of write.kanjiOccurrences) {
|
||||
const kanjiId = incrementKanjiAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
|
||||
stmts.kanjiLineOccurrenceUpsertStmt.run(
|
||||
lineId,
|
||||
kanjiId,
|
||||
occurrence.occurrenceCount,
|
||||
currentMs,
|
||||
);
|
||||
stmts.kanjiLineOccurrenceUpsertStmt.run(lineId, kanjiId, occurrence.occurrenceCount);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
export const SCHEMA_VERSION = 19;
|
||||
export const SCHEMA_VERSION = 18;
|
||||
export const DEFAULT_QUEUE_CAP = 1_000;
|
||||
export const DEFAULT_BATCH_SIZE = 25;
|
||||
export const DEFAULT_FLUSH_INTERVAL_MS = 500;
|
||||
|
||||
@@ -190,11 +190,4 @@ export function registerStatsLibraryRoutes(
|
||||
await tracker.deleteVideo(videoId);
|
||||
return c.json(statsJson('deleteVideo', { ok: true }));
|
||||
});
|
||||
|
||||
app.delete('/api/stats/anime/:animeId', async (c) => {
|
||||
const animeId = parseIntQuery(c.req.param('animeId'), 0);
|
||||
if (animeId <= 0) return c.body(null, 400);
|
||||
await tracker.deleteAnime(animeId);
|
||||
return c.json(statsJson('deleteAnime', { ok: true }));
|
||||
});
|
||||
}
|
||||
|
||||
@@ -6,18 +6,11 @@ import {
|
||||
getConfiguredWordFieldName,
|
||||
getPreferredNoteFieldValue,
|
||||
} from '../../../anki-field-config.js';
|
||||
import {
|
||||
knownWordsFromState,
|
||||
parseKnownWordCacheState,
|
||||
} from '../../../anki-integration/known-word-cache-format.js';
|
||||
import { createLogger } from '../../../logger.js';
|
||||
import type { AnkiConnectConfig } from '../../../types.js';
|
||||
import type { StatsExcludedWord } from '../../../types/stats-wire.js';
|
||||
import type { ImmersionTrackerService } from '../immersion-tracker-service.js';
|
||||
import { splitSentenceSearchTerms } from '../immersion-tracker/query-lexical.js';
|
||||
|
||||
const statsKnownWordsLogger = createLogger('stats:known-words');
|
||||
|
||||
const STATS_STATIC_CONTENT_TYPES: Record<string, string> = {
|
||||
'.css': 'text/css; charset=utf-8',
|
||||
'.gif': 'image/gif',
|
||||
@@ -91,14 +84,24 @@ export function parseExcludedWordsBody(body: unknown): StatsExcludedWord[] | nul
|
||||
export function loadKnownWordsSet(cachePath: string | undefined): Set<string> | null {
|
||||
if (!cachePath || !existsSync(cachePath)) return null;
|
||||
try {
|
||||
const state = parseKnownWordCacheState(JSON.parse(readFileSync(cachePath, 'utf-8')) as unknown);
|
||||
if (!state) {
|
||||
// A cache that exists but does not parse is a format mismatch, not an
|
||||
// empty collection; say so instead of reporting zero known words.
|
||||
statsKnownWordsLogger.warn(`Unrecognized known-word cache format at ${cachePath}`);
|
||||
return null;
|
||||
const raw = JSON.parse(readFileSync(cachePath, 'utf-8')) as {
|
||||
version?: number;
|
||||
words?: string[];
|
||||
notes?: Record<string, Array<{ word?: unknown; reading?: unknown }>>;
|
||||
};
|
||||
if ((raw.version === 1 || raw.version === 2) && Array.isArray(raw.words)) {
|
||||
return new Set(raw.words);
|
||||
}
|
||||
if (raw.version === 3 && raw.notes && typeof raw.notes === 'object') {
|
||||
const words = new Set<string>();
|
||||
for (const entries of Object.values(raw.notes)) {
|
||||
if (!Array.isArray(entries)) continue;
|
||||
for (const entry of entries) {
|
||||
if (entry && typeof entry.word === 'string' && entry.word) words.add(entry.word);
|
||||
}
|
||||
}
|
||||
return words;
|
||||
}
|
||||
return knownWordsFromState(state);
|
||||
} catch {
|
||||
// Treat an unreadable cache as unavailable.
|
||||
}
|
||||
|
||||
@@ -1,100 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { Database } from '../immersion-tracker/sqlite';
|
||||
import { ensureSchema } from '../immersion-tracker/storage';
|
||||
import { mergeSnapshotIntoDb } from './merge';
|
||||
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
const DAY_MS = 86_400_000;
|
||||
|
||||
/**
|
||||
* Build a database holding one anime, one episode, one ended session and a
|
||||
* single word occurrence.
|
||||
*
|
||||
* `legacyOccurrences` reproduces a peer whose schema predates the denormalised
|
||||
* `seen_ms` column, so the merge has to recover the timestamp from the line.
|
||||
*/
|
||||
function buildDb(
|
||||
dir: string,
|
||||
name: string,
|
||||
options: { word: string; seenMs: number; legacyOccurrences: boolean },
|
||||
): string {
|
||||
const dbPath = path.join(dir, name);
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'key-${name}', 'Show ${name}', ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'video-${name}', 1, 'Episode 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'uuid-${name}', 1, '${options.seenMs}', '${options.seenMs + 1000}', 2, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_subtitle_lines(line_id, session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 1, 1, 1, 0, 'line', ${options.seenMs}, ${options.seenMs});
|
||||
INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (1, '${options.word}', '${options.word}', '', 'noun', '名詞',
|
||||
${Math.floor(options.seenMs / 1000)}, ${Math.floor(options.seenMs / 1000)}, 1);
|
||||
`);
|
||||
db.exec(
|
||||
options.legacyOccurrences
|
||||
? 'INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count) VALUES (1, 1, 1)'
|
||||
: `INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms) VALUES (1, 1, 1, ${options.seenMs})`,
|
||||
);
|
||||
db.close();
|
||||
return dbPath;
|
||||
}
|
||||
|
||||
function mergedOccurrences(dbPath: string): Array<{ word: string; seenMs: number | null }> {
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT w.word AS word, o.seen_ms AS seenMs
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_words w ON w.id = o.word_id
|
||||
ORDER BY w.word`,
|
||||
)
|
||||
.all() as Array<{ word: string; seenMs: number | null }>;
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
for (const legacyOccurrences of [false, true]) {
|
||||
const label = legacyOccurrences ? 'a peer predating the seen_ms column' : 'a current peer';
|
||||
|
||||
test(`sync merge carries occurrence timestamps in from ${label}`, () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-merge-occurrences-test-'));
|
||||
try {
|
||||
const local = buildDb(dir, 'local.sqlite', {
|
||||
word: '猫',
|
||||
seenMs: BASE_MS,
|
||||
legacyOccurrences: false,
|
||||
});
|
||||
const remoteSeenMs = BASE_MS + 5 * DAY_MS;
|
||||
const remote = buildDb(dir, 'remote.sqlite', {
|
||||
word: '犬',
|
||||
seenMs: remoteSeenMs,
|
||||
legacyOccurrences,
|
||||
});
|
||||
|
||||
mergeSnapshotIntoDb(local, remote);
|
||||
|
||||
const rows = mergedOccurrences(local);
|
||||
assert.equal(rows.length, 2, 'local and remote occurrences both survive the merge');
|
||||
for (const row of rows) {
|
||||
assert.ok(row.seenMs, `${row.word} must carry a timestamp so aggregates stay index-only`);
|
||||
}
|
||||
assert.equal(
|
||||
Number(rows.find((row) => row.word === '犬')?.seenMs),
|
||||
remoteSeenMs,
|
||||
"the remote line's own timestamp is preserved rather than stamped with merge time",
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -215,16 +215,12 @@ function copySubtitleLines(
|
||||
'SELECT kanji_id, occurrence_count FROM imm_kanji_line_occurrences WHERE line_id = ?',
|
||||
);
|
||||
const insertWordOccurrence = local.query(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET
|
||||
occurrence_count = occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count) VALUES (?, ?, ?)
|
||||
ON CONFLICT(line_id, word_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
const insertKanjiOccurrence = local.query(
|
||||
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count, seen_ms) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
|
||||
occurrence_count = occurrence_count + excluded.occurrence_count,
|
||||
seen_ms = COALESCE(seen_ms, excluded.seen_ms)`,
|
||||
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count) VALUES (?, ?, ?)
|
||||
ON CONFLICT(line_id, kanji_id) DO UPDATE SET occurrence_count = occurrence_count + excluded.occurrence_count`,
|
||||
);
|
||||
|
||||
for (const row of rows) {
|
||||
@@ -245,20 +241,17 @@ function copySubtitleLines(
|
||||
],
|
||||
);
|
||||
summary.subtitleLinesAdded += 1;
|
||||
// Taken from the line rather than the remote occurrence row so this also
|
||||
// works when the remote database predates the seen_ms column.
|
||||
const seenMs = row.CREATED_DATE ?? row.LAST_UPDATE_DATE ?? null;
|
||||
|
||||
for (const occurrence of wordOccurrences.all(row.line_id) as SqlRow[]) {
|
||||
const localWordId = lexicon.resolveWord(Number(occurrence.word_id));
|
||||
const count = Number(occurrence.occurrence_count);
|
||||
insertWordOccurrence.run(localLineId, localWordId, count, seenMs);
|
||||
insertWordOccurrence.run(localLineId, localWordId, count);
|
||||
lexicon.addWordOccurrences(Number(occurrence.word_id), count);
|
||||
}
|
||||
for (const occurrence of kanjiOccurrences.all(row.line_id) as SqlRow[]) {
|
||||
const localKanjiId = lexicon.resolveKanji(Number(occurrence.kanji_id));
|
||||
const count = Number(occurrence.occurrence_count);
|
||||
insertKanjiOccurrence.run(localLineId, localKanjiId, count, seenMs);
|
||||
insertKanjiOccurrence.run(localLineId, localKanjiId, count);
|
||||
lexicon.addKanjiOccurrences(Number(occurrence.kanji_id), count);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,7 +63,7 @@ export interface AnnotationStageOptions {
|
||||
sourceText?: string;
|
||||
}
|
||||
|
||||
export function resolveKnownWordText(
|
||||
function resolveKnownWordText(
|
||||
surface: string,
|
||||
headword: string,
|
||||
matchMode: NPlusOneMatchMode,
|
||||
@@ -571,7 +571,7 @@ function isCompleteReadingForSurface(surface: string, reading: string): boolean
|
||||
// (see isCompleteReadingForSurface); undefined otherwise. Shared so the
|
||||
// known-word reading disambiguation and the reading fallback stay in sync if the
|
||||
// validity rule changes.
|
||||
export function resolveCompleteTokenReading(token: MergedToken): string | undefined {
|
||||
function resolveCompleteTokenReading(token: MergedToken): string | undefined {
|
||||
const normalizedReading = token.reading.trim();
|
||||
if (!normalizedReading || !isCompleteReadingForSurface(token.surface, normalizedReading)) {
|
||||
return undefined;
|
||||
@@ -584,7 +584,7 @@ export function resolveCompleteTokenReading(token: MergedToken): string | undefi
|
||||
// inflected surface's reading does not match the dictionary form's reading,
|
||||
// and partial furigana readings (see isCompleteReadingForSurface) would cause
|
||||
// false negatives. Undefined falls back to text-only matching (fail-open).
|
||||
export function resolveKnownWordReadingForMatch(
|
||||
function resolveKnownWordReadingForMatch(
|
||||
token: MergedToken,
|
||||
knownWordMatchMode: NPlusOneMatchMode,
|
||||
): string | undefined {
|
||||
|
||||
@@ -9,9 +9,7 @@ import {
|
||||
createSessionHelpModal,
|
||||
describeSessionHelpCommand,
|
||||
formatSessionHelpKeybinding,
|
||||
isKnownWordMaturityLegendEnabled,
|
||||
} from './session-help.js';
|
||||
import type { RuntimeOptionId, RuntimeOptionState } from '../../types/runtime-options.js';
|
||||
|
||||
test('session help describes sub-seek commands as subtitle-line navigation', () => {
|
||||
assert.equal(describeSessionHelpCommand(['sub-seek', 1]), 'Jump to next subtitle');
|
||||
@@ -106,34 +104,6 @@ test('session help builds rows from canonical session bindings and fixed overlay
|
||||
assert.ok(rows.some((row) => row.shortcut === 'Y then D' && row.action === 'Toggle DevTools'));
|
||||
});
|
||||
|
||||
function booleanRuntimeOption(id: RuntimeOptionId, value: boolean): RuntimeOptionState {
|
||||
return {
|
||||
id,
|
||||
label: id,
|
||||
scope: 'subtitle',
|
||||
valueType: 'boolean',
|
||||
value,
|
||||
allowedValues: [true, false],
|
||||
requiresRestart: false,
|
||||
};
|
||||
}
|
||||
|
||||
test('maturity legend requires both known-word highlighting and maturity coloring', () => {
|
||||
const highlightOn = booleanRuntimeOption('subtitle.annotation.knownWords.highlightEnabled', true);
|
||||
const highlightOff = booleanRuntimeOption(
|
||||
'subtitle.annotation.knownWords.highlightEnabled',
|
||||
false,
|
||||
);
|
||||
const maturityOn = booleanRuntimeOption('subtitle.annotation.knownWords.maturityEnabled', true);
|
||||
const maturityOff = booleanRuntimeOption('subtitle.annotation.knownWords.maturityEnabled', false);
|
||||
|
||||
assert.equal(isKnownWordMaturityLegendEnabled([highlightOn, maturityOn]), true);
|
||||
assert.equal(isKnownWordMaturityLegendEnabled([highlightOff, maturityOn]), false);
|
||||
assert.equal(isKnownWordMaturityLegendEnabled([highlightOn, maturityOff]), false);
|
||||
assert.equal(isKnownWordMaturityLegendEnabled([maturityOn]), false);
|
||||
assert.equal(isKnownWordMaturityLegendEnabled([]), false);
|
||||
});
|
||||
|
||||
function colorLegendRows(input: Parameters<typeof buildSessionHelpSections>[0]) {
|
||||
const sections = buildSessionHelpSections(input);
|
||||
return sections.find((section) => section.title === 'Color legend')?.rows ?? [];
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import type { ModalStateReader, RendererContext } from '../context';
|
||||
import type { RuntimeOptionId, RuntimeOptionState } from '../../types/runtime-options';
|
||||
import {
|
||||
buildSessionHelpSections,
|
||||
type SessionHelpSection,
|
||||
@@ -20,19 +19,6 @@ type SessionHelpBindingInfo = {
|
||||
fallbackUnavailable: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Tiers only render when known-word highlighting is also on, matching
|
||||
* getKnownWordMaturityEnabled in anki-integration/known-word-maturity.
|
||||
*/
|
||||
export function isKnownWordMaturityLegendEnabled(runtimeOptions: RuntimeOptionState[]): boolean {
|
||||
const isOn = (id: RuntimeOptionId): boolean =>
|
||||
runtimeOptions.some((option) => option.id === id && option.value === true);
|
||||
return (
|
||||
isOn('subtitle.annotation.knownWords.highlightEnabled') &&
|
||||
isOn('subtitle.annotation.knownWords.maturityEnabled')
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Maturity coloring is a live runtime toggle, so the color legend reads it from
|
||||
* runtime options instead of the resolved subtitle style. A missing or failing
|
||||
@@ -40,7 +26,11 @@ export function isKnownWordMaturityLegendEnabled(runtimeOptions: RuntimeOptionSt
|
||||
*/
|
||||
async function readKnownWordMaturityEnabled(): Promise<boolean> {
|
||||
try {
|
||||
return isKnownWordMaturityLegendEnabled(await window.electronAPI.getRuntimeOptions());
|
||||
const runtimeOptions = await window.electronAPI.getRuntimeOptions();
|
||||
return runtimeOptions.some(
|
||||
(option) =>
|
||||
option.id === 'subtitle.annotation.knownWords.maturityEnabled' && option.value === true,
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -141,7 +141,6 @@ export interface StatsJsonResponseMap {
|
||||
deleteSessions: StatsOkResponse;
|
||||
deleteSession: StatsOkResponse;
|
||||
deleteVideo: StatsOkResponse;
|
||||
deleteAnime: StatsOkResponse;
|
||||
anilistSearch: StatsAnilistSearchResult[];
|
||||
knownWords: string[];
|
||||
knownWordsSummary: StatsKnownWordsSummary;
|
||||
@@ -220,7 +219,6 @@ export interface StatsHttpClient {
|
||||
deleteSession: (sessionId: number) => Promise<void>;
|
||||
deleteSessions: (sessionIds: number[]) => Promise<void>;
|
||||
deleteVideo: (videoId: number) => Promise<void>;
|
||||
deleteAnime: (animeId: number) => Promise<void>;
|
||||
getKnownWords: () => Promise<string[]>;
|
||||
getKnownWordsSummary: () => Promise<StatsKnownWordsSummary>;
|
||||
getAnimeKnownWordsSummary: (animeId: number) => Promise<StatsKnownWordsSummary>;
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { Suspense, lazy, useCallback, useState } from 'react';
|
||||
import { DeleteConfirmDialog } from './components/layout/DeleteConfirmDialog';
|
||||
import { DeleteProgressToast } from './components/common/DeleteProgressToast';
|
||||
import { TabBar } from './components/layout/TabBar';
|
||||
import { OverviewTab } from './components/overview/OverviewTab';
|
||||
import { useExcludedWords } from './hooks/useExcludedWords';
|
||||
@@ -294,7 +293,6 @@ export function App() {
|
||||
</Suspense>
|
||||
) : null}
|
||||
<DeleteConfirmDialog />
|
||||
<DeleteProgressToast />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { useState, useEffect, useRef } from 'react';
|
||||
import { useState, useEffect } from 'react';
|
||||
import { useAnimeDetail } from '../../hooks/useAnimeDetail';
|
||||
import { getStatsClient } from '../../hooks/useStatsApi';
|
||||
import { confirmAnimeDelete } from '../../lib/delete-confirm';
|
||||
import { epochDayToDate } from '../../lib/formatters';
|
||||
import { AnimeHeader } from './AnimeHeader';
|
||||
import { EpisodeList } from './EpisodeList';
|
||||
@@ -17,8 +16,6 @@ interface AnimeDetailViewProps {
|
||||
onBack: () => void;
|
||||
onNavigateToWord?: (wordId: number) => void;
|
||||
onOpenEpisodeDetail?: (videoId: number) => void;
|
||||
/** Called after the whole library entry is deleted, so the caller can refresh. */
|
||||
onAnimeDeleted?: () => void;
|
||||
}
|
||||
|
||||
type Range = 14 | 30 | 90;
|
||||
@@ -142,14 +139,10 @@ export function AnimeDetailView({
|
||||
onBack,
|
||||
onNavigateToWord,
|
||||
onOpenEpisodeDetail,
|
||||
onAnimeDeleted,
|
||||
}: AnimeDetailViewProps) {
|
||||
const { data, loading, error, reload } = useAnimeDetail(animeId);
|
||||
const [showAnilistSelector, setShowAnilistSelector] = useState(false);
|
||||
const [coverRetryToken, setCoverRetryToken] = useState(0);
|
||||
const [isDeletingAnime, setIsDeletingAnime] = useState(false);
|
||||
const [deleteError, setDeleteError] = useState<string | null>(null);
|
||||
const isDeletingAnimeRef = useRef(false);
|
||||
const knownWordsSummary = useAnimeKnownWords(animeId);
|
||||
|
||||
useEffect(() => {
|
||||
@@ -161,40 +154,6 @@ export function AnimeDetailView({
|
||||
if (!data?.detail) return <div className="text-ctp-overlay2 p-4">Anime not found</div>;
|
||||
|
||||
const { detail, episodes, anilistEntries } = data;
|
||||
|
||||
const handleDeleteAnime = async () => {
|
||||
if (isDeletingAnimeRef.current) return;
|
||||
isDeletingAnimeRef.current = true;
|
||||
// Cleared up front so cancelling a retry doesn't leave the previous
|
||||
// attempt's failure on screen.
|
||||
setDeleteError(null);
|
||||
let confirmed = false;
|
||||
try {
|
||||
confirmed = await confirmAnimeDelete(detail.canonicalTitle, detail.episodeCount);
|
||||
} catch (err) {
|
||||
setDeleteError(err instanceof Error ? err.message : 'Failed to confirm delete.');
|
||||
isDeletingAnimeRef.current = false;
|
||||
return;
|
||||
}
|
||||
if (!confirmed) {
|
||||
isDeletingAnimeRef.current = false;
|
||||
return;
|
||||
}
|
||||
|
||||
setDeleteError(null);
|
||||
setIsDeletingAnime(true);
|
||||
try {
|
||||
await getStatsClient().deleteAnime(animeId);
|
||||
onAnimeDeleted?.();
|
||||
onBack();
|
||||
} catch (err) {
|
||||
setDeleteError(err instanceof Error ? err.message : 'Failed to delete this title.');
|
||||
setIsDeletingAnime(false);
|
||||
} finally {
|
||||
isDeletingAnimeRef.current = false;
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
<button
|
||||
@@ -209,10 +168,7 @@ export function AnimeDetailView({
|
||||
anilistEntries={anilistEntries ?? []}
|
||||
coverRetryToken={coverRetryToken}
|
||||
onChangeAnilist={() => setShowAnilistSelector(true)}
|
||||
onDeleteAnime={() => void handleDeleteAnime()}
|
||||
isDeletingAnime={isDeletingAnime}
|
||||
/>
|
||||
{deleteError ? <div className="text-sm text-ctp-red">{deleteError}</div> : null}
|
||||
<AnimeOverviewStats detail={detail} knownWordsSummary={knownWordsSummary} />
|
||||
<EpisodeList
|
||||
episodes={episodes}
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { AnimeHeader } from './AnimeHeader';
|
||||
import { confirmAnimeDelete, setDeleteConfirmPresenter } from '../../lib/delete-confirm';
|
||||
import type { AnimeDetailData } from '../../types/stats';
|
||||
|
||||
const DETAIL: AnimeDetailData['detail'] = {
|
||||
animeId: 3,
|
||||
canonicalTitle: 'Project Radio Noise Season 2',
|
||||
anilistId: 20661,
|
||||
titleRomaji: 'Toaru Kagaku no Railgun S',
|
||||
titleEnglish: 'A Certain Scientific Railgun S',
|
||||
titleNative: null,
|
||||
description: null,
|
||||
totalSessions: 1,
|
||||
totalActiveMs: 960_000,
|
||||
totalCards: 0,
|
||||
totalTokensSeen: 1_655,
|
||||
totalLinesSeen: 300,
|
||||
totalLookupCount: 0,
|
||||
totalLookupHits: 0,
|
||||
totalYomitanLookupCount: 0,
|
||||
episodeCount: 1,
|
||||
lastWatchedMs: 1_700_000_000_000,
|
||||
};
|
||||
|
||||
test('AnimeHeader offers a delete-entry action alongside the AniList actions', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<AnimeHeader detail={DETAIL} anilistEntries={[]} onDeleteAnime={() => {}} />,
|
||||
);
|
||||
|
||||
assert.match(markup, /Delete Entry/);
|
||||
assert.match(markup, /Delete this title and every session and stat recorded for it/);
|
||||
});
|
||||
|
||||
test('AnimeHeader hides the delete action when no handler is wired', () => {
|
||||
const markup = renderToStaticMarkup(<AnimeHeader detail={DETAIL} anilistEntries={[]} />);
|
||||
|
||||
assert.doesNotMatch(markup, /Delete Entry/);
|
||||
});
|
||||
|
||||
test('AnimeHeader shows a pending state while the entry is being deleted', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<AnimeHeader detail={DETAIL} anilistEntries={[]} onDeleteAnime={() => {}} isDeletingAnime />,
|
||||
);
|
||||
|
||||
assert.match(markup, /disabled=""/);
|
||||
assert.match(markup, /animate-spin/);
|
||||
assert.doesNotMatch(markup, /Delete Entry/);
|
||||
});
|
||||
|
||||
test('confirmAnimeDelete spells out how much data the entry deletion removes', async () => {
|
||||
const seen: string[] = [];
|
||||
const restore = setDeleteConfirmPresenter((message) => {
|
||||
seen.push(message);
|
||||
return false;
|
||||
});
|
||||
|
||||
try {
|
||||
await confirmAnimeDelete('Railgun Season 2', 1);
|
||||
await confirmAnimeDelete('Railgun Season 2', 3);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
|
||||
assert.match(seen[0] ?? '', /"Railgun Season 2"/);
|
||||
assert.match(seen[0] ?? '', /1 episode\b/);
|
||||
assert.match(seen[1] ?? '', /3 episodes/);
|
||||
assert.match(seen[0] ?? '', /every session and stat/);
|
||||
});
|
||||
@@ -6,8 +6,6 @@ interface AnimeHeaderProps {
|
||||
anilistEntries: AnilistEntry[];
|
||||
coverRetryToken?: number;
|
||||
onChangeAnilist?: () => void;
|
||||
onDeleteAnime?: () => void;
|
||||
isDeletingAnime?: boolean;
|
||||
}
|
||||
|
||||
function AnilistButton({ entry }: { entry: AnilistEntry }) {
|
||||
@@ -34,8 +32,6 @@ export function AnimeHeader({
|
||||
anilistEntries,
|
||||
coverRetryToken = 0,
|
||||
onChangeAnilist,
|
||||
onDeleteAnime,
|
||||
isDeletingAnime = false,
|
||||
}: AnimeHeaderProps) {
|
||||
const altTitles = [detail.titleRomaji, detail.titleEnglish, detail.titleNative].filter(
|
||||
(t): t is string => t != null && t !== detail.canonicalTitle,
|
||||
@@ -99,25 +95,6 @@ export function AnimeHeader({
|
||||
: 'Link to AniList'}
|
||||
</button>
|
||||
)}
|
||||
{onDeleteAnime && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={onDeleteAnime}
|
||||
disabled={isDeletingAnime}
|
||||
title="Delete this title and every session and stat recorded for it"
|
||||
className="inline-flex items-center gap-1 px-2 py-1 text-xs rounded bg-ctp-surface1 text-ctp-red/80 hover:bg-ctp-red/15 hover:text-ctp-red transition-colors disabled:opacity-50 disabled:cursor-not-allowed"
|
||||
>
|
||||
{isDeletingAnime ? (
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className="h-3 w-3 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
|
||||
/>
|
||||
) : (
|
||||
<span aria-hidden="true">{'\u2715'}</span>
|
||||
)}
|
||||
{isDeletingAnime ? 'Deleting\u2026' : 'Delete Entry'}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
{detail.description && (
|
||||
<p className="text-xs text-ctp-subtext0 mt-3 line-clamp-3 leading-relaxed">
|
||||
|
||||
@@ -53,7 +53,7 @@ export function AnimeTab({
|
||||
onNavigateToWord,
|
||||
onOpenEpisodeDetail,
|
||||
}: AnimeTabProps) {
|
||||
const { anime, loading, error, reload } = useAnimeLibrary();
|
||||
const { anime, loading, error } = useAnimeLibrary();
|
||||
const [search, setSearch] = useState('');
|
||||
const [sortKey, setSortKey] = useState<SortKey>('lastWatched');
|
||||
const [cardSize, setCardSize] = useState<LibraryCardSize>(() =>
|
||||
@@ -98,7 +98,6 @@ export function AnimeTab({
|
||||
? (videoId) => onOpenEpisodeDetail(selectedAnimeId, videoId)
|
||||
: undefined
|
||||
}
|
||||
onAnimeDeleted={reload}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,49 +1,32 @@
|
||||
import { useSyncExternalStore } from 'react';
|
||||
import { getDeleteProgressSnapshot, subscribeDeleteProgress } from '../../lib/delete-progress';
|
||||
interface DeleteProgressToastProps {
|
||||
/** Number of sessions currently being deleted. The toast is hidden when 0. */
|
||||
count: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Global "deletion in progress" indicator.
|
||||
* Fixed-position toast shown while session deletions are in flight.
|
||||
*
|
||||
* Mounted once at the app root, outside every tab panel, so it shows no matter
|
||||
* which view started the delete — per-tab copies were hidden along with their
|
||||
* `hidden` tab panel, and detail views had no indicator at all. State comes
|
||||
* from the delete-progress store rather than props so it also survives the
|
||||
* initiating component unmounting mid-request.
|
||||
*
|
||||
* Renders two signals: a sweeping bar pinned to the top edge (visible even when
|
||||
* the eye is on the content being deleted) and a bottom-right toast naming the
|
||||
* work.
|
||||
* The per-row delete buttons are only visible on hover, so once the confirm
|
||||
* dialog closes the user has no signal that a (potentially slow) batch delete
|
||||
* is still running. This stays on screen, independent of hover, until the work
|
||||
* finishes.
|
||||
*/
|
||||
export function DeleteProgressToast() {
|
||||
const { count, label } = useSyncExternalStore(
|
||||
subscribeDeleteProgress,
|
||||
getDeleteProgressSnapshot,
|
||||
getDeleteProgressSnapshot,
|
||||
);
|
||||
|
||||
export function DeleteProgressToast({ count }: DeleteProgressToastProps) {
|
||||
if (count <= 0) return null;
|
||||
|
||||
const message = count > 1 ? `Deleting ${count} items` : (label ?? 'Deleting');
|
||||
|
||||
return (
|
||||
<>
|
||||
<div
|
||||
<div
|
||||
role="status"
|
||||
aria-live="polite"
|
||||
className="fixed bottom-4 right-4 z-50 flex items-center gap-3 rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-4 py-3 shadow-lg shadow-black/30"
|
||||
>
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className="fixed inset-x-0 top-0 z-[2147483646] h-0.5 overflow-hidden bg-ctp-surface1"
|
||||
>
|
||||
<div className="h-full w-1/3 animate-indeterminate rounded-full bg-gradient-to-r from-ctp-red via-ctp-peach to-ctp-red" />
|
||||
</div>
|
||||
<div
|
||||
role="status"
|
||||
aria-live="polite"
|
||||
className="fixed bottom-4 right-4 z-[2147483646] flex items-center gap-3 rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-4 py-3 shadow-lg shadow-black/30"
|
||||
>
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className="h-4 w-4 shrink-0 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
|
||||
/>
|
||||
<span className="text-sm text-ctp-text">{message}…</span>
|
||||
</div>
|
||||
</>
|
||||
className="h-4 w-4 shrink-0 animate-spin rounded-full border-2 border-ctp-surface2 border-t-ctp-red"
|
||||
/>
|
||||
<span className="text-sm text-ctp-text">
|
||||
Deleting {count} session{count === 1 ? '' : 's'}…
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import { StreakCalendar } from './StreakCalendar';
|
||||
import { RecentSessions } from './RecentSessions';
|
||||
import { TrackingSnapshot } from './TrackingSnapshot';
|
||||
import { TrendChart } from '../trends/TrendChart';
|
||||
import { DeleteProgressToast } from '../common/DeleteProgressToast';
|
||||
import { buildOverviewSummary, buildStreakCalendar } from '../../lib/dashboard-data';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import { getStatsClient } from '../../hooks/useStatsApi';
|
||||
@@ -159,6 +160,8 @@ export function OverviewTab({
|
||||
deletingIds={deletingIds}
|
||||
isActive={isActive}
|
||||
/>
|
||||
|
||||
<DeleteProgressToast count={deletingIds.size} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useEffect, useMemo, useState } from 'react';
|
||||
import { useSessions } from '../../hooks/useSessions';
|
||||
import { SessionRow } from './SessionRow';
|
||||
import { SessionDetail } from './SessionDetail';
|
||||
import { DeleteProgressToast } from '../common/DeleteProgressToast';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import { confirmBucketDelete, confirmSessionDelete } from '../../lib/delete-confirm';
|
||||
import { formatDuration, formatNumber, formatSessionDayLabel } from '../../lib/formatters';
|
||||
@@ -343,6 +344,8 @@ export function SessionsTab({
|
||||
{search.trim() ? 'No sessions matching your search.' : 'No sessions recorded yet.'}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<DeleteProgressToast count={deletingSessionIds.size} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useCallback, useState, useEffect } from 'react';
|
||||
import { useState, useEffect } from 'react';
|
||||
import { getStatsClient } from './useStatsApi';
|
||||
import type { AnimeLibraryItem } from '../types/stats';
|
||||
|
||||
@@ -6,11 +6,6 @@ export function useAnimeLibrary() {
|
||||
const [anime, setAnime] = useState<AnimeLibraryItem[]>([]);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [reloadToken, setReloadToken] = useState(0);
|
||||
|
||||
const reload = useCallback(() => {
|
||||
setReloadToken((token) => token + 1);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
@@ -28,7 +23,7 @@ export function useAnimeLibrary() {
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [reloadToken]);
|
||||
}, []);
|
||||
|
||||
return { anime, loading, error, reload };
|
||||
return { anime, loading, error };
|
||||
}
|
||||
|
||||
@@ -15,7 +15,6 @@ import type {
|
||||
} from '../types/stats';
|
||||
import type { StatsMineCardParams, StatsMineCardResponse } from './mining';
|
||||
import { appendCoverRetryToken } from './cover-retry';
|
||||
import { trackDelete } from './delete-progress';
|
||||
|
||||
type StatsLocationLike = Pick<Location, 'protocol' | 'origin' | 'search'>;
|
||||
|
||||
@@ -170,29 +169,17 @@ export const apiClient = {
|
||||
});
|
||||
},
|
||||
deleteSession: async (sessionId: number): Promise<void> => {
|
||||
await trackDelete('Deleting session', () =>
|
||||
fetchResponse(`/api/stats/sessions/${sessionId}`, { method: 'DELETE' }),
|
||||
);
|
||||
await fetchResponse(`/api/stats/sessions/${sessionId}`, { method: 'DELETE' });
|
||||
},
|
||||
deleteSessions: async (sessionIds: number[]): Promise<void> => {
|
||||
const label = `Deleting ${sessionIds.length} session${sessionIds.length === 1 ? '' : 's'}`;
|
||||
await trackDelete(label, () =>
|
||||
fetchResponse('/api/stats/sessions', {
|
||||
method: 'DELETE',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ sessionIds } satisfies StatsDeleteSessionsRequest),
|
||||
}),
|
||||
);
|
||||
await fetchResponse('/api/stats/sessions', {
|
||||
method: 'DELETE',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ sessionIds } satisfies StatsDeleteSessionsRequest),
|
||||
});
|
||||
},
|
||||
deleteVideo: async (videoId: number): Promise<void> => {
|
||||
await trackDelete('Deleting episode', () =>
|
||||
fetchResponse(`/api/stats/media/${videoId}`, { method: 'DELETE' }),
|
||||
);
|
||||
},
|
||||
deleteAnime: async (animeId: number): Promise<void> => {
|
||||
await trackDelete('Deleting library entry', () =>
|
||||
fetchResponse(`/api/stats/anime/${animeId}`, { method: 'DELETE' }),
|
||||
);
|
||||
await fetchResponse(`/api/stats/media/${videoId}`, { method: 'DELETE' });
|
||||
},
|
||||
getKnownWords: () => fetchJson('knownWords', '/api/stats/known-words'),
|
||||
getKnownWordsSummary: () => fetchJson('knownWordsSummary', '/api/stats/known-words-summary'),
|
||||
|
||||
@@ -60,13 +60,6 @@ export function confirmAnimeGroupDelete(title: string, count: number): Promise<b
|
||||
);
|
||||
}
|
||||
|
||||
export function confirmAnimeDelete(title: string, episodeCount: number): Promise<boolean> {
|
||||
const episodes = `${episodeCount} episode${episodeCount === 1 ? '' : 's'}`;
|
||||
return confirmWithStatsNativeDialogLayer(
|
||||
`Delete "${title}" from your library? This removes ${episodes} plus every session and stat recorded for them.`,
|
||||
);
|
||||
}
|
||||
|
||||
export function confirmEpisodeDelete(title: string): Promise<boolean> {
|
||||
return confirmWithStatsNativeDialogLayer(`Delete "${title}" and all its sessions?`);
|
||||
}
|
||||
|
||||
@@ -1,127 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { DeleteProgressToast } from '../components/common/DeleteProgressToast';
|
||||
import { apiClient } from './api-client';
|
||||
import {
|
||||
beginDeleteTask,
|
||||
getDeleteProgressSnapshot,
|
||||
resetDeleteProgress,
|
||||
subscribeDeleteProgress,
|
||||
trackDelete,
|
||||
} from './delete-progress';
|
||||
|
||||
test('delete progress store reports the oldest label and notifies subscribers', () => {
|
||||
resetDeleteProgress();
|
||||
let notifications = 0;
|
||||
const unsubscribe = subscribeDeleteProgress(() => {
|
||||
notifications += 1;
|
||||
});
|
||||
|
||||
try {
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 0, label: null });
|
||||
|
||||
const endFirst = beginDeleteTask('Deleting session');
|
||||
const endSecond = beginDeleteTask('Deleting episode');
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 2, label: 'Deleting session' });
|
||||
assert.equal(notifications, 2);
|
||||
|
||||
endFirst();
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 1, label: 'Deleting episode' });
|
||||
|
||||
endFirst();
|
||||
assert.deepEqual(
|
||||
getDeleteProgressSnapshot(),
|
||||
{ count: 1, label: 'Deleting episode' },
|
||||
'ending the same task twice must not drop another task',
|
||||
);
|
||||
|
||||
endSecond();
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 0, label: null });
|
||||
} finally {
|
||||
unsubscribe();
|
||||
resetDeleteProgress();
|
||||
}
|
||||
});
|
||||
|
||||
test('trackDelete clears the indicator even when the request rejects', async () => {
|
||||
resetDeleteProgress();
|
||||
await assert.rejects(
|
||||
trackDelete('Deleting library entry', async () => {
|
||||
assert.equal(getDeleteProgressSnapshot().count, 1);
|
||||
throw new Error('boom');
|
||||
}),
|
||||
/boom/,
|
||||
);
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 0, label: null });
|
||||
});
|
||||
|
||||
test('DeleteProgressToast stays hidden while idle and reports active deletes', () => {
|
||||
resetDeleteProgress();
|
||||
assert.equal(renderToStaticMarkup(<DeleteProgressToast />), '');
|
||||
|
||||
const end = beginDeleteTask('Deleting library entry');
|
||||
try {
|
||||
const markup = renderToStaticMarkup(<DeleteProgressToast />);
|
||||
assert.match(markup, /role="status"/);
|
||||
assert.match(markup, /Deleting library entry/);
|
||||
assert.match(markup, /animate-indeterminate/);
|
||||
} finally {
|
||||
end();
|
||||
}
|
||||
|
||||
const endFirst = beginDeleteTask('Deleting session');
|
||||
const endSecond = beginDeleteTask('Deleting episode');
|
||||
try {
|
||||
assert.match(renderToStaticMarkup(<DeleteProgressToast />), /Deleting 2 items/);
|
||||
} finally {
|
||||
endFirst();
|
||||
endSecond();
|
||||
resetDeleteProgress();
|
||||
}
|
||||
});
|
||||
|
||||
test('the delete indicator is mounted once at the app root, not inside tab panels', () => {
|
||||
const srcDir = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const read = (relativePath: string): string =>
|
||||
fs.readFileSync(path.join(srcDir, relativePath), 'utf8');
|
||||
|
||||
const app = read('App.tsx');
|
||||
assert.match(app, /<DeleteProgressToast \/>/);
|
||||
// Sibling of the confirm dialog: outside every `hidden` tab panel, so the
|
||||
// indicator survives tab switches and detail-view navigation.
|
||||
assert.match(app, /<DeleteConfirmDialog \/>\s*<DeleteProgressToast \/>/);
|
||||
|
||||
for (const tab of [
|
||||
'components/overview/OverviewTab.tsx',
|
||||
'components/sessions/SessionsTab.tsx',
|
||||
]) {
|
||||
assert.doesNotMatch(read(tab), /DeleteProgressToast/, `${tab} must not mount its own toast`);
|
||||
}
|
||||
});
|
||||
|
||||
test('every api client delete registers with the global progress indicator', async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
const seenCounts: number[] = [];
|
||||
globalThis.fetch = (async () => {
|
||||
seenCounts.push(getDeleteProgressSnapshot().count);
|
||||
return new Response('{}', { status: 200, headers: { 'Content-Type': 'application/json' } });
|
||||
}) as typeof globalThis.fetch;
|
||||
|
||||
try {
|
||||
resetDeleteProgress();
|
||||
await apiClient.deleteSession(1);
|
||||
await apiClient.deleteSessions([1, 2]);
|
||||
await apiClient.deleteVideo(3);
|
||||
await apiClient.deleteAnime(4);
|
||||
|
||||
assert.deepEqual(seenCounts, [1, 1, 1, 1]);
|
||||
assert.deepEqual(getDeleteProgressSnapshot(), { count: 0, label: null });
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
resetDeleteProgress();
|
||||
}
|
||||
});
|
||||
@@ -1,74 +0,0 @@
|
||||
/**
|
||||
* Tiny external store tracking in-flight delete requests.
|
||||
*
|
||||
* Every delete goes through the API client, which registers here, so a single
|
||||
* globally mounted indicator can report progress no matter which tab, detail
|
||||
* view or overlay window started the work. Keeping the state outside React
|
||||
* means the indicator survives the deleting component unmounting mid-request
|
||||
* (navigating back after deleting a library entry, for example).
|
||||
*/
|
||||
|
||||
export interface DeleteProgressSnapshot {
|
||||
/** Number of delete requests currently in flight. */
|
||||
count: number;
|
||||
/** Label for the oldest in-flight request, or null when idle. */
|
||||
label: string | null;
|
||||
}
|
||||
|
||||
const IDLE_SNAPSHOT: DeleteProgressSnapshot = { count: 0, label: null };
|
||||
|
||||
const activeTasks = new Map<number, string>();
|
||||
const listeners = new Set<() => void>();
|
||||
let nextTaskId = 1;
|
||||
let snapshot: DeleteProgressSnapshot = IDLE_SNAPSHOT;
|
||||
|
||||
function publish(): void {
|
||||
if (activeTasks.size === 0) {
|
||||
snapshot = IDLE_SNAPSHOT;
|
||||
} else {
|
||||
const [oldestLabel] = activeTasks.values();
|
||||
snapshot = { count: activeTasks.size, label: oldestLabel ?? null };
|
||||
}
|
||||
for (const listener of listeners) listener();
|
||||
}
|
||||
|
||||
export function subscribeDeleteProgress(listener: () => void): () => void {
|
||||
listeners.add(listener);
|
||||
return () => {
|
||||
listeners.delete(listener);
|
||||
};
|
||||
}
|
||||
|
||||
export function getDeleteProgressSnapshot(): DeleteProgressSnapshot {
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
/** Register a delete as started. Returns the function that ends it. */
|
||||
export function beginDeleteTask(label: string): () => void {
|
||||
const taskId = nextTaskId++;
|
||||
activeTasks.set(taskId, label);
|
||||
publish();
|
||||
let ended = false;
|
||||
return () => {
|
||||
if (ended) return;
|
||||
ended = true;
|
||||
activeTasks.delete(taskId);
|
||||
publish();
|
||||
};
|
||||
}
|
||||
|
||||
/** Run a delete request with the global progress indicator active. */
|
||||
export async function trackDelete<T>(label: string, run: () => Promise<T>): Promise<T> {
|
||||
const end = beginDeleteTask(label);
|
||||
try {
|
||||
return await run();
|
||||
} finally {
|
||||
end();
|
||||
}
|
||||
}
|
||||
|
||||
/** Test helper: drop every in-flight task so cases don't leak into each other. */
|
||||
export function resetDeleteProgress(): void {
|
||||
activeTasks.clear();
|
||||
publish();
|
||||
}
|
||||
@@ -80,7 +80,7 @@ body.overlay-mode #root {
|
||||
}
|
||||
|
||||
/* Tab content entrance animation */
|
||||
@keyframes fade-slide-in {
|
||||
@keyframes fadeSlideIn {
|
||||
from {
|
||||
opacity: 0;
|
||||
transform: translateY(6px);
|
||||
@@ -92,19 +92,5 @@ body.overlay-mode #root {
|
||||
}
|
||||
|
||||
.animate-fade-in {
|
||||
animation: fade-slide-in 0.25s ease-out;
|
||||
}
|
||||
|
||||
/* Indeterminate progress sweep for the global delete indicator */
|
||||
@keyframes indeterminate-sweep {
|
||||
from {
|
||||
transform: translateX(-100%);
|
||||
}
|
||||
to {
|
||||
transform: translateX(400%);
|
||||
}
|
||||
}
|
||||
|
||||
.animate-indeterminate {
|
||||
animation: indeterminate-sweep 1.1s ease-in-out infinite;
|
||||
animation: fadeSlideIn 0.25s ease-out;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user