diff --git a/backlog/tasks/task-195 - Keep-final-card-mine-OSD-result-from-being-overwritten-by-progress-spinner.md b/backlog/tasks/task-195 - Keep-final-card-mine-OSD-result-from-being-overwritten-by-progress-spinner.md new file mode 100644 index 0000000..e7f6fcf --- /dev/null +++ b/backlog/tasks/task-195 - Keep-final-card-mine-OSD-result-from-being-overwritten-by-progress-spinner.md @@ -0,0 +1,64 @@ +--- +id: TASK-195 +title: Keep final card-mine OSD result from being overwritten by progress spinner +status: Done +assignee: + - Codex +created_date: '2026-03-18 19:40' +updated_date: '2026-03-18 19:49' +labels: + - anki + - ui + - bug +milestone: m-1 +dependencies: [] +references: + - src/anki-integration/ui-feedback.ts + - src/anki-integration.ts + - src/anki-integration/card-creation.ts +priority: medium +ordinal: 105610 +--- + +## Description + + + +When a card mine finishes, the mpv OSD currently tries to show the final status text but the in-flight Anki progress spinner can immediately overwrite it on the next tick. Stop the spinner first, then show a single-line final result with a success/failure marker and the mined-word notification. + + + +## Acceptance Criteria + + + +- [x] #1 Successful mine/update OSD results render after the spinner is stopped and do not get overwritten by a later spinner tick. +- [x] #2 Failure results that replace the spinner show an `x` marker and stay visible on the same OSD line. +- [x] #3 Regression coverage locks the spinner teardown/result-notification ordering. + + +## Implementation Plan + + + +1. Add a focused failing regression test around the Anki UI-feedback spinner/result helper. +2. Add a helper that stops progress before emitting the final OSD result line with `✓`/`x`. +3. Route mine/update result notifications through that helper, then run targeted verification. + + +## Outcome + + + +Added a dedicated Anki UI-feedback result helper that force-clears the in-flight spinner state before emitting the final OSD result line. Successful card-update notifications now render as `✓ Updated card: ...`, and sentence-card creation failures now render as `x Sentence card failed: ...` without a later spinner tick reclaiming the line. + +Verification: + +- `bun test src/anki-integration/ui-feedback.test.ts` +- `bun test src/anki-integration/ui-feedback.test.ts src/anki-integration/note-update-workflow.test.ts src/anki-integration.test.ts src/core/services/mining.test.ts src/main/runtime/mining-actions.test.ts` +- `bun x prettier --check src/anki-integration/ui-feedback.ts src/anki-integration/ui-feedback.test.ts src/anki-integration.ts src/anki-integration/card-creation.ts "backlog/tasks/task-195 - Keep-final-card-mine-OSD-result-from-being-overwritten-by-progress-spinner.md" changes/2026-03-18-mine-osd-spinner-result.md` +- `bun run changelog:lint` +- `bash .agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh --lane core src/anki-integration/ui-feedback.ts src/anki-integration/ui-feedback.test.ts src/anki-integration.ts src/anki-integration/card-creation.ts changes/2026-03-18-mine-osd-spinner-result.md` +- Verifier artifacts: `.tmp/skill-verification/subminer-verify-20260318-194614-uZMrAx/` + + diff --git a/changes/2026-03-18-mine-osd-spinner-result.md b/changes/2026-03-18-mine-osd-spinner-result.md new file mode 100644 index 0000000..3a8e501 --- /dev/null +++ b/changes/2026-03-18-mine-osd-spinner-result.md @@ -0,0 +1,4 @@ +type: fixed +area: anki + +- Fixed card-mine OSD feedback so the final mine result stops the Anki spinner first, then shows a single-line `✓`/`x` status without being overwritten by a later spinner tick. diff --git a/changes/2026-03-18-stats-daemon-decoupling.md b/changes/2026-03-18-stats-daemon-decoupling.md new file mode 100644 index 0000000..5a826d8 --- /dev/null +++ b/changes/2026-03-18-stats-daemon-decoupling.md @@ -0,0 +1,5 @@ +type: fixed +area: stats + +- `subminer stats -b` now runs as a standalone background stats daemon instead of reusing the main SubMiner app process, so the overlay app can still be launched separately for normal video watching. +- Dashboard word mining still works against the background daemon by using a short-lived hidden helper for the Yomitan add-note flow. diff --git a/changes/2026-03-18-subtitle-noise-filtering.md b/changes/2026-03-18-subtitle-noise-filtering.md index eb302c7..ccb65cf 100644 --- a/changes/2026-03-18-subtitle-noise-filtering.md +++ b/changes/2026-03-18-subtitle-noise-filtering.md @@ -1,4 +1,5 @@ type: changed area: overlay -- Excluded interjections and sound-effect tokens from subtitle annotation styling so they no longer inherit misleading lexical highlight treatment while still remaining visible and non-interactive in the subtitle line. +- Excluded interjections and sound-effect tokens from subtitle annotation styling so they no longer inherit misleading lexical highlight treatment while still remaining visible and hoverable as plain subtitle tokens. +- Expanded subtitle annotation noise filtering to also strip annotation metadata from standalone grammar-only helper tokens such as particles, auxiliaries, adnominals, common explanatory endings like `んです` / `のだ`, and merged trailing quote-particle forms like `...って` while keeping them tokenized for hover lookup. diff --git a/docs-site/immersion-tracking.md b/docs-site/immersion-tracking.md index f7519ec..076c396 100644 --- a/docs-site/immersion-tracking.md +++ b/docs-site/immersion-tracking.md @@ -26,7 +26,7 @@ The same immersion data powers the stats dashboard. - In-app overlay: focus the visible overlay, then press the key from `stats.toggleKey` (default: `` ` `` / `Backquote`). - Launcher command: run `subminer stats` to start the local stats server on demand and open the dashboard in your browser. -- Background server: run `subminer stats -b` to start or reuse a dedicated background stats server without keeping the launcher attached, and `subminer stats -s` to stop that background server. +- Background server: run `subminer stats -b` to start or reuse a dedicated background stats daemon without keeping the launcher attached, and `subminer stats -s` to stop that daemon. - Maintenance command: run `subminer stats cleanup` or `subminer stats cleanup -v` to backfill/repair vocabulary metadata (`headword`, `reading`, POS) and purge stale or excluded rows from `imm_words` on demand. - Browser page: open `http://127.0.0.1:5175` directly if the local stats server is already running. @@ -80,8 +80,9 @@ Stats server config lives under `stats`: - `autoStartServer` starts the local stats HTTP server on launch once immersion tracking is active, or reuses the dedicated background stats server when one is already running. - `autoOpenBrowser` controls whether `subminer stats` launches the dashboard URL in your browser after ensuring the server is running. - `subminer stats` forces the dashboard server to start even when `autoStartServer` is `false`. -- `subminer stats -b` starts or reuses the dedicated background stats server and exits after startup acknowledgement. -- `subminer stats -s` stops the dedicated background stats server without closing any browser tabs. +- `subminer stats -b` starts or reuses the dedicated background stats daemon and exits after startup acknowledgement. +- The background stats daemon is separate from the normal SubMiner overlay app, so you can leave it running and still launch SubMiner later to watch or mine from video. +- `subminer stats -s` stops the dedicated background stats daemon without closing any browser tabs. - `subminer stats` fails with an error when `immersionTracking.enabled` is `false`. - `subminer stats cleanup` defaults to vocabulary cleanup, repairs stale `headword`, `reading`, and `part_of_speech` values, attempts best-effort MeCab backfill for legacy rows, and removes rows that still fail vocab filtering. @@ -89,7 +90,7 @@ Stats server config lives under `stats`: The Vocabulary tab's word detail panel shows example lines from your viewing history. Each example line with a valid source file offers three mining buttons: -- **Mine Word** — performs a full Yomitan dictionary lookup for the word (definition, reading, pitch accent, etc.) via the hidden search page, then enriches the card with sentence audio, a screenshot or animated AVIF clip, the highlighted sentence, and metadata extracted from the source video file. Requires Anki and Yomitan dictionaries to be loaded. +- **Mine Word** — performs a full Yomitan dictionary lookup for the word (definition, reading, pitch accent, etc.) via a short-lived hidden helper, then enriches the card with sentence audio, a screenshot or animated AVIF clip, the highlighted sentence, and metadata extracted from the source video file. Requires Anki and Yomitan dictionaries to be loaded. - **Mine Sentence** — creates a sentence card directly with the `IsSentenceCard` flag set (for Lapis/Kiku workflows), along with audio, image, and translation from the secondary subtitle if available. - **Mine Audio** — creates an audio-only card with the `IsAudioCard` flag, attaching only the sentence audio clip. diff --git a/docs-site/public/screenshots/annotations.png b/docs-site/public/screenshots/annotations.png index 37918a5..b289395 100644 Binary files a/docs-site/public/screenshots/annotations.png and b/docs-site/public/screenshots/annotations.png differ diff --git a/docs-site/subtitle-annotations.md b/docs-site/subtitle-annotations.md index 98fdcb2..6dea3b8 100644 --- a/docs-site/subtitle-annotations.md +++ b/docs-site/subtitle-annotations.md @@ -4,6 +4,8 @@ SubMiner annotates subtitle tokens in real time as they appear in the overlay. F All four are opt-in and configured under `subtitleStyle`, `ankiConnect.knownWords`, and `ankiConnect.nPlusOne` in your config. They apply independently — you can enable any combination. +Before any of those layers render, SubMiner strips annotation metadata from tokens that are usually just subtitle glue or annotation noise. Standalone particles, auxiliaries, adnominals, common explanatory endings like `んです` / `のだ`, merged trailing quote-particle forms like `...って`, repeated kana interjections, and similar non-lexical helper tokens remain hoverable in the subtitle text, but they render as plain tokens without known-word, N+1, frequency, JLPT, or name-match annotation styling. + ## N+1 Word Highlighting N+1 highlighting identifies sentences where you know every word except one, making them ideal mining targets. When enabled, SubMiner builds a local cache of your known vocabulary from Anki and highlights tokens accordingly. @@ -80,6 +82,10 @@ When `sourcePath` is omitted, SubMiner searches default install/runtime location Frequency highlighting skips tokens that look like non-lexical noise (kana reduplication, short kana endings like `っ`), even when dictionary ranks exist. ::: +::: info +Frequency, JLPT, and N+1 metadata are only shown for tokens that survive the subtitle-annotation noise filter. Standalone grammar tokens like `は`, `です`, and `この` are intentionally left unannotated even if a dictionary can assign them metadata. +::: + ## JLPT Tagging JLPT tagging adds colored underlines to tokens based on their JLPT level (N1–N5), giving you an at-a-glance sense of difficulty distribution in each subtitle line. diff --git a/launcher/commands/command-modules.test.ts b/launcher/commands/command-modules.test.ts index 15d436c..c3867bb 100644 --- a/launcher/commands/command-modules.test.ts +++ b/launcher/commands/command-modules.test.ts @@ -150,16 +150,17 @@ test('stats command launches attached app command with response path', async () assert.equal(handled, true); assert.deepEqual(forwarded, [ [ - '--stats', + '--stats-daemon-start', '--stats-response-path', '/tmp/subminer-stats-test/response.json', + '--stats-daemon-open-browser', '--log-level', 'debug', ], ]); }); -test('stats background command launches detached app command with response path', async () => { +test('stats background command launches attached daemon control command with response path', async () => { const context = createContext(); context.args.stats = true; (context.args as typeof context.args & { statsBackground?: boolean }).statsBackground = true; @@ -168,11 +169,9 @@ test('stats background command launches detached app command with response path' const handled = await runStatsCommand(context, { createTempDir: () => '/tmp/subminer-stats-test', joinPath: (...parts) => parts.join('/'), - runAppCommandAttached: async () => { - throw new Error('attached path should not run for stats -b'); - }, - launchAppCommandDetached: (_appPath, appArgs) => { + runAppCommandAttached: async (_appPath, appArgs) => { forwarded.push(appArgs); + return 0; }, waitForStatsResponse: async () => ({ ok: true, url: 'http://127.0.0.1:5175' }), removeDir: () => {}, @@ -181,10 +180,9 @@ test('stats background command launches detached app command with response path' assert.equal(handled, true); assert.deepEqual(forwarded, [ [ - '--stats', + '--stats-daemon-start', '--stats-response-path', '/tmp/subminer-stats-test/response.json', - '--stats-background', ], ]); }); @@ -215,7 +213,12 @@ test('stats command returns after startup response even if app process stays run const final = await statsCommand; assert.equal(final, true); assert.deepEqual(forwarded, [ - ['--stats', '--stats-response-path', '/tmp/subminer-stats-test/response.json'], + [ + '--stats-daemon-start', + '--stats-response-path', + '/tmp/subminer-stats-test/response.json', + '--stats-daemon-open-browser', + ], ]); }); @@ -268,7 +271,11 @@ test('stats stop command forwards stop flag to the app', async () => { assert.equal(handled, true); assert.deepEqual(forwarded, [ - ['--stats', '--stats-response-path', '/tmp/subminer-stats-test/response.json', '--stats-stop'], + [ + '--stats-daemon-stop', + '--stats-response-path', + '/tmp/subminer-stats-test/response.json', + ], ]); }); diff --git a/launcher/commands/stats-command.ts b/launcher/commands/stats-command.ts index e21e8f6..95ff576 100644 --- a/launcher/commands/stats-command.ts +++ b/launcher/commands/stats-command.ts @@ -1,7 +1,7 @@ import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; -import { launchAppCommandDetached, runAppCommandAttached } from '../mpv.js'; +import { runAppCommandAttached } from '../mpv.js'; import { sleep } from '../util.js'; import type { LauncherCommandContext } from './context.js'; @@ -20,12 +20,6 @@ type StatsCommandDeps = { logLevel: LauncherCommandContext['args']['logLevel'], label: string, ) => Promise; - launchAppCommandDetached: ( - appPath: string, - appArgs: string[], - logLevel: LauncherCommandContext['args']['logLevel'], - label: string, - ) => void; waitForStatsResponse: (responsePath: string) => Promise; removeDir: (targetPath: string) => void; }; @@ -37,8 +31,6 @@ const defaultDeps: StatsCommandDeps = { joinPath: (...parts) => path.join(...parts), runAppCommandAttached: (appPath, appArgs, logLevel, label) => runAppCommandAttached(appPath, appArgs, logLevel, label), - launchAppCommandDetached: (appPath, appArgs, logLevel, label) => - launchAppCommandDetached(appPath, appArgs, logLevel, label), waitForStatsResponse: async (responsePath) => { const deadline = Date.now() + STATS_STARTUP_RESPONSE_TIMEOUT_MS; while (Date.now() < deadline) { @@ -75,12 +67,15 @@ export async function runStatsCommand( const responsePath = resolvedDeps.joinPath(tempDir, 'response.json'); try { - const forwarded = ['--stats', '--stats-response-path', responsePath]; - if (args.statsBackground) { - forwarded.push('--stats-background'); - } - if (args.statsStop) { - forwarded.push('--stats-stop'); + const forwarded = args.statsCleanup + ? ['--stats', '--stats-response-path', responsePath] + : [ + args.statsStop ? '--stats-daemon-stop' : '--stats-daemon-start', + '--stats-response-path', + responsePath, + ]; + if (!args.statsCleanup && !args.statsBackground && !args.statsStop) { + forwarded.push('--stats-daemon-open-browser'); } if (args.statsCleanup) { forwarded.push('--stats-cleanup'); @@ -94,14 +89,6 @@ export async function runStatsCommand( if (args.logLevel !== 'info') { forwarded.push('--log-level', args.logLevel); } - if (args.statsBackground) { - resolvedDeps.launchAppCommandDetached(appPath, forwarded, args.logLevel, 'stats'); - const startupResult = await resolvedDeps.waitForStatsResponse(responsePath); - if (!startupResult.ok) { - throw new Error(startupResult.error || 'Stats dashboard failed to start.'); - } - return true; - } const attachedExitPromise = resolvedDeps.runAppCommandAttached( appPath, forwarded, diff --git a/launcher/config/cli-parser-builder.ts b/launcher/config/cli-parser-builder.ts index 2fd64ba..81c435b 100644 --- a/launcher/config/cli-parser-builder.ts +++ b/launcher/config/cli-parser-builder.ts @@ -276,6 +276,16 @@ export function parseCliPrograms( if (statsBackground && statsStop) { throw new Error('Stats background and stop flags cannot be combined.'); } + if ( + normalizedAction && + normalizedAction !== 'cleanup' && + normalizedAction !== 'rebuild' && + normalizedAction !== 'backfill' + ) { + throw new Error( + 'Invalid stats action. Valid values are cleanup, rebuild, or backfill.', + ); + } if (normalizedAction && (statsBackground || statsStop)) { throw new Error('Stats background and stop flags cannot be combined with stats actions.'); } diff --git a/launcher/main.test.ts b/launcher/main.test.ts index 83751a0..2f26d4f 100644 --- a/launcher/main.test.ts +++ b/launcher/main.test.ts @@ -536,7 +536,7 @@ exit 0 assert.equal(result.status, 0, `stdout:\n${result.stdout}\nstderr:\n${result.stderr}`); assert.match( fs.readFileSync(capturePath, 'utf8'), - /^--stats\n--stats-response-path\n.+\n--log-level\ndebug\n$/, + /^--stats-daemon-start\n--stats-response-path\n.+\n--stats-daemon-open-browser\n--log-level\ndebug\n$/, ); }); }, diff --git a/launcher/mpv.ts b/launcher/mpv.ts index a487b09..dc82a04 100644 --- a/launcher/mpv.ts +++ b/launcher/mpv.ts @@ -45,6 +45,8 @@ export function parseMpvArgString(input: string): string[] { let inSingleQuote = false; let inDoubleQuote = false; let escaping = false; + const canEscape = (nextChar: string | undefined): boolean => + nextChar === undefined || nextChar === '"' || nextChar === "'" || nextChar === '\\' || /\s/.test(nextChar); for (let i = 0; i < chars.length; i += 1) { const ch = chars[i] || ''; @@ -65,7 +67,11 @@ export function parseMpvArgString(input: string): string[] { if (inDoubleQuote) { if (ch === '\\') { - escaping = true; + if (canEscape(chars[i + 1])) { + escaping = true; + } else { + current += ch; + } continue; } if (ch === '"') { @@ -77,7 +83,11 @@ export function parseMpvArgString(input: string): string[] { } if (ch === '\\') { - escaping = true; + if (canEscape(chars[i + 1])) { + escaping = true; + } else { + current += ch; + } continue; } if (ch === "'") { @@ -857,8 +867,14 @@ export function runAppCommandAttached( proc.once('error', (error) => { reject(error); }); - proc.once('exit', (code) => { - resolve(code ?? 0); + proc.once('exit', (code, signal) => { + if (code !== null) { + resolve(code); + } else if (signal) { + resolve(128); + } else { + resolve(0); + } }); }); } diff --git a/src/anki-integration.ts b/src/anki-integration.ts index 2f867c7..7654407 100644 --- a/src/anki-integration.ts +++ b/src/anki-integration.ts @@ -40,8 +40,10 @@ import { createLogger } from './logger'; import { createUiFeedbackState, beginUpdateProgress, + clearUpdateProgress, endUpdateProgress, showStatusNotification, + showUpdateResult, withUpdateProgress, UiFeedbackState, } from './anki-integration/ui-feedback'; @@ -310,6 +312,8 @@ export class AnkiIntegration { ), }, showOsdNotification: (text: string) => this.showOsdNotification(text), + showUpdateResult: (message: string, success: boolean) => + this.showUpdateResult(message, success), showStatusNotification: (message: string) => this.showStatusNotification(message), showNotification: (noteId, label, errorSuffix) => this.showNotification(noteId, label, errorSuffix), @@ -773,6 +777,12 @@ export class AnkiIntegration { }); } + private clearUpdateProgress(): void { + clearUpdateProgress(this.uiFeedbackState, (timer) => { + clearInterval(timer); + }); + } + private async withUpdateProgress( initialMessage: string, action: () => Promise, @@ -903,7 +913,9 @@ export class AnkiIntegration { const type = this.config.behavior?.notificationType || 'osd'; if (type === 'osd' || type === 'both') { - this.showOsdNotification(message); + this.showUpdateResult(message, true); + } else { + this.clearUpdateProgress(); } if ((type === 'system' || type === 'both') && this.notificationCallback) { @@ -938,6 +950,21 @@ export class AnkiIntegration { } } + private showUpdateResult(message: string, success: boolean): void { + showUpdateResult( + this.uiFeedbackState, + { + clearProgressTimer: (timer) => { + clearInterval(timer); + }, + showOsdNotification: (text) => { + this.showOsdNotification(text); + }, + }, + { message, success }, + ); + } + private mergeFieldValue(existing: string, newValue: string, overwrite: boolean): string { if (overwrite || !existing.trim()) { return newValue; diff --git a/src/anki-integration/card-creation.ts b/src/anki-integration/card-creation.ts index 6820fe1..1bb5670 100644 --- a/src/anki-integration/card-creation.ts +++ b/src/anki-integration/card-creation.ts @@ -75,6 +75,7 @@ interface CardCreationDeps { client: CardCreationClient; mediaGenerator: CardCreationMediaGenerator; showOsdNotification: (text: string) => void; + showUpdateResult: (message: string, success: boolean) => void; showStatusNotification: (message: string) => void; showNotification: (noteId: number, label: string | number, errorSuffix?: string) => Promise; beginUpdateProgress: (initialMessage: string) => void; @@ -261,8 +262,7 @@ export class CardCreationService { if (this.deps.getConfig().media?.generateImage) { try { - const animatedLeadInSeconds = - await this.deps.getAnimatedImageLeadInSeconds(noteInfo); + const animatedLeadInSeconds = await this.deps.getAnimatedImageLeadInSeconds(noteInfo); const imageFilename = this.generateImageFilename(); const imageBuffer = await this.generateImageBuffer( mpvClient.currentVideoPath, @@ -420,8 +420,7 @@ export class CardCreationService { if (this.deps.getConfig().media?.generateImage) { try { - const animatedLeadInSeconds = - await this.deps.getAnimatedImageLeadInSeconds(noteInfo); + const animatedLeadInSeconds = await this.deps.getAnimatedImageLeadInSeconds(noteInfo); const imageFilename = this.generateImageFilename(); const imageBuffer = await this.generateImageBuffer( mpvClient.currentVideoPath, @@ -554,7 +553,7 @@ export class CardCreationService { this.deps.trackLastAddedNoteId?.(noteId); } catch (error) { log.error('Failed to create sentence card:', (error as Error).message); - this.deps.showOsdNotification(`Sentence card failed: ${(error as Error).message}`); + this.deps.showUpdateResult(`Sentence card failed: ${(error as Error).message}`, false); return false; } @@ -651,7 +650,7 @@ export class CardCreationService { }); } catch (error) { log.error('Error creating sentence card:', (error as Error).message); - this.deps.showOsdNotification(`Sentence card failed: ${(error as Error).message}`); + this.deps.showUpdateResult(`Sentence card failed: ${(error as Error).message}`, false); return false; } } diff --git a/src/anki-integration/ui-feedback.test.ts b/src/anki-integration/ui-feedback.test.ts new file mode 100644 index 0000000..b4c2d7e --- /dev/null +++ b/src/anki-integration/ui-feedback.test.ts @@ -0,0 +1,67 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { + beginUpdateProgress, + createUiFeedbackState, + showProgressTick, + showUpdateResult, +} from './ui-feedback'; + +test('showUpdateResult stops spinner before success notification and suppresses stale ticks', () => { + const state = createUiFeedbackState(); + const osdMessages: string[] = []; + + beginUpdateProgress(state, 'Creating sentence card', () => { + showProgressTick(state, (text) => { + osdMessages.push(text); + }); + }); + + showUpdateResult( + state, + { + clearProgressTimer: (timer) => { + clearInterval(timer); + }, + showOsdNotification: (text) => { + osdMessages.push(text); + }, + }, + { success: true, message: 'Updated card: taberu' }, + ); + + showProgressTick(state, (text) => { + osdMessages.push(text); + }); + + assert.deepEqual(osdMessages, ['Creating sentence card |', '✓ Updated card: taberu']); +}); + +test('showUpdateResult renders failed updates with an x marker', () => { + const state = createUiFeedbackState(); + const osdMessages: string[] = []; + + beginUpdateProgress(state, 'Creating sentence card', () => { + showProgressTick(state, (text) => { + osdMessages.push(text); + }); + }); + + showUpdateResult( + state, + { + clearProgressTimer: (timer) => { + clearInterval(timer); + }, + showOsdNotification: (text) => { + osdMessages.push(text); + }, + }, + { success: false, message: 'Sentence card failed: deck missing' }, + ); + + assert.deepEqual(osdMessages, [ + 'Creating sentence card |', + 'x Sentence card failed: deck missing', + ]); +}); diff --git a/src/anki-integration/ui-feedback.ts b/src/anki-integration/ui-feedback.ts index 09844d7..ea43e70 100644 --- a/src/anki-integration/ui-feedback.ts +++ b/src/anki-integration/ui-feedback.ts @@ -7,6 +7,11 @@ export interface UiFeedbackState { progressFrame: number; } +export interface UiFeedbackResult { + success: boolean; + message: string; +} + export interface UiFeedbackNotificationContext { getNotificationType: () => string | undefined; showOsd: (text: string) => void; @@ -66,6 +71,15 @@ export function endUpdateProgress( state.progressDepth = Math.max(0, state.progressDepth - 1); if (state.progressDepth > 0) return; + clearUpdateProgress(state, clearProgressTimer); +} + +export function clearUpdateProgress( + state: UiFeedbackState, + clearProgressTimer: (timer: ReturnType) => void, +): void { + state.progressDepth = 0; + if (state.progressTimer) { clearProgressTimer(state.progressTimer); state.progressTimer = null; @@ -85,6 +99,19 @@ export function showProgressTick( showOsdNotification(`${state.progressMessage} ${frame}`); } +export function showUpdateResult( + state: UiFeedbackState, + options: { + clearProgressTimer: (timer: ReturnType) => void; + showOsdNotification: (text: string) => void; + }, + result: UiFeedbackResult, +): void { + clearUpdateProgress(state, options.clearProgressTimer); + const prefix = result.success ? '✓' : 'x'; + options.showOsdNotification(`${prefix} ${result.message}`); +} + export async function withUpdateProgress( state: UiFeedbackState, options: UiFeedbackOptions, diff --git a/src/core/services/subtitle-ws.test.ts b/src/core/services/subtitle-ws.test.ts index ac4be72..011594f 100644 --- a/src/core/services/subtitle-ws.test.ts +++ b/src/core/services/subtitle-ws.test.ts @@ -130,6 +130,30 @@ test('serializeSubtitleMarkup preserves tooltip attrs and name-match precedence' assert.doesNotMatch(markup, /data-frequency-rank="12"|data-jlpt-level="N5"|word-jlpt-n5/); }); +test('serializeSubtitleMarkup keeps filtered tokens hoverable without annotation attrs', () => { + const payload: SubtitleData = { + text: 'は', + tokens: [ + { + surface: 'は', + reading: 'は', + headword: 'は', + startPos: 0, + endPos: 1, + partOfSpeech: PartOfSpeech.particle, + pos1: '助詞', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + isNameMatch: false, + }, + ], + }; + + const markup = serializeSubtitleMarkup(payload, frequencyOptions); + assert.equal(markup, 'は'); +}); + test('serializeSubtitleWebsocketMessage emits sentence payload', () => { const payload: SubtitleData = { text: '字幕', diff --git a/src/core/services/tokenizer.test.ts b/src/core/services/tokenizer.test.ts index b8def55..c228bb0 100644 --- a/src/core/services/tokenizer.test.ts +++ b/src/core/services/tokenizer.test.ts @@ -1305,7 +1305,7 @@ test('tokenizeSubtitle ignores frequency lookup failures', async () => { assert.equal(result.tokens?.[0]?.frequencyRank, undefined); }); -test('tokenizeSubtitle skips frequency rank when Yomitan token is enriched as particle by mecab pos1', async () => { +test('tokenizeSubtitle keeps standalone particle token hoverable while clearing annotation metadata', async () => { const result = await tokenizeSubtitle( 'は', makeDeps({ @@ -1350,9 +1350,33 @@ test('tokenizeSubtitle skips frequency rank when Yomitan token is enriched as pa }), ); - assert.equal(result.tokens?.length, 1); - assert.equal(result.tokens?.[0]?.pos1, '助詞'); - assert.equal(result.tokens?.[0]?.frequencyRank, undefined); + assert.equal(result.text, 'は'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + reading: token.reading, + headword: token.headword, + pos1: token.pos1, + isKnown: token.isKnown, + isNPlusOneTarget: token.isNPlusOneTarget, + isNameMatch: token.isNameMatch, + jlptLevel: token.jlptLevel, + frequencyRank: token.frequencyRank, + })), + [ + { + surface: 'は', + reading: 'は', + headword: 'は', + pos1: '助詞', + isKnown: false, + isNPlusOneTarget: false, + isNameMatch: false, + jlptLevel: undefined, + frequencyRank: undefined, + }, + ], + ); }); test('tokenizeSubtitle keeps frequency rank when mecab tags classify token as content-bearing', async () => { @@ -1460,7 +1484,7 @@ test('tokenizeSubtitle skips JLPT level for excluded demonstratives', async () = assert.equal(result.tokens?.[0]?.jlptLevel, undefined); }); -test('tokenizeSubtitle excludes repeated kana interjections from annotation payloads entirely', async () => { +test('tokenizeSubtitle keeps repeated kana interjections tokenized while clearing annotation metadata', async () => { const result = await tokenizeSubtitle( 'ああ', makeDeps({ @@ -1491,7 +1515,29 @@ test('tokenizeSubtitle excludes repeated kana interjections from annotation payl }), ); - assert.deepEqual(result, { text: 'ああ', tokens: null }); + assert.equal(result.text, 'ああ'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + headword: token.headword, + reading: token.reading, + jlptLevel: token.jlptLevel, + frequencyRank: token.frequencyRank, + isKnown: token.isKnown, + isNPlusOneTarget: token.isNPlusOneTarget, + })), + [ + { + surface: 'ああ', + headword: 'ああ', + reading: 'ああ', + jlptLevel: undefined, + frequencyRank: undefined, + isKnown: false, + isNPlusOneTarget: false, + }, + ], + ); }); test('tokenizeSubtitle assigns JLPT level to Yomitan tokens', async () => { @@ -2578,7 +2624,15 @@ test('tokenizeSubtitle keeps correct MeCab pos1 enrichment when Yomitan offsets const gaToken = result.tokens?.find((token) => token.surface === 'が'); const desuToken = result.tokens?.find((token) => token.surface === 'です'); assert.equal(gaToken?.pos1, '助詞'); + assert.equal(gaToken?.isKnown, false); + assert.equal(gaToken?.isNPlusOneTarget, false); + assert.equal(gaToken?.jlptLevel, undefined); + assert.equal(gaToken?.frequencyRank, undefined); assert.equal(desuToken?.pos1, '助動詞'); + assert.equal(desuToken?.isKnown, false); + assert.equal(desuToken?.isNPlusOneTarget, false); + assert.equal(desuToken?.jlptLevel, undefined); + assert.equal(desuToken?.frequencyRank, undefined); assert.equal(targets.length, 1); assert.equal(targets[0]?.surface, '仮面'); }); @@ -3056,7 +3110,7 @@ test('tokenizeSubtitle excludes default non-independent pos2 from N+1 and freque assert.equal(result.tokens?.[0]?.isNPlusOneTarget, false); }); -test('tokenizeSubtitle excludes mecab-tagged interjections from annotation payloads entirely', async () => { +test('tokenizeSubtitle keeps mecab-tagged interjections tokenized while clearing annotation metadata', async () => { const result = await tokenizeSubtitle( 'ぐはっ', makeDepsFromYomitanTokens([{ surface: 'ぐはっ', reading: 'ぐはっ', headword: 'ぐはっ' }], { @@ -3080,10 +3134,34 @@ test('tokenizeSubtitle excludes mecab-tagged interjections from annotation paylo }), ); - assert.deepEqual(result, { text: 'ぐはっ', tokens: null }); + assert.equal(result.text, 'ぐはっ'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + headword: token.headword, + reading: token.reading, + pos1: token.pos1, + jlptLevel: token.jlptLevel, + frequencyRank: token.frequencyRank, + isKnown: token.isKnown, + isNPlusOneTarget: token.isNPlusOneTarget, + })), + [ + { + surface: 'ぐはっ', + headword: 'ぐはっ', + reading: 'ぐはっ', + pos1: '感動詞', + jlptLevel: undefined, + frequencyRank: undefined, + isKnown: false, + isNPlusOneTarget: false, + }, + ], + ); }); -test('tokenizeSubtitle keeps visible text while excluding interjections from mixed annotation payloads', async () => { +test('tokenizeSubtitle keeps excluded interjections hoverable while clearing only their annotation metadata', async () => { const result = await tokenizeSubtitle( 'ぐはっ 猫', makeDeps({ @@ -3147,8 +3225,261 @@ test('tokenizeSubtitle keeps visible text while excluding interjections from mix result.tokens?.map((token) => ({ surface: token.surface, headword: token.headword, + frequencyRank: token.frequencyRank, + jlptLevel: token.jlptLevel, })), - [{ surface: '猫', headword: '猫' }], + [ + { surface: 'ぐはっ', headword: 'ぐはっ', frequencyRank: undefined, jlptLevel: undefined }, + { surface: '猫', headword: '猫', frequencyRank: 11, jlptLevel: 'N5' }, + ], + ); +}); + +test('tokenizeSubtitle keeps explanatory ending variants hoverable while clearing only their annotation metadata', async () => { + const result = await tokenizeSubtitle( + '猫んです', + makeDepsFromYomitanTokens( + [ + { surface: '猫', reading: 'ねこ', headword: '猫' }, + { surface: 'んです', reading: 'んです', headword: 'ん' }, + ], + { + getFrequencyDictionaryEnabled: () => true, + getFrequencyRank: (text) => (text === '猫' ? 11 : 500), + getJlptLevel: (text) => (text === '猫' ? 'N5' : null), + tokenizeWithMecab: async () => [ + { + headword: '猫', + surface: '猫', + reading: 'ネコ', + startPos: 0, + endPos: 1, + partOfSpeech: PartOfSpeech.noun, + pos1: '名詞', + pos2: '一般', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'ん', + surface: 'ん', + reading: 'ン', + startPos: 1, + endPos: 2, + partOfSpeech: PartOfSpeech.other, + pos1: '名詞', + pos2: '非自立', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'です', + surface: 'です', + reading: 'デス', + startPos: 2, + endPos: 4, + partOfSpeech: PartOfSpeech.bound_auxiliary, + pos1: '助動詞', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + }, + ], + }, + ), + ); + + assert.equal(result.text, '猫んです'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + headword: token.headword, + jlptLevel: token.jlptLevel, + frequencyRank: token.frequencyRank, + })), + [ + { surface: '猫', headword: '猫', jlptLevel: 'N5', frequencyRank: 11 }, + { surface: 'んです', headword: 'ん', jlptLevel: undefined, frequencyRank: undefined }, + ], + ); +}); + +test('tokenizeSubtitle keeps standalone grammar-only tokens hoverable while clearing only their annotation metadata', async () => { + const result = await tokenizeSubtitle( + '私はこの猫です', + makeDeps({ + getFrequencyDictionaryEnabled: () => true, + getFrequencyRank: (text) => (text === '私' ? 50 : text === '猫' ? 11 : 500), + getJlptLevel: (text) => (text === '私' ? 'N5' : text === '猫' ? 'N5' : null), + getYomitanExt: () => ({ id: 'dummy-ext' }) as any, + getYomitanParserWindow: () => + ({ + isDestroyed: () => false, + webContents: { + executeJavaScript: async (script: string) => { + if (script.includes('getTermFrequencies')) { + return []; + } + + return [ + { + source: 'scanning-parser', + index: 0, + content: [ + [{ text: '私', reading: 'わたし', headwords: [[{ term: '私' }]] }], + [{ text: 'は', reading: 'は', headwords: [[{ term: 'は' }]] }], + [{ text: 'この', reading: 'この', headwords: [[{ term: 'この' }]] }], + [{ text: '猫', reading: 'ねこ', headwords: [[{ term: '猫' }]] }], + [{ text: 'です', reading: 'です', headwords: [[{ term: 'です' }]] }], + ], + }, + ]; + }, + }, + }) as unknown as Electron.BrowserWindow, + tokenizeWithMecab: async () => [ + { + headword: '私', + surface: '私', + reading: 'ワタシ', + startPos: 0, + endPos: 1, + partOfSpeech: PartOfSpeech.noun, + pos1: '名詞', + pos2: '代名詞', + isMerged: true, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'は', + surface: 'は', + reading: 'ハ', + startPos: 1, + endPos: 2, + partOfSpeech: PartOfSpeech.particle, + pos1: '助詞', + pos2: '係助詞', + isMerged: true, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'この', + surface: 'この', + reading: 'コノ', + startPos: 2, + endPos: 4, + partOfSpeech: PartOfSpeech.other, + pos1: '連体詞', + isMerged: true, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: '猫', + surface: '猫', + reading: 'ネコ', + startPos: 4, + endPos: 5, + partOfSpeech: PartOfSpeech.noun, + pos1: '名詞', + pos2: '一般', + isMerged: true, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'です', + surface: 'です', + reading: 'デス', + startPos: 5, + endPos: 7, + partOfSpeech: PartOfSpeech.bound_auxiliary, + pos1: '助動詞', + isMerged: true, + isKnown: false, + isNPlusOneTarget: false, + }, + ], + }), + ); + + assert.equal(result.text, '私はこの猫です'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + headword: token.headword, + frequencyRank: token.frequencyRank, + jlptLevel: token.jlptLevel, + })), + [ + { surface: '私', headword: '私', frequencyRank: 50, jlptLevel: 'N5' }, + { surface: 'は', headword: 'は', frequencyRank: undefined, jlptLevel: undefined }, + { surface: 'この', headword: 'この', frequencyRank: undefined, jlptLevel: undefined }, + { surface: '猫', headword: '猫', frequencyRank: 11, jlptLevel: 'N5' }, + { surface: 'です', headword: 'です', frequencyRank: undefined, jlptLevel: undefined }, + ], + ); +}); + +test('tokenizeSubtitle keeps trailing quote-particle merged tokens hoverable while clearing only their annotation metadata', async () => { + const result = await tokenizeSubtitle( + 'どうしてもって', + makeDepsFromYomitanTokens([{ surface: 'どうしてもって', reading: 'どうしてもって', headword: 'どうしても' }], { + getFrequencyDictionaryEnabled: () => true, + getFrequencyRank: (text) => (text === 'どうしても' ? 123 : null), + getJlptLevel: (text) => (text === 'どうしても' ? 'N3' : null), + tokenizeWithMecab: async () => [ + { + headword: 'どうしても', + surface: 'どうしても', + reading: 'ドウシテモ', + startPos: 0, + endPos: 5, + partOfSpeech: PartOfSpeech.other, + pos1: '副詞', + pos2: '一般', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + }, + { + headword: 'って', + surface: 'って', + reading: 'ッテ', + startPos: 5, + endPos: 7, + partOfSpeech: PartOfSpeech.particle, + pos1: '助詞', + pos2: '格助詞', + isMerged: false, + isKnown: false, + isNPlusOneTarget: false, + }, + ], + getMinSentenceWordsForNPlusOne: () => 1, + }), + ); + + assert.equal(result.text, 'どうしてもって'); + assert.deepEqual( + result.tokens?.map((token) => ({ + surface: token.surface, + headword: token.headword, + jlptLevel: token.jlptLevel, + frequencyRank: token.frequencyRank, + })), + [ + { + surface: 'どうしてもって', + headword: 'どうしても', + jlptLevel: undefined, + frequencyRank: undefined, + }, + ], ); }); diff --git a/src/core/services/tokenizer.ts b/src/core/services/tokenizer.ts index 2e476c3..240a97a 100644 --- a/src/core/services/tokenizer.ts +++ b/src/core/services/tokenizer.ts @@ -178,7 +178,7 @@ async function applyAnnotationStage( ); } -async function filterSubtitleAnnotationTokens(tokens: MergedToken[]): Promise { +async function stripSubtitleAnnotationMetadata(tokens: MergedToken[]): Promise { if (tokens.length === 0) { return tokens; } @@ -188,9 +188,7 @@ async function filterSubtitleAnnotationTokens(tokens: MergedToken[]): Promise !annotationStage.shouldExcludeTokenFromSubtitleAnnotations(token), - ); + return tokens.map((token) => annotationStage.stripSubtitleAnnotationMetadata(token)); } export function createTokenizerDepsRuntime( @@ -721,12 +719,12 @@ export async function tokenizeSubtitle( const yomitanTokens = await parseWithYomitanInternalParser(tokenizeText, deps, annotationOptions); if (yomitanTokens && yomitanTokens.length > 0) { - const filteredTokens = await filterSubtitleAnnotationTokens( + const annotatedTokens = await stripSubtitleAnnotationMetadata( await applyAnnotationStage(yomitanTokens, deps, annotationOptions), ); return { text: displayText, - tokens: filteredTokens.length > 0 ? filteredTokens : null, + tokens: annotatedTokens.length > 0 ? annotatedTokens : null, }; } diff --git a/src/core/services/tokenizer/annotation-stage.test.ts b/src/core/services/tokenizer/annotation-stage.test.ts index dd9fdf8..9af0661 100644 --- a/src/core/services/tokenizer/annotation-stage.test.ts +++ b/src/core/services/tokenizer/annotation-stage.test.ts @@ -1,7 +1,12 @@ import assert from 'node:assert/strict'; import test from 'node:test'; import { MergedToken, PartOfSpeech } from '../../../types'; -import { annotateTokens, AnnotationStageDeps } from './annotation-stage'; +import { + annotateTokens, + AnnotationStageDeps, + shouldExcludeTokenFromSubtitleAnnotations, + stripSubtitleAnnotationMetadata, +} from './annotation-stage'; function makeToken(overrides: Partial = {}): MergedToken { return { @@ -150,6 +155,170 @@ test('annotateTokens handles JLPT disabled and eligibility exclusion paths', () assert.equal(excludedLookupCalls, 0); }); +test('shouldExcludeTokenFromSubtitleAnnotations excludes explanatory ending variants', () => { + const tokens = [ + makeToken({ + surface: 'んです', + headword: 'ん', + reading: 'ンデス', + pos1: '名詞|助動詞', + pos2: '非自立', + }), + makeToken({ + surface: 'のだ', + headword: 'の', + reading: 'ノダ', + pos1: '名詞|助動詞', + pos2: '非自立', + }), + makeToken({ + surface: 'んだ', + headword: 'ん', + reading: 'ンダ', + pos1: '名詞|助動詞', + pos2: '非自立', + }), + makeToken({ + surface: 'のです', + headword: 'の', + reading: 'ノデス', + pos1: '名詞|助動詞', + pos2: '非自立', + }), + makeToken({ + surface: 'なんです', + headword: 'だ', + reading: 'ナンデス', + pos1: '助動詞|名詞|助動詞', + pos2: '|非自立', + }), + makeToken({ + surface: 'んでした', + headword: 'ん', + reading: 'ンデシタ', + pos1: '助動詞|助動詞|助動詞', + }), + makeToken({ + surface: 'のでは', + headword: 'の', + reading: 'ノデハ', + pos1: '助詞|接続詞', + }), + ]; + + for (const token of tokens) { + assert.equal(shouldExcludeTokenFromSubtitleAnnotations(token), true, token.surface); + } +}); + +test('shouldExcludeTokenFromSubtitleAnnotations keeps lexical tokens outside explanatory ending family', () => { + const token = makeToken({ + surface: '問題', + headword: '問題', + reading: 'モンダイ', + partOfSpeech: PartOfSpeech.noun, + pos1: '名詞', + pos2: '一般', + }); + + assert.equal(shouldExcludeTokenFromSubtitleAnnotations(token), false); +}); + +test('shouldExcludeTokenFromSubtitleAnnotations excludes standalone particles auxiliaries and adnominals', () => { + const tokens = [ + makeToken({ + surface: 'は', + headword: 'は', + reading: 'ハ', + partOfSpeech: PartOfSpeech.particle, + pos1: '助詞', + }), + makeToken({ + surface: 'です', + headword: 'です', + reading: 'デス', + partOfSpeech: PartOfSpeech.bound_auxiliary, + pos1: '助動詞', + }), + makeToken({ + surface: 'この', + headword: 'この', + reading: 'コノ', + partOfSpeech: PartOfSpeech.other, + pos1: '連体詞', + }), + ]; + + for (const token of tokens) { + assert.equal(shouldExcludeTokenFromSubtitleAnnotations(token), true, token.surface); + } +}); + +test('shouldExcludeTokenFromSubtitleAnnotations keeps mixed content tokens with trailing helpers', () => { + const token = makeToken({ + surface: '行きます', + headword: '行く', + reading: 'イキマス', + partOfSpeech: PartOfSpeech.verb, + pos1: '動詞|助動詞', + pos2: '自立', + }); + + assert.equal(shouldExcludeTokenFromSubtitleAnnotations(token), false); +}); + +test('shouldExcludeTokenFromSubtitleAnnotations excludes merged lexical tokens with trailing quote particles', () => { + const token = makeToken({ + surface: 'どうしてもって', + headword: 'どうしても', + reading: 'ドウシテモッテ', + partOfSpeech: PartOfSpeech.other, + pos1: '副詞|助詞', + pos2: '一般|格助詞', + }); + + assert.equal(shouldExcludeTokenFromSubtitleAnnotations(token), true); +}); + +test('stripSubtitleAnnotationMetadata keeps token hover data while clearing annotation fields', () => { + const token = makeToken({ + surface: 'は', + headword: 'は', + reading: 'ハ', + partOfSpeech: PartOfSpeech.particle, + pos1: '助詞', + isKnown: true, + isNPlusOneTarget: true, + isNameMatch: true, + jlptLevel: 'N5', + frequencyRank: 12, + }); + + assert.deepEqual(stripSubtitleAnnotationMetadata(token), { + ...token, + isKnown: false, + isNPlusOneTarget: false, + isNameMatch: false, + jlptLevel: undefined, + frequencyRank: undefined, + }); +}); + +test('stripSubtitleAnnotationMetadata leaves content tokens unchanged', () => { + const token = makeToken({ + surface: '猫', + headword: '猫', + reading: 'ネコ', + partOfSpeech: PartOfSpeech.noun, + pos1: '名詞', + isKnown: true, + jlptLevel: 'N5', + frequencyRank: 42, + }); + + assert.strictEqual(stripSubtitleAnnotationMetadata(token), token); +}); + test('annotateTokens prioritizes name matches over n+1, frequency, and JLPT when enabled', () => { let jlptLookupCalls = 0; const tokens = [ diff --git a/src/core/services/tokenizer/annotation-stage.ts b/src/core/services/tokenizer/annotation-stage.ts index 026c9c7..5617b8f 100644 --- a/src/core/services/tokenizer/annotation-stage.ts +++ b/src/core/services/tokenizer/annotation-stage.ts @@ -25,6 +25,45 @@ const SUBTITLE_ANNOTATION_EXCLUDED_TERMS = new Set([ 'ふう', 'ほう', ]); +const SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_PREFIXES = ['ん', 'の', 'なん', 'なの']; +const SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_CORES = [ + 'だ', + 'です', + 'でした', + 'だった', + 'では', + 'じゃ', + 'でしょう', + 'だろう', +] as const; +const SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_TRAILING_PARTICLES = [ + '', + 'か', + 'ね', + 'よ', + 'な', + 'よね', + 'かな', + 'かね', +] as const; +const SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDINGS = new Set( + SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_PREFIXES.flatMap((prefix) => + SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_CORES.flatMap((core) => + SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDING_TRAILING_PARTICLES.map( + (particle) => `${prefix}${core}${particle}`, + ), + ), + ), +); +const SUBTITLE_ANNOTATION_EXCLUDED_TRAILING_PARTICLE_SUFFIXES = new Set([ + 'って', + 'ってよ', + 'ってね', + 'ってな', + 'ってさ', + 'ってか', + 'ってば', +]); const jlptLevelLookupCaches = new WeakMap< (text: string) => JlptLevel | null, @@ -60,6 +99,7 @@ function normalizePos1Tag(pos1: string | undefined): string { } const SUBTITLE_ANNOTATION_EXCLUDED_POS1 = new Set(['感動詞']); +const SUBTITLE_ANNOTATION_GRAMMAR_ONLY_POS1 = new Set(['助詞', '助動詞', '連体詞']); function splitNormalizedTagParts(normalizedTag: string): string[] { if (!normalizedTag) { @@ -84,7 +124,36 @@ function isExcludedByTagSet(normalizedTag: string, exclusions: ReadonlySet SUBTITLE_ANNOTATION_EXCLUDED_POS1.has(part)); + if (parts.some((part) => SUBTITLE_ANNOTATION_EXCLUDED_POS1.has(part))) { + return true; + } + + return parts.length > 0 && parts.every((part) => SUBTITLE_ANNOTATION_GRAMMAR_ONLY_POS1.has(part)); +} + +function isExcludedTrailingParticleMergedToken(token: MergedToken): boolean { + const normalizedSurface = normalizeJlptTextForExclusion(token.surface); + const normalizedHeadword = normalizeJlptTextForExclusion(token.headword); + if (!normalizedSurface || !normalizedHeadword || !normalizedSurface.startsWith(normalizedHeadword)) { + return false; + } + + const suffix = normalizedSurface.slice(normalizedHeadword.length); + if (!SUBTITLE_ANNOTATION_EXCLUDED_TRAILING_PARTICLE_SUFFIXES.has(suffix)) { + return false; + } + + const pos1Parts = splitNormalizedTagParts(normalizePos1Tag(token.pos1)); + if (pos1Parts.length < 2) { + return false; + } + + const [leadingPos1, ...trailingPos1] = pos1Parts; + if (!leadingPos1 || SUBTITLE_ANNOTATION_GRAMMAR_ONLY_POS1.has(leadingPos1)) { + return false; + } + + return trailingPos1.length > 0 && trailingPos1.every((part) => part === '助詞'); } function resolvePos1Exclusions(options: AnnotationStageOptions): ReadonlySet { @@ -520,12 +589,7 @@ function isJlptEligibleToken(token: MergedToken): boolean { } function isExcludedFromSubtitleAnnotationsByTerm(token: MergedToken): boolean { - const candidates = [ - resolveJlptLookupText(token), - token.surface, - token.headword, - token.reading, - ].filter( + const candidates = [token.surface, token.reading, resolveJlptLookupText(token)].filter( (candidate): candidate is string => typeof candidate === 'string' && candidate.length > 0, ); @@ -542,7 +606,9 @@ function isExcludedFromSubtitleAnnotationsByTerm(token: MergedToken): boolean { if ( SUBTITLE_ANNOTATION_EXCLUDED_TERMS.has(trimmedCandidate) || - SUBTITLE_ANNOTATION_EXCLUDED_TERMS.has(normalizedCandidate) + SUBTITLE_ANNOTATION_EXCLUDED_TERMS.has(normalizedCandidate) || + SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDINGS.has(trimmedCandidate) || + SUBTITLE_ANNOTATION_EXCLUDED_EXPLANATORY_ENDINGS.has(normalizedCandidate) ) { return true; } @@ -565,9 +631,28 @@ export function shouldExcludeTokenFromSubtitleAnnotations(token: MergedToken): b return true; } + if (isExcludedTrailingParticleMergedToken(token)) { + return true; + } + return isExcludedFromSubtitleAnnotationsByTerm(token); } +export function stripSubtitleAnnotationMetadata(token: MergedToken): MergedToken { + if (!shouldExcludeTokenFromSubtitleAnnotations(token)) { + return token; + } + + return { + ...token, + isKnown: false, + isNPlusOneTarget: false, + isNameMatch: false, + jlptLevel: undefined, + frequencyRank: undefined, + }; +} + function computeTokenKnownStatus( token: MergedToken, isKnownWord: (text: string) => boolean, diff --git a/src/main-entry-runtime.test.ts b/src/main-entry-runtime.test.ts index f07110a..dd1f7a2 100644 --- a/src/main-entry-runtime.test.ts +++ b/src/main-entry-runtime.test.ts @@ -11,6 +11,7 @@ import { shouldDetachBackgroundLaunch, shouldHandleHelpOnlyAtEntry, shouldHandleLaunchMpvAtEntry, + shouldHandleStatsDaemonCommandAtEntry, } from './main-entry-runtime'; test('normalizeStartupArgv defaults no-arg startup to --start --background on non-Windows', () => { @@ -71,6 +72,25 @@ test('launch-mpv entry helpers detect and normalize targets', () => { ]); }); +test('stats-daemon entry helper detects internal daemon commands', () => { + assert.equal( + shouldHandleStatsDaemonCommandAtEntry(['SubMiner.AppImage', '--stats-daemon-start'], {}), + true, + ); + assert.equal( + shouldHandleStatsDaemonCommandAtEntry(['SubMiner.AppImage', '--stats-daemon-stop'], {}), + true, + ); + assert.equal( + shouldHandleStatsDaemonCommandAtEntry( + ['SubMiner.AppImage', '--stats-daemon-start'], + { ELECTRON_RUN_AS_NODE: '1' }, + ), + false, + ); + assert.equal(shouldHandleStatsDaemonCommandAtEntry(['SubMiner.AppImage', '--start'], {}), false); +}); + test('sanitizeStartupEnv suppresses warnings and lsfg layer', () => { const env = sanitizeStartupEnv({ VK_INSTANCE_LAYERS: 'foo:lsfg-vk:bar', diff --git a/src/main-entry-runtime.ts b/src/main-entry-runtime.ts index 90a04ce..b6405fa 100644 --- a/src/main-entry-runtime.ts +++ b/src/main-entry-runtime.ts @@ -112,6 +112,14 @@ export function shouldHandleLaunchMpvAtEntry(argv: string[], env: NodeJS.Process return parseCliArgs(argv).launchMpv; } +export function shouldHandleStatsDaemonCommandAtEntry( + argv: string[], + env: NodeJS.ProcessEnv, +): boolean { + if (env.ELECTRON_RUN_AS_NODE === '1') return false; + return argv.includes('--stats-daemon-start') || argv.includes('--stats-daemon-stop'); +} + export function normalizeLaunchMpvTargets(argv: string[]): string[] { return parseCliArgs(argv).launchMpvTargets; } diff --git a/src/main-entry.ts b/src/main-entry.ts index eb337f0..5012813 100644 --- a/src/main-entry.ts +++ b/src/main-entry.ts @@ -12,9 +12,11 @@ import { shouldDetachBackgroundLaunch, shouldHandleHelpOnlyAtEntry, shouldHandleLaunchMpvAtEntry, + shouldHandleStatsDaemonCommandAtEntry, } from './main-entry-runtime'; import { requestSingleInstanceLockEarly } from './main/early-single-instance'; import { createWindowsMpvLaunchDeps, launchWindowsMpv } from './main/runtime/windows-mpv-launch'; +import { runStatsDaemonControlFromProcess } from './stats-daemon-entry'; const DEFAULT_TEXTHOOKER_PORT = 5174; @@ -69,6 +71,11 @@ if (shouldHandleLaunchMpvAtEntry(process.argv, process.env)) { ); app.exit(result.ok ? 0 : 1); }); +} else if (shouldHandleStatsDaemonCommandAtEntry(process.argv, process.env)) { + void app.whenReady().then(async () => { + const exitCode = await runStatsDaemonControlFromProcess(app.getPath('userData')); + app.exit(exitCode); + }); } else { const gotSingleInstanceLock = requestSingleInstanceLockEarly(app); if (!gotSingleInstanceLock) { diff --git a/src/renderer/subtitle-render.test.ts b/src/renderer/subtitle-render.test.ts index aa22c9f..64b8309 100644 --- a/src/renderer/subtitle-render.test.ts +++ b/src/renderer/subtitle-render.test.ts @@ -682,7 +682,7 @@ test('renderSubtitle preserves unsupported punctuation while keeping it non-inte } }); -test('renderSubtitle keeps excluded interjection text visible while only rendering remaining tokens as interactive', () => { +test('renderSubtitle keeps excluded interjection tokens hoverable while rendering them without annotation styling', () => { const restoreDocument = installFakeDocument(); try { @@ -718,13 +718,19 @@ test('renderSubtitle keeps excluded interjection text visible while only renderi renderer.renderSubtitle({ text: 'ぐはっ 猫', - tokens: [createToken({ surface: '猫', headword: '猫', reading: 'ねこ' })], + tokens: [ + createToken({ surface: 'ぐはっ', headword: 'ぐはっ', reading: 'ぐはっ' }), + createToken({ surface: '猫', headword: '猫', reading: 'ねこ' }), + ], }); assert.equal(subtitleRoot.textContent, 'ぐはっ 猫'); assert.deepEqual( collectWordNodes(subtitleRoot).map((node) => [node.textContent, node.dataset.tokenIndex]), - [['猫', '0']], + [ + ['ぐはっ', '0'], + ['猫', '1'], + ], ); } finally { restoreDocument(); diff --git a/src/stats-daemon-control.test.ts b/src/stats-daemon-control.test.ts new file mode 100644 index 0000000..acacc29 --- /dev/null +++ b/src/stats-daemon-control.test.ts @@ -0,0 +1,158 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createRunStatsDaemonControlHandler } from './stats-daemon-control'; + +test('stats daemon control reuses live daemon and writes launcher response', async () => { + const calls: string[] = []; + const responses: Array<{ path: string; payload: { ok: boolean; url?: string; error?: string } }> = + []; + const handler = createRunStatsDaemonControlHandler({ + statePath: '/tmp/stats-daemon.json', + readState: () => ({ pid: 4242, port: 5175, startedAtMs: 1 }), + removeState: () => { + calls.push('removeState'); + }, + isProcessAlive: (pid) => { + calls.push(`isProcessAlive:${pid}`); + return true; + }, + resolveUrl: (state) => `http://127.0.0.1:${state.port}`, + spawnDaemon: async () => { + calls.push('spawnDaemon'); + return 1; + }, + waitForDaemonResponse: async () => { + calls.push('waitForDaemonResponse'); + return { ok: true, url: 'http://127.0.0.1:5175' }; + }, + openExternal: async (url) => { + calls.push(`openExternal:${url}`); + }, + writeResponse: (responsePath, payload) => { + responses.push({ path: responsePath, payload }); + }, + killProcess: () => { + calls.push('killProcess'); + }, + sleep: async () => {}, + }); + + const exitCode = await handler({ + action: 'start', + responsePath: '/tmp/response.json', + openBrowser: true, + daemonScriptPath: '/tmp/stats-daemon-runner.js', + userDataPath: '/tmp/SubMiner', + }); + + assert.equal(exitCode, 0); + assert.deepEqual(calls, ['isProcessAlive:4242', 'openExternal:http://127.0.0.1:5175']); + assert.deepEqual(responses, [ + { + path: '/tmp/response.json', + payload: { ok: true, url: 'http://127.0.0.1:5175' }, + }, + ]); +}); + +test('stats daemon control clears stale state, starts daemon, and waits for response', async () => { + const calls: string[] = []; + const handler = createRunStatsDaemonControlHandler({ + statePath: '/tmp/stats-daemon.json', + readState: () => ({ pid: 4242, port: 5175, startedAtMs: 1 }), + removeState: () => { + calls.push('removeState'); + }, + isProcessAlive: (pid) => { + calls.push(`isProcessAlive:${pid}`); + return false; + }, + resolveUrl: (state) => `http://127.0.0.1:${state.port}`, + spawnDaemon: async (options) => { + calls.push(`spawnDaemon:${options.scriptPath}:${options.responsePath}:${options.userDataPath}`); + return 999; + }, + waitForDaemonResponse: async (responsePath) => { + calls.push(`waitForDaemonResponse:${responsePath}`); + return { ok: true, url: 'http://127.0.0.1:5175' }; + }, + openExternal: async (url) => { + calls.push(`openExternal:${url}`); + }, + writeResponse: () => { + calls.push('writeResponse'); + }, + killProcess: () => { + calls.push('killProcess'); + }, + sleep: async () => {}, + }); + + const exitCode = await handler({ + action: 'start', + responsePath: '/tmp/response.json', + openBrowser: false, + daemonScriptPath: '/tmp/stats-daemon-runner.js', + userDataPath: '/tmp/SubMiner', + }); + + assert.equal(exitCode, 0); + assert.deepEqual(calls, [ + 'isProcessAlive:4242', + 'removeState', + 'spawnDaemon:/tmp/stats-daemon-runner.js:/tmp/response.json:/tmp/SubMiner', + 'waitForDaemonResponse:/tmp/response.json', + ]); +}); + +test('stats daemon control stops live daemon and treats stale state as success', async () => { + const responses: Array<{ path: string; payload: { ok: boolean; url?: string; error?: string } }> = + []; + const calls: string[] = []; + let aliveChecks = 0; + const handler = createRunStatsDaemonControlHandler({ + statePath: '/tmp/stats-daemon.json', + readState: () => ({ pid: 4242, port: 5175, startedAtMs: 1 }), + removeState: () => { + calls.push('removeState'); + }, + isProcessAlive: (pid) => { + aliveChecks += 1; + calls.push(`isProcessAlive:${pid}:${aliveChecks}`); + return aliveChecks === 1; + }, + resolveUrl: (state) => `http://127.0.0.1:${state.port}`, + spawnDaemon: async () => 1, + waitForDaemonResponse: async () => ({ ok: true, url: 'http://127.0.0.1:5175' }), + openExternal: async () => {}, + writeResponse: (responsePath, payload) => { + responses.push({ path: responsePath, payload }); + }, + killProcess: (pid, signal) => { + calls.push(`killProcess:${pid}:${signal}`); + }, + sleep: async () => {}, + }); + + const exitCode = await handler({ + action: 'stop', + responsePath: '/tmp/response.json', + openBrowser: false, + daemonScriptPath: '/tmp/stats-daemon-runner.js', + userDataPath: '/tmp/SubMiner', + }); + + assert.equal(exitCode, 0); + assert.deepEqual(calls, [ + 'isProcessAlive:4242:1', + 'killProcess:4242:SIGTERM', + 'isProcessAlive:4242:2', + 'removeState', + ]); + assert.deepEqual(responses, [ + { + path: '/tmp/response.json', + payload: { ok: true }, + }, + ]); +}); diff --git a/src/stats-daemon-control.ts b/src/stats-daemon-control.ts new file mode 100644 index 0000000..a51e6a6 --- /dev/null +++ b/src/stats-daemon-control.ts @@ -0,0 +1,102 @@ +import type { BackgroundStatsServerState } from './main/runtime/stats-daemon'; +import type { StatsCliCommandResponse } from './main/runtime/stats-cli-command'; + +export type StatsDaemonControlAction = 'start' | 'stop'; + +export type StatsDaemonControlArgs = { + action: StatsDaemonControlAction; + responsePath?: string; + openBrowser: boolean; + daemonScriptPath: string; + userDataPath: string; +}; + +type SpawnStatsDaemonOptions = { + scriptPath: string; + responsePath?: string; + userDataPath: string; +}; + +export function createRunStatsDaemonControlHandler(deps: { + statePath: string; + readState: () => BackgroundStatsServerState | null; + removeState: () => void; + isProcessAlive: (pid: number) => boolean; + resolveUrl: (state: Pick) => string; + spawnDaemon: (options: SpawnStatsDaemonOptions) => Promise | number; + waitForDaemonResponse: (responsePath: string) => Promise; + openExternal: (url: string) => Promise; + writeResponse: (responsePath: string, payload: StatsCliCommandResponse) => void; + killProcess: (pid: number, signal: NodeJS.Signals) => void; + sleep: (ms: number) => Promise; +}) { + const writeResponseSafe = ( + responsePath: string | undefined, + payload: StatsCliCommandResponse, + ): void => { + if (!responsePath) return; + deps.writeResponse(responsePath, payload); + }; + + return async (args: StatsDaemonControlArgs): Promise => { + if (args.action === 'start') { + const state = deps.readState(); + if (state) { + if (deps.isProcessAlive(state.pid)) { + const url = deps.resolveUrl(state); + writeResponseSafe(args.responsePath, { ok: true, url }); + if (args.openBrowser) { + await deps.openExternal(url); + } + return 0; + } + deps.removeState(); + } + + if (!args.responsePath) { + throw new Error('Missing --stats-response-path for stats daemon start.'); + } + + await deps.spawnDaemon({ + scriptPath: args.daemonScriptPath, + responsePath: args.responsePath, + userDataPath: args.userDataPath, + }); + const response = await deps.waitForDaemonResponse(args.responsePath); + if (response.ok && args.openBrowser && response.url) { + await deps.openExternal(response.url); + } + return response.ok ? 0 : 1; + } + + const state = deps.readState(); + if (!state) { + deps.removeState(); + writeResponseSafe(args.responsePath, { ok: true }); + return 0; + } + + if (!deps.isProcessAlive(state.pid)) { + deps.removeState(); + writeResponseSafe(args.responsePath, { ok: true }); + return 0; + } + + deps.killProcess(state.pid, 'SIGTERM'); + const deadline = Date.now() + 2_000; + while (Date.now() < deadline) { + if (!deps.isProcessAlive(state.pid)) { + deps.removeState(); + writeResponseSafe(args.responsePath, { ok: true }); + return 0; + } + await deps.sleep(50); + } + + writeResponseSafe(args.responsePath, { + ok: false, + error: 'Timed out stopping background stats server.', + }); + return 1; + }; +} diff --git a/src/stats-daemon-entry.ts b/src/stats-daemon-entry.ts new file mode 100644 index 0000000..0099f9e --- /dev/null +++ b/src/stats-daemon-entry.ts @@ -0,0 +1,135 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { spawn } from 'node:child_process'; +import { shell } from 'electron'; +import { sanitizeStartupEnv } from './main-entry-runtime'; +import { + isBackgroundStatsServerProcessAlive, + readBackgroundStatsServerState, + removeBackgroundStatsServerState, + resolveBackgroundStatsServerUrl, +} from './main/runtime/stats-daemon'; +import { + createRunStatsDaemonControlHandler, + type StatsDaemonControlArgs, +} from './stats-daemon-control'; +import { + type StatsCliCommandResponse, + writeStatsCliCommandResponse, +} from './main/runtime/stats-cli-command'; + +const STATS_DAEMON_RESPONSE_TIMEOUT_MS = 12_000; + +function readFlagValue(argv: string[], flag: string): string | undefined { + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + if (!arg) continue; + if (arg === flag) { + const value = argv[i + 1]; + if (value && !value.startsWith('--')) { + return value; + } + return undefined; + } + if (arg.startsWith(`${flag}=`)) { + return arg.split('=', 2)[1]; + } + } + return undefined; +} + +function hasFlag(argv: string[], flag: string): boolean { + return argv.includes(flag); +} + +function parseControlArgs(argv: string[], userDataPath: string): StatsDaemonControlArgs { + return { + action: hasFlag(argv, '--stats-daemon-stop') ? 'stop' : 'start', + responsePath: readFlagValue(argv, '--stats-response-path'), + openBrowser: hasFlag(argv, '--stats-daemon-open-browser'), + daemonScriptPath: path.join(__dirname, 'stats-daemon-runner.js'), + userDataPath, + }; +} + +async function waitForDaemonResponse(responsePath: string): Promise { + const deadline = Date.now() + STATS_DAEMON_RESPONSE_TIMEOUT_MS; + while (Date.now() < deadline) { + try { + if (fs.existsSync(responsePath)) { + return JSON.parse(fs.readFileSync(responsePath, 'utf8')) as StatsCliCommandResponse; + } + } catch { + // retry until timeout + } + await new Promise((resolve) => setTimeout(resolve, 100)); + } + + return { + ok: false, + error: 'Timed out waiting for stats daemon startup response.', + }; +} + +export async function runStatsDaemonControlFromProcess(userDataPath: string): Promise { + const args = parseControlArgs(process.argv, userDataPath); + const statePath = path.join(userDataPath, 'stats-daemon.json'); + + const writeFailureResponse = (message: string): void => { + if (args.responsePath) { + try { + writeStatsCliCommandResponse(args.responsePath, { + ok: false, + error: message, + }); + } catch { + // ignore secondary response-write failures + } + } + }; + + const handler = createRunStatsDaemonControlHandler({ + statePath, + readState: () => readBackgroundStatsServerState(statePath), + removeState: () => { + removeBackgroundStatsServerState(statePath); + }, + isProcessAlive: (pid) => isBackgroundStatsServerProcessAlive(pid), + resolveUrl: (state) => resolveBackgroundStatsServerUrl(state), + spawnDaemon: async (options) => { + const childArgs = [options.scriptPath, '--stats-user-data-path', options.userDataPath]; + if (options.responsePath) { + childArgs.push('--stats-response-path', options.responsePath); + } + const logLevel = readFlagValue(process.argv, '--log-level'); + if (logLevel) { + childArgs.push('--log-level', logLevel); + } + const child = spawn(process.execPath, childArgs, { + detached: true, + stdio: 'ignore', + env: { + ...sanitizeStartupEnv(process.env), + ELECTRON_RUN_AS_NODE: '1', + }, + }); + child.unref(); + return child.pid ?? 0; + }, + waitForDaemonResponse, + openExternal: async (url) => shell.openExternal(url), + writeResponse: writeStatsCliCommandResponse, + killProcess: (pid, signal) => { + process.kill(pid, signal); + }, + sleep: async (ms) => new Promise((resolve) => setTimeout(resolve, ms)), + }); + + try { + return await handler(args); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + writeFailureResponse(message); + return 1; + } +} diff --git a/src/stats-daemon-runner.ts b/src/stats-daemon-runner.ts new file mode 100644 index 0000000..2210b01 --- /dev/null +++ b/src/stats-daemon-runner.ts @@ -0,0 +1,225 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { spawn } from 'node:child_process'; +import { ConfigService } from './config/service'; +import { createLogger, setLogLevel } from './logger'; +import { ImmersionTrackerService } from './core/services/immersion-tracker-service'; +import { createCoverArtFetcher } from './core/services/anilist/cover-art-fetcher'; +import { createAnilistRateLimiter } from './core/services/anilist/rate-limiter'; +import { startStatsServer } from './core/services/stats-server'; +import { + removeBackgroundStatsServerState, + writeBackgroundStatsServerState, +} from './main/runtime/stats-daemon'; +import { writeStatsCliCommandResponse } from './main/runtime/stats-cli-command'; +import { createInvokeStatsWordHelperHandler, type StatsWordHelperResponse } from './stats-word-helper-client'; + +const logger = createLogger('stats-daemon'); +const STATS_WORD_HELPER_RESPONSE_TIMEOUT_MS = 20_000; + +function readFlagValue(argv: string[], flag: string): string | undefined { + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + if (!arg) continue; + if (arg === flag) { + const value = argv[i + 1]; + if (value && !value.startsWith('--')) { + return value; + } + return undefined; + } + if (arg.startsWith(`${flag}=`)) { + return arg.split('=', 2)[1]; + } + } + return undefined; +} + +async function waitForWordHelperResponse(responsePath: string): Promise { + const deadline = Date.now() + STATS_WORD_HELPER_RESPONSE_TIMEOUT_MS; + while (Date.now() < deadline) { + try { + if (fs.existsSync(responsePath)) { + return JSON.parse(fs.readFileSync(responsePath, 'utf8')) as StatsWordHelperResponse; + } + } catch { + // retry until timeout + } + await new Promise((resolve) => setTimeout(resolve, 100)); + } + + return { + ok: false, + error: 'Timed out waiting for stats word helper response.', + }; +} + +const invokeStatsWordHelper = createInvokeStatsWordHelperHandler({ + createTempDir: (prefix) => fs.mkdtempSync(path.join(os.tmpdir(), prefix)), + joinPath: (...parts) => path.join(...parts), + spawnHelper: async (options) => { + const childArgs = [ + options.scriptPath, + '--stats-word-helper-response-path', + options.responsePath, + '--stats-word-helper-user-data-path', + options.userDataPath, + '--stats-word-helper-word', + options.word, + ]; + const logLevel = readFlagValue(process.argv, '--log-level'); + if (logLevel) { + childArgs.push('--log-level', logLevel); + } + const child = spawn(process.execPath, childArgs, { + stdio: 'ignore', + env: { + ...process.env, + ELECTRON_RUN_AS_NODE: undefined, + }, + }); + return await new Promise((resolve) => { + child.once('exit', (code) => resolve(code ?? 1)); + child.once('error', () => resolve(1)); + }); + }, + waitForResponse: waitForWordHelperResponse, + removeDir: (targetPath) => { + fs.rmSync(targetPath, { recursive: true, force: true }); + }, +}); + +const userDataPath = readFlagValue(process.argv, '--stats-user-data-path')?.trim(); +const responsePath = readFlagValue(process.argv, '--stats-response-path')?.trim(); +const logLevel = readFlagValue(process.argv, '--log-level'); + +if (logLevel) { + setLogLevel(logLevel, 'cli'); +} + +if (!userDataPath) { + if (responsePath) { + writeStatsCliCommandResponse(responsePath, { + ok: false, + error: 'Missing --stats-user-data-path for stats daemon runner.', + }); + } + process.exit(1); +} + +const daemonUserDataPath = userDataPath; + +const statePath = path.join(userDataPath, 'stats-daemon.json'); +const knownWordCachePath = path.join(userDataPath, 'known-words-cache.json'); +const statsDistPath = path.join(__dirname, '..', 'stats', 'dist'); +const wordHelperScriptPath = path.join(__dirname, 'stats-word-helper.js'); + +let tracker: ImmersionTrackerService | null = null; +let statsServer: ReturnType | null = null; + +function writeFailureResponse(message: string): void { + if (!responsePath) return; + writeStatsCliCommandResponse(responsePath, { ok: false, error: message }); +} + +function clearOwnedState(): void { + const rawState = (() => { + try { + return JSON.parse(fs.readFileSync(statePath, 'utf8')) as { pid?: number }; + } catch { + return null; + } + })(); + if (rawState?.pid === process.pid) { + removeBackgroundStatsServerState(statePath); + } +} + +function shutdown(code = 0): void { + try { + statsServer?.close(); + } catch { + // ignore + } + statsServer = null; + try { + tracker?.destroy(); + } catch { + // ignore + } + tracker = null; + clearOwnedState(); + process.exit(code); +} + +process.on('SIGINT', () => shutdown(0)); +process.on('SIGTERM', () => shutdown(0)); + +async function main(): Promise { + try { + const configService = new ConfigService(daemonUserDataPath); + const config = configService.getConfig(); + if (config.immersionTracking?.enabled === false) { + throw new Error('Immersion tracking is disabled in config.'); + } + + const configuredDbPath = config.immersionTracking?.dbPath?.trim() || ''; + tracker = new ImmersionTrackerService({ + dbPath: configuredDbPath || path.join(daemonUserDataPath, 'immersion.sqlite'), + policy: { + batchSize: config.immersionTracking.batchSize, + flushIntervalMs: config.immersionTracking.flushIntervalMs, + queueCap: config.immersionTracking.queueCap, + payloadCapBytes: config.immersionTracking.payloadCapBytes, + maintenanceIntervalMs: config.immersionTracking.maintenanceIntervalMs, + retention: { + eventsDays: config.immersionTracking.retention.eventsDays, + telemetryDays: config.immersionTracking.retention.telemetryDays, + sessionsDays: config.immersionTracking.retention.sessionsDays, + dailyRollupsDays: config.immersionTracking.retention.dailyRollupsDays, + monthlyRollupsDays: config.immersionTracking.retention.monthlyRollupsDays, + vacuumIntervalDays: config.immersionTracking.retention.vacuumIntervalDays, + }, + }, + }); + tracker.setCoverArtFetcher( + createCoverArtFetcher(createAnilistRateLimiter(), createLogger('stats-daemon:cover-art')), + ); + + statsServer = startStatsServer({ + port: config.stats.serverPort, + staticDir: statsDistPath, + tracker, + knownWordCachePath, + ankiConnectConfig: config.ankiConnect, + addYomitanNote: async (word: string) => + await invokeStatsWordHelper({ + helperScriptPath: wordHelperScriptPath, + userDataPath: daemonUserDataPath, + word, + }), + }); + + writeBackgroundStatsServerState(statePath, { + pid: process.pid, + port: config.stats.serverPort, + startedAtMs: Date.now(), + }); + + if (responsePath) { + writeStatsCliCommandResponse(responsePath, { + ok: true, + url: `http://127.0.0.1:${config.stats.serverPort}`, + }); + } + logger.info(`Background stats daemon listening on http://127.0.0.1:${config.stats.serverPort}`); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + logger.error('Failed to start stats daemon', message); + writeFailureResponse(message); + shutdown(1); + } +} + +void main(); diff --git a/src/stats-word-helper-client.test.ts b/src/stats-word-helper-client.test.ts new file mode 100644 index 0000000..6cb0e48 --- /dev/null +++ b/src/stats-word-helper-client.test.ts @@ -0,0 +1,57 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createInvokeStatsWordHelperHandler } from './stats-word-helper-client'; + +test('word helper client returns note id when helper responds before exit', async () => { + const calls: string[] = []; + const handler = createInvokeStatsWordHelperHandler({ + createTempDir: () => '/tmp/stats-word-helper', + joinPath: (...parts) => parts.join('/'), + spawnHelper: async (options) => { + calls.push( + `spawnHelper:${options.scriptPath}:${options.responsePath}:${options.userDataPath}:${options.word}`, + ); + return new Promise((resolve) => setTimeout(() => resolve(0), 20)); + }, + waitForResponse: async (responsePath) => { + calls.push(`waitForResponse:${responsePath}`); + return { ok: true, noteId: 123 }; + }, + removeDir: (targetPath) => { + calls.push(`removeDir:${targetPath}`); + }, + }); + + const noteId = await handler({ + helperScriptPath: '/tmp/stats-word-helper.js', + userDataPath: '/tmp/SubMiner', + word: '猫', + }); + + assert.equal(noteId, 123); + assert.deepEqual(calls, [ + 'spawnHelper:/tmp/stats-word-helper.js:/tmp/stats-word-helper/response.json:/tmp/SubMiner:猫', + 'waitForResponse:/tmp/stats-word-helper/response.json', + 'removeDir:/tmp/stats-word-helper', + ]); +}); + +test('word helper client throws helper response errors', async () => { + const handler = createInvokeStatsWordHelperHandler({ + createTempDir: () => '/tmp/stats-word-helper', + joinPath: (...parts) => parts.join('/'), + spawnHelper: async () => 0, + waitForResponse: async () => ({ ok: false, error: 'helper failed' }), + removeDir: () => {}, + }); + + await assert.rejects( + async () => + handler({ + helperScriptPath: '/tmp/stats-word-helper.js', + userDataPath: '/tmp/SubMiner', + word: '猫', + }), + /helper failed/, + ); +}); diff --git a/src/stats-word-helper-client.ts b/src/stats-word-helper-client.ts new file mode 100644 index 0000000..ab71425 --- /dev/null +++ b/src/stats-word-helper-client.ts @@ -0,0 +1,62 @@ +export type StatsWordHelperResponse = { + ok: boolean; + noteId?: number; + error?: string; +}; + +export function createInvokeStatsWordHelperHandler(deps: { + createTempDir: (prefix: string) => string; + joinPath: (...parts: string[]) => string; + spawnHelper: (options: { + scriptPath: string; + responsePath: string; + userDataPath: string; + word: string; + }) => Promise; + waitForResponse: (responsePath: string) => Promise; + removeDir: (targetPath: string) => void; +}) { + return async (options: { + helperScriptPath: string; + userDataPath: string; + word: string; + }): Promise => { + const tempDir = deps.createTempDir('subminer-stats-word-helper-'); + const responsePath = deps.joinPath(tempDir, 'response.json'); + + try { + const helperExitPromise = deps.spawnHelper({ + scriptPath: options.helperScriptPath, + responsePath, + userDataPath: options.userDataPath, + word: options.word, + }); + + const startupResult = await Promise.race([ + deps.waitForResponse(responsePath).then((response) => ({ kind: 'response' as const, response })), + helperExitPromise.then((status) => ({ kind: 'exit' as const, status })), + ]); + + let response: StatsWordHelperResponse; + if (startupResult.kind === 'response') { + response = startupResult.response; + } else { + if (startupResult.status !== 0) { + throw new Error(`Stats word helper exited before response (status ${startupResult.status}).`); + } + response = await deps.waitForResponse(responsePath); + } + + const exitStatus = await helperExitPromise; + if (exitStatus !== 0) { + throw new Error(`Stats word helper exited with status ${exitStatus}.`); + } + if (!response.ok || typeof response.noteId !== 'number') { + throw new Error(response.error || 'Stats word helper failed.'); + } + return response.noteId; + } finally { + deps.removeDir(tempDir); + } + }; +} diff --git a/src/stats-word-helper.ts b/src/stats-word-helper.ts new file mode 100644 index 0000000..d1e9a5b --- /dev/null +++ b/src/stats-word-helper.ts @@ -0,0 +1,193 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { app, protocol } from 'electron'; +import type { BrowserWindow, Extension, Session } from 'electron'; +import { ConfigService } from './config/service'; +import { createLogger, setLogLevel } from './logger'; +import { loadYomitanExtension } from './core/services/yomitan-extension-loader'; +import { + addYomitanNoteViaSearch, + syncYomitanDefaultAnkiServer, +} from './core/services/tokenizer/yomitan-parser-runtime'; +import type { StatsWordHelperResponse } from './stats-word-helper-client'; +import { clearYomitanExtensionRuntimeState } from './core/services/yomitan-extension-runtime-state'; + +protocol.registerSchemesAsPrivileged([ + { + scheme: 'chrome-extension', + privileges: { + standard: true, + secure: true, + supportFetchAPI: true, + corsEnabled: true, + bypassCSP: true, + }, + }, +]); + +const logger = createLogger('stats-word-helper'); + +function readFlagValue(argv: string[], flag: string): string | undefined { + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + if (!arg) continue; + if (arg === flag) { + const value = argv[i + 1]; + if (value && !value.startsWith('--')) { + return value; + } + return undefined; + } + if (arg.startsWith(`${flag}=`)) { + return arg.split('=', 2)[1]; + } + } + return undefined; +} + +function writeResponse(responsePath: string | undefined, payload: StatsWordHelperResponse): void { + if (!responsePath) return; + fs.mkdirSync(path.dirname(responsePath), { recursive: true }); + fs.writeFileSync(responsePath, JSON.stringify(payload, null, 2), 'utf8'); +} + +const responsePath = readFlagValue(process.argv, '--stats-word-helper-response-path')?.trim(); +const userDataPath = readFlagValue(process.argv, '--stats-word-helper-user-data-path')?.trim(); +const word = readFlagValue(process.argv, '--stats-word-helper-word'); +const logLevel = readFlagValue(process.argv, '--log-level'); + +if (logLevel) { + setLogLevel(logLevel, 'cli'); +} + +if (!userDataPath || !word) { + writeResponse(responsePath, { + ok: false, + error: 'Missing stats word helper arguments.', + }); + app.exit(1); +} + +app.setName('SubMiner'); +app.setPath('userData', userDataPath!); + +let yomitanExt: Extension | null = null; +let yomitanSession: Session | null = null; +let yomitanParserWindow: BrowserWindow | null = null; +let yomitanParserReadyPromise: Promise | null = null; +let yomitanParserInitPromise: Promise | null = null; + +function cleanup(): void { + clearYomitanExtensionRuntimeState({ + getYomitanParserWindow: () => yomitanParserWindow, + setYomitanParserWindow: () => { + yomitanParserWindow = null; + }, + setYomitanParserReadyPromise: () => { + yomitanParserReadyPromise = null; + }, + setYomitanParserInitPromise: () => { + yomitanParserInitPromise = null; + }, + setYomitanExtension: () => { + yomitanExt = null; + }, + setYomitanSession: () => { + yomitanSession = null; + }, + }); +} + +async function main(): Promise { + try { + const configService = new ConfigService(userDataPath!); + const config = configService.getConfig(); + const extension = await loadYomitanExtension({ + userDataPath: userDataPath!, + getYomitanParserWindow: () => yomitanParserWindow, + setYomitanParserWindow: (window) => { + yomitanParserWindow = window; + }, + setYomitanParserReadyPromise: (promise) => { + yomitanParserReadyPromise = promise; + }, + setYomitanParserInitPromise: (promise) => { + yomitanParserInitPromise = promise; + }, + setYomitanExtension: (extensionValue) => { + yomitanExt = extensionValue; + }, + setYomitanSession: (sessionValue) => { + yomitanSession = sessionValue; + }, + }); + if (!extension) { + throw new Error('Yomitan extension failed to load.'); + } + + await syncYomitanDefaultAnkiServer( + config.ankiConnect?.url || 'http://127.0.0.1:8765', + { + getYomitanExt: () => yomitanExt, + getYomitanSession: () => yomitanSession, + getYomitanParserWindow: () => yomitanParserWindow, + setYomitanParserWindow: (window) => { + yomitanParserWindow = window; + }, + getYomitanParserReadyPromise: () => yomitanParserReadyPromise, + setYomitanParserReadyPromise: (promise) => { + yomitanParserReadyPromise = promise; + }, + getYomitanParserInitPromise: () => yomitanParserInitPromise, + setYomitanParserInitPromise: (promise) => { + yomitanParserInitPromise = promise; + }, + }, + logger, + { forceOverride: true }, + ); + + const noteId = await addYomitanNoteViaSearch( + word!, + { + getYomitanExt: () => yomitanExt, + getYomitanSession: () => yomitanSession, + getYomitanParserWindow: () => yomitanParserWindow, + setYomitanParserWindow: (window) => { + yomitanParserWindow = window; + }, + getYomitanParserReadyPromise: () => yomitanParserReadyPromise, + setYomitanParserReadyPromise: (promise) => { + yomitanParserReadyPromise = promise; + }, + getYomitanParserInitPromise: () => yomitanParserInitPromise, + setYomitanParserInitPromise: (promise) => { + yomitanParserInitPromise = promise; + }, + }, + logger, + ); + + if (typeof noteId !== 'number') { + throw new Error('Yomitan failed to create note.'); + } + + writeResponse(responsePath, { + ok: true, + noteId, + }); + cleanup(); + app.exit(0); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + logger.error('Stats word helper failed', message); + writeResponse(responsePath, { + ok: false, + error: message, + }); + cleanup(); + app.exit(1); + } +} + +void app.whenReady().then(() => main());