mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-05-27 00:55:16 -07:00
430373f010
* feat(tokenizer): use Yomitan word classes for subtitle POS filtering - Carry matched headword wordClasses from termsFind into YomitanScanToken - Map recognized Yomitan wordClasses to SubMiner coarse POS before annotation - MeCab enrichment now fills only missing POS fields, preserving existing coarse pos1 - Exclude standalone grammar particles, して helper fragments, and single-kana surfaces from annotations - Respect source-text punctuation gaps when counting N+1 sentence words - Preserve known-word highlight on excluded kanji-containing tokens - Add backlog tasks 304 (N+1 boundary bug) and 305 (wordClasses POS, done) * fix(tokenizer): preserve annotation and enrichment behavior * fix: restore jlpt subtitle underlines * fix: exclude kana-only n+1 targets * fix: refresh overlay on Hyprland fullscreen * fix: address fullscreen and n-plus-one review notes * fix: address CodeRabbit review comments * fix: accept modified digits for multi-line sentence mining * Cancel pending Linux MPV fullscreen overlay refresh bursts - return a cancel handle from the Linux refresh burst scheduler - clear pending refresh bursts when overlays hide or windows close - tighten the burst test polling to wait for the async refresh * fix: suppress N+1 for kana-only candidates and fix minSentenceWords coun - Treat kana-only tokens with surrounding subtitle punctuation (…, ―, etc.) as kana-only so they are not promoted to N+1 targets - Exclude unknown tokens filtered from N+1 targeting from the minSentenceWords count so filtered kana-only unknowns cannot satisfy sentence length threshold - Add regression tests for kana-only candidate suppression and filtered-unknown padding cases * Suppress subtitle annotations for grammar fragments - Hide annotation metadata for auxiliary inflection and ja-nai endings - Preserve lexical `くれる` forms and add regression coverage * Fix kana-only N+1 tokenizer regression test - Use a pure-kana fixture for the subtitle token N+1 case - Update task notes for the latest CodeRabbit follow-up * Fix managed playback exit and tokenizer grammar splits - Ignore background stats daemons during regular app startup - Split standalone grammar endings before applying annotations - Clear helper-span annotations for auxiliary-only tokens * fix: refresh current subtitle after known-word mining * fix: suppress sigh interjection annotations * fix: preserve jlpt underline color after lookup * Replace grammar-ending permutations with shared matcher; preserve word a - Extract `grammar-ending.ts` with `isStandaloneGrammarEndingText` / `isSubtitleGrammarEndingText` pattern matchers - Replace `STANDALONE_GRAMMAR_ENDINGS` set in parser-selection-stage with shared matcher - Replace generated phrase sets in subtitle-annotation-filter with shared matcher - Remove stale duplicate subtitle-exclusion constants and helpers from annotation-stage - Manual clipboard card updates now write only to the sentence audio field, leaving word/expression audio untouched * fix: CI changelog, annotation options threading, and Jellyfin quit - Add `type: fixed` / `area:` frontmatter to `changes/319` to pass `changelog:lint` - Thread `TokenizerAnnotationOptions` through `stripSubtitleAnnotationMetadata` so `sourceText` is honored - Include `jellyfinPlay` in `shouldQuitOnDisconnectWhenOverlayRuntimeInitialized` predicate - Make mouse test `elementFromPoint` stubs coordinate-sensitive - Make Lua test `.tmp` mkdir portable on Windows * Preserve overlay across macOS flaps and mpv playlist changes - keep visible overlays alive during transient macOS tracker loss - reuse the running mpv overlay path on playlist navigation - update regression coverage and changelog fragments * fix: restore stats daemon deferral * fix: keep subtitle prefetch alive after cache hits * Fix JLPT underline color drift and AniList skipped-threshold sync - Replace JLPT `text-decoration` underlines with `border-bottom` so Chromium selection/hover cannot repaint them to another annotation's color - Lock JLPT underline color for combined annotation selectors (known, n+1, frequency) and character hover/selection states - Trigger AniList post-watch check on every mpv time-position update to catch skipped completion thresholds - Fall back to filename-parser season/episode when guessit omits them * fix: address coderabbit feedback * fix: sync AniList after seeked completion * fix: preserve ordinal frequency annotations * fix: preserve known highlighting for filtered tokens * fix: address PR #57 CodeRabbit feedback - Acquire AniList post-watch in-flight lock before async gating to prevent duplicate writes - Isolate manual watched mark result from AniList post-watch callback failures - Report known-word cache clears as mutations during immediate append when state existed - Add regression tests for each fix * fix: stop AniList setup reopening on Linux when keyring token exists - Gate setup success on token persistence: `saveToken` now returns `boolean`; on failure, keeps the setup window open instead of reporting success - Config reload passes `allowSetupPrompt: false` so playback reloads don't re-open the setup window - Add regression test for persistence-failure path * fix: suppress known highlights for subtitle particles * fix: retry transient AniList safeStorage failures * fix: hide overlay focus ring * fix: align Hyprland fullscreen overlays * fix: restore subtitle playback keybindings * fix: align Hyprland overlay windows to mpv and stop pinning them - Force-apply exact Hyprland move/resize/setprop dispatches when bounds are provided - Stop pinning overlay windows; toggle pin off when Hyprland reports pinned=true - Compensate stats overlay outer placement for Electron/Wayland content insets - Make stats overlay window and page opaque so mpv cannot show through transparent insets - Constrain stats app to h-screen with internal scroll so content covers mpv from y=0 - Lock overlay/stats window titles against page-title-updated events - Add regression coverage for placement dispatches, inset compensation, and CSS overlay mode * fix: retain frequency rank for honorific prefix-noun tokens - Add `shouldAllowHonorificPrefixNounFrequency` to exempt お/ご/御 + noun merged tokens from frequency exclusion - Add regression test for `ご機嫌` asserting rank 5484 is preserved after MeCab enrichment and annotation - Close TASK-341 * fix: map openCharacterDictionary session action to --open-character-dict - Add missing Lua CLI dispatch entry for openCharacterDictionary - Add regression test for Alt+Meta+A binding and CLI flag forwarding * fix: keep macOS overlay interactive while mpv remains active - Overlay no longer hides or becomes click-through during tracker refreshes when mpv is the focused window - Preserve already-visible overlay when tracker is temporarily not ready but mpv target signal is active - Add regression tests for active-mpv tracker refresh and transient tracker-not-ready paths * fix: address coderabbit subtitle follow-ups * fix: resolve media detail from sessions when lifetime summary is absent - Change `getMediaDetail` JOIN to LEFT JOIN on `imm_lifetime_media` and fall back to aggregated session metrics when no lifetime row exists - Add filter `AND (lm.video_id IS NOT NULL OR s.session_id IS NOT NULL)` to keep results valid - Add regression test covering the session-visible / media-detail-missing mismatch * fix: address PR-57 CodeRabbit findings and CI failures - use filtered word counts in media detail session token aggregation - cancel fullscreen refresh burst on exit via updateLinuxMpvFullscreenOverlayRefreshBurst - guard Hyprland JSON.parse in try/catch; exclude windowtitle from geometry events - narrow focus suppression from :focus to :focus-visible - apply JLPT lock selectors to word-name-match tokens (N1–N5) * fix: macOS overlay z-order and Yomitan compound token known highlighting - Release always-on-top when tracked mpv loses foreground on macOS - Skip visible overlay blur restacking on macOS to avoid covering unrelated windows - Prefer Yomitan internal parse tokens over fragmented scanner output for known-word decisions - Add regression tests for both behaviors * fix: macOS visible-overlay blur no longer invokes Windows-only blur call - Split win32/darwin branches in handleOverlayWindowBlurred so darwin visible blur returns early without calling onWindowsVisibleOverlayBlur - Add regression test asserting Windows callback stays inactive on macOS visible overlay blur - Close TASK-347
438 lines
12 KiB
TypeScript
438 lines
12 KiB
TypeScript
import test from 'node:test';
|
|
import assert from 'node:assert/strict';
|
|
import * as fs from 'fs';
|
|
import * as os from 'os';
|
|
import * as path from 'path';
|
|
import { AnkiIntegration } from './anki-integration';
|
|
import { FieldGroupingMergeCollaborator } from './anki-integration/field-grouping-merge';
|
|
import { AnkiConnectConfig } from './types';
|
|
|
|
interface IntegrationTestContext {
|
|
integration: AnkiIntegration;
|
|
calls: {
|
|
findNotes: number;
|
|
notesInfo: number;
|
|
};
|
|
stateDir: string;
|
|
}
|
|
|
|
function createIntegrationTestContext(
|
|
options: {
|
|
highlightEnabled?: boolean;
|
|
onFindNotes?: () => Promise<number[]>;
|
|
onNotesInfo?: () => Promise<unknown[]>;
|
|
stateDirPrefix?: string;
|
|
} = {},
|
|
): IntegrationTestContext {
|
|
const calls = {
|
|
findNotes: 0,
|
|
notesInfo: 0,
|
|
};
|
|
|
|
const stateDir = fs.mkdtempSync(
|
|
path.join(os.tmpdir(), options.stateDirPrefix ?? 'subminer-anki-integration-'),
|
|
);
|
|
const knownWordCacheStatePath = path.join(stateDir, 'known-words-cache.json');
|
|
|
|
const client = {
|
|
findNotes: async () => {
|
|
calls.findNotes += 1;
|
|
if (options.onFindNotes) {
|
|
return options.onFindNotes();
|
|
}
|
|
return [] as number[];
|
|
},
|
|
notesInfo: async () => {
|
|
calls.notesInfo += 1;
|
|
if (options.onNotesInfo) {
|
|
return options.onNotesInfo();
|
|
}
|
|
return [] as unknown[];
|
|
},
|
|
} as {
|
|
findNotes: () => Promise<number[]>;
|
|
notesInfo: () => Promise<unknown[]>;
|
|
};
|
|
|
|
const integration = new AnkiIntegration(
|
|
{
|
|
knownWords: {
|
|
highlightEnabled: options.highlightEnabled ?? true,
|
|
},
|
|
},
|
|
{} as never,
|
|
{} as never,
|
|
undefined,
|
|
undefined,
|
|
undefined,
|
|
knownWordCacheStatePath,
|
|
);
|
|
|
|
const integrationWithClient = integration as unknown as {
|
|
client: {
|
|
findNotes: () => Promise<number[]>;
|
|
notesInfo: () => Promise<unknown[]>;
|
|
};
|
|
};
|
|
integrationWithClient.client = client;
|
|
|
|
const privateState = integration as unknown as {
|
|
knownWordsScope: string;
|
|
knownWordsLastRefreshedAtMs: number;
|
|
};
|
|
privateState.knownWordsScope = 'is:note';
|
|
privateState.knownWordsLastRefreshedAtMs = Date.now();
|
|
|
|
return {
|
|
integration,
|
|
calls,
|
|
stateDir,
|
|
};
|
|
}
|
|
|
|
function cleanupIntegrationTestContext(ctx: IntegrationTestContext): void {
|
|
fs.rmSync(ctx.stateDir, { recursive: true, force: true });
|
|
}
|
|
|
|
function resolveFieldName(availableFieldNames: string[], preferredName: string): string | null {
|
|
const exact = availableFieldNames.find((name) => name === preferredName);
|
|
if (exact) return exact;
|
|
|
|
const lower = preferredName.toLowerCase();
|
|
return availableFieldNames.find((name) => name.toLowerCase() === lower) ?? null;
|
|
}
|
|
|
|
function createFieldGroupingMergeCollaborator(options?: {
|
|
config?: Partial<AnkiConnectConfig>;
|
|
currentSubtitleText?: string;
|
|
generatedMedia?: {
|
|
audioField?: string;
|
|
audioValue?: string;
|
|
imageField?: string;
|
|
imageValue?: string;
|
|
miscInfoValue?: string;
|
|
};
|
|
}): FieldGroupingMergeCollaborator {
|
|
const config = {
|
|
fields: {
|
|
sentence: 'Sentence',
|
|
audio: 'ExpressionAudio',
|
|
image: 'Picture',
|
|
...(options?.config?.fields ?? {}),
|
|
},
|
|
...(options?.config ?? {}),
|
|
} as AnkiConnectConfig;
|
|
|
|
return new FieldGroupingMergeCollaborator({
|
|
getConfig: () => config,
|
|
getEffectiveSentenceCardConfig: () => ({
|
|
sentenceField: 'Sentence',
|
|
audioField: 'SentenceAudio',
|
|
}),
|
|
getCurrentSubtitleText: () => options?.currentSubtitleText,
|
|
resolveFieldName,
|
|
resolveNoteFieldName: (noteInfo, preferredName) => {
|
|
if (!preferredName) return null;
|
|
return resolveFieldName(Object.keys(noteInfo.fields), preferredName);
|
|
},
|
|
extractFields: (fields) => {
|
|
const result: Record<string, string> = {};
|
|
for (const [key, value] of Object.entries(fields)) {
|
|
result[key.toLowerCase()] = value.value || '';
|
|
}
|
|
return result;
|
|
},
|
|
processSentence: (mpvSentence) => `${mpvSentence}::processed`,
|
|
generateMediaForMerge: async () => options?.generatedMedia ?? {},
|
|
warnFieldParseOnce: () => undefined,
|
|
});
|
|
}
|
|
|
|
test('AnkiIntegration.refreshKnownWordCache bypasses stale checks', async () => {
|
|
const ctx = createIntegrationTestContext();
|
|
|
|
try {
|
|
await ctx.integration.refreshKnownWordCache();
|
|
|
|
assert.equal(ctx.calls.findNotes, 1);
|
|
assert.equal(ctx.calls.notesInfo, 0);
|
|
} finally {
|
|
cleanupIntegrationTestContext(ctx);
|
|
}
|
|
});
|
|
|
|
test('AnkiIntegration.refreshKnownWordCache skips work when highlight mode is disabled', async () => {
|
|
const ctx = createIntegrationTestContext({
|
|
highlightEnabled: false,
|
|
stateDirPrefix: 'subminer-anki-integration-disabled-',
|
|
});
|
|
|
|
try {
|
|
await ctx.integration.refreshKnownWordCache();
|
|
|
|
assert.equal(ctx.calls.findNotes, 0);
|
|
assert.equal(ctx.calls.notesInfo, 0);
|
|
} finally {
|
|
cleanupIntegrationTestContext(ctx);
|
|
}
|
|
});
|
|
|
|
test('AnkiIntegration notifies when mined note info updates known words', () => {
|
|
const ctx = createIntegrationTestContext({
|
|
stateDirPrefix: 'subminer-anki-integration-known-update-',
|
|
});
|
|
let notifications = 0;
|
|
|
|
try {
|
|
const integrationState = ctx.integration as unknown as {
|
|
config: AnkiConnectConfig;
|
|
appendKnownWordsFromNoteInfo: (noteInfo: {
|
|
noteId: number;
|
|
fields: Record<string, { value: string }>;
|
|
}) => void;
|
|
};
|
|
integrationState.config.deck = 'Mining';
|
|
integrationState.config.knownWords = {
|
|
...integrationState.config.knownWords,
|
|
decks: {
|
|
Mining: ['Word'],
|
|
},
|
|
};
|
|
ctx.integration.setKnownWordCacheUpdatedCallback(() => {
|
|
notifications += 1;
|
|
});
|
|
integrationState.appendKnownWordsFromNoteInfo({
|
|
noteId: 42,
|
|
fields: {
|
|
Word: { value: '食べる' },
|
|
},
|
|
});
|
|
|
|
assert.equal(ctx.integration.isKnownWord('食べる'), true);
|
|
assert.equal(notifications, 1);
|
|
} finally {
|
|
cleanupIntegrationTestContext(ctx);
|
|
}
|
|
});
|
|
|
|
test('AnkiIntegration.refreshKnownWordCache deduplicates concurrent refreshes', async () => {
|
|
let releaseFindNotes: (() => void) | undefined;
|
|
const findNotesPromise = new Promise<void>((resolve) => {
|
|
releaseFindNotes = resolve;
|
|
});
|
|
|
|
const ctx = createIntegrationTestContext({
|
|
onFindNotes: async () => {
|
|
await findNotesPromise;
|
|
return [] as number[];
|
|
},
|
|
stateDirPrefix: 'subminer-anki-integration-concurrent-',
|
|
});
|
|
|
|
const first = ctx.integration.refreshKnownWordCache();
|
|
await Promise.resolve();
|
|
const second = ctx.integration.refreshKnownWordCache();
|
|
|
|
if (releaseFindNotes !== undefined) {
|
|
releaseFindNotes();
|
|
}
|
|
|
|
await Promise.all([first, second]);
|
|
|
|
try {
|
|
assert.equal(ctx.calls.findNotes, 1);
|
|
assert.equal(ctx.calls.notesInfo, 0);
|
|
} finally {
|
|
cleanupIntegrationTestContext(ctx);
|
|
}
|
|
});
|
|
|
|
test('AnkiIntegration resolves merged-away note ids to the kept note id', () => {
|
|
const ctx = createIntegrationTestContext({
|
|
stateDirPrefix: 'subminer-anki-integration-note-redirect-',
|
|
});
|
|
|
|
try {
|
|
const integrationWithInternals = ctx.integration as unknown as {
|
|
rememberMergedNoteIds: (deletedNoteId: number, keptNoteId: number) => void;
|
|
};
|
|
integrationWithInternals.rememberMergedNoteIds(111, 222);
|
|
integrationWithInternals.rememberMergedNoteIds(222, 333);
|
|
|
|
assert.equal(ctx.integration.resolveCurrentNoteId(111), 333);
|
|
assert.equal(ctx.integration.resolveCurrentNoteId(222), 333);
|
|
assert.equal(ctx.integration.resolveCurrentNoteId(333), 333);
|
|
assert.equal(ctx.integration.resolveCurrentNoteId(444), 444);
|
|
} finally {
|
|
cleanupIntegrationTestContext(ctx);
|
|
}
|
|
});
|
|
|
|
test('AnkiIntegration does not allocate proxy server when proxy transport is disabled', () => {
|
|
const integration = new AnkiIntegration(
|
|
{
|
|
enabled: true,
|
|
proxy: {
|
|
enabled: false,
|
|
},
|
|
} as never,
|
|
{} as never,
|
|
{} as never,
|
|
);
|
|
|
|
const privateState = integration as unknown as {
|
|
runtime: {
|
|
proxyServer: unknown | null;
|
|
};
|
|
};
|
|
assert.equal(privateState.runtime.proxyServer, null);
|
|
});
|
|
|
|
test('AnkiIntegration marks partial update notifications as failures in OSD mode', async () => {
|
|
const osdMessages: string[] = [];
|
|
const integration = new AnkiIntegration(
|
|
{
|
|
behavior: {
|
|
notificationType: 'osd',
|
|
},
|
|
},
|
|
{} as never,
|
|
{} as never,
|
|
(text) => {
|
|
osdMessages.push(text);
|
|
},
|
|
);
|
|
|
|
await (
|
|
integration as unknown as {
|
|
showNotification: (
|
|
noteId: number,
|
|
label: string | number,
|
|
errorSuffix?: string,
|
|
) => Promise<void>;
|
|
}
|
|
).showNotification(42, 'taberu', 'image failed');
|
|
|
|
assert.deepEqual(osdMessages, ['x Updated card: taberu (image failed)']);
|
|
});
|
|
|
|
test('FieldGroupingMergeCollaborator synchronizes ExpressionAudio from merged SentenceAudio', async () => {
|
|
const collaborator = createFieldGroupingMergeCollaborator();
|
|
|
|
const merged = await collaborator.computeFieldGroupingMergedFields(
|
|
101,
|
|
202,
|
|
{
|
|
noteId: 101,
|
|
fields: {
|
|
SentenceAudio: { value: '[sound:keep.mp3]' },
|
|
ExpressionAudio: { value: '[sound:stale.mp3]' },
|
|
},
|
|
},
|
|
{
|
|
noteId: 202,
|
|
fields: {
|
|
SentenceAudio: { value: '[sound:new.mp3]' },
|
|
},
|
|
},
|
|
false,
|
|
);
|
|
|
|
assert.equal(
|
|
merged.SentenceAudio,
|
|
'<span data-group-id="101">[sound:keep.mp3]</span><span data-group-id="202">[sound:new.mp3]</span>',
|
|
);
|
|
assert.equal(merged.ExpressionAudio, merged.SentenceAudio);
|
|
});
|
|
|
|
test('FieldGroupingMergeCollaborator uses generated media fallback when source lacks audio', async () => {
|
|
const collaborator = createFieldGroupingMergeCollaborator({
|
|
generatedMedia: {
|
|
audioField: 'SentenceAudio',
|
|
audioValue: '[sound:generated.mp3]',
|
|
},
|
|
});
|
|
|
|
const merged = await collaborator.computeFieldGroupingMergedFields(
|
|
11,
|
|
22,
|
|
{
|
|
noteId: 11,
|
|
fields: {
|
|
SentenceAudio: { value: '' },
|
|
},
|
|
},
|
|
{
|
|
noteId: 22,
|
|
fields: {
|
|
SentenceAudio: { value: '' },
|
|
},
|
|
},
|
|
true,
|
|
);
|
|
|
|
assert.equal(merged.SentenceAudio, '<span data-group-id="22">[sound:generated.mp3]</span>');
|
|
});
|
|
|
|
test('FieldGroupingMergeCollaborator deduplicates identical sentence, audio, and image values when merging into a new duplicate card', async () => {
|
|
const collaborator = createFieldGroupingMergeCollaborator();
|
|
|
|
const merged = await collaborator.computeFieldGroupingMergedFields(
|
|
202,
|
|
101,
|
|
{
|
|
noteId: 202,
|
|
fields: {
|
|
Sentence: { value: 'same sentence' },
|
|
SentenceAudio: { value: '[sound:same.mp3]' },
|
|
Picture: { value: '<img src="same.png">' },
|
|
ExpressionAudio: { value: '[sound:same.mp3]' },
|
|
},
|
|
},
|
|
{
|
|
noteId: 101,
|
|
fields: {
|
|
Sentence: { value: 'same sentence' },
|
|
SentenceAudio: { value: '[sound:same.mp3]' },
|
|
Picture: { value: '<img src="same.png">' },
|
|
},
|
|
},
|
|
false,
|
|
);
|
|
|
|
assert.equal(merged.Sentence, '<span data-group-id="202">same sentence</span>');
|
|
assert.equal(merged.SentenceAudio, '<span data-group-id="202">[sound:same.mp3]</span>');
|
|
assert.equal(merged.Picture, '<img data-group-id="202" src="same.png">');
|
|
assert.equal(merged.ExpressionAudio, merged.SentenceAudio);
|
|
});
|
|
|
|
test('AnkiIntegration.formatMiscInfoPattern avoids leaking Jellyfin api_key query params', () => {
|
|
const integration = new AnkiIntegration(
|
|
{
|
|
metadata: {
|
|
pattern: '[SubMiner] %f (%t)',
|
|
},
|
|
} as never,
|
|
{} as never,
|
|
{
|
|
currentSubText: '',
|
|
currentVideoPath:
|
|
'stream?static=true&api_key=secret-token&MediaSourceId=a762ab23d26d4347e3cacdb83aaae405&AudioStreamIndex=3',
|
|
currentTimePos: 426,
|
|
currentSubStart: 426,
|
|
currentSubEnd: 428,
|
|
currentAudioStreamIndex: 3,
|
|
currentMediaTitle: '[Jellyfin/direct] Bocchi the Rock! - S01E02',
|
|
send: () => true,
|
|
} as unknown as never,
|
|
);
|
|
|
|
const privateApi = integration as unknown as {
|
|
formatMiscInfoPattern: (fallbackFilename: string, startTimeSeconds?: number) => string;
|
|
};
|
|
const result = privateApi.formatMiscInfoPattern('audio_123.mp3', 426);
|
|
|
|
assert.equal(result, '[SubMiner] [Jellyfin/direct] Bocchi the Rock! - S01E02 (00:07:06)');
|
|
assert.equal(result.includes('api_key='), false);
|
|
});
|