mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-10-20 17:01:24 -07:00
feat(subtitles): generate Jellyfin subtitles and widen halfwidth kana
- Whisper subtitle generation now works for Jellyfin playback. It reads the item's original file through toJellyfinOriginalStreamUrl and writes to a reused cache folder. For transcoded playback it uses Jellyfin's audio index - generateJapaneseSubtitles accepts http(s) streams when an output path is set; subprocess errors redact URL tokens - Add widenHalfwidthKana/narrowFullwidthKana and widen subtitle text at ingest (mpv sub-text, parsed cues, live primary text), so halfwidth and fullwidth duplicates merge into one cue - Stats queries widen older stored lines, sentence search matches katakana in either width, and stats mining widens sentence/word - Update Jellyfin, subtitle generation, and architecture docs; add changelog fragments
This commit is contained in:
30 files changed
+580
-51
No files matched your search
@@ -0,0 +1,4 @@
|
||||
type: added
|
||||
area: subtitles
|
||||
|
||||
- Japanese subtitle generation now works while playing a Jellyfin episode. SubMiner reads the original file from the server, saves the subtitles to a cache folder, and loads them into mpv. Direct-play and transcoded playback are both supported.
|
||||
@@ -0,0 +1,5 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Subtitle lines recorded before halfwidth katakana widening now show fullwidth katakana in the stats dashboard (word/kanji occurrences, sentence search), and cards mined from stats history get the fullwidth sentence and word.
|
||||
- Sentence search matches katakana in either width, so searching アンタ also finds older lines stored as アンタ.
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: overlay
|
||||
|
||||
- Halfwidth katakana in subtitles (common in broadcast-derived Japanese subs, e.g. アンタ, ダキニ) now displays as normal fullwidth katakana, so lines no longer mix squashed and normal-width text. Mined sentences and stats get the fullwidth form too.
|
||||
@@ -23,6 +23,8 @@ SubMiner starts mpv if it is not already running. Pause, seek, stop, and track c
|
||||
|
||||
SubMiner selects a Japanese subtitle track automatically and resets mpv's subtitle delay to zero. It direct-plays files when it can and asks Jellyfin to transcode the rest.
|
||||
|
||||
If an episode has no Japanese subtitles, you can [generate them with Whisper](/subtitle-generation#generating-from-the-overlay) while it plays.
|
||||
|
||||
On Windows, casting finds mpv through `mpv.executablePath`, then `SUBMINER_MPV_PATH`, then `PATH`. An invalid `mpv.executablePath` stops mpv from starting.
|
||||
|
||||
## Playing from the terminal
|
||||
|
||||
@@ -16,7 +16,7 @@ The modal's **Local tools** section lists anything missing. After you install a
|
||||
|
||||
## Generating from the overlay
|
||||
|
||||
1. Open a local video in mpv and select its Japanese audio track, or play a YouTube video.
|
||||
1. Open a local video in mpv and select its Japanese audio track, or play a Jellyfin or YouTube video.
|
||||
2. Press `Ctrl+Shift+G`. If the subtitle sidebar is empty, its **Generate Japanese subtitles** button opens the same modal.
|
||||
3. Pick a model and download it if needed.
|
||||
4. Optionally check **Focus on spoken dialogue** (see below).
|
||||
@@ -26,6 +26,8 @@ The modal shows progress. **Cancel** stops the job. Closing the modal lets the j
|
||||
|
||||
SubMiner saves `<video>.ja.generated.srt` next to the video and adds a number if that name is taken. If the same file is still playing, it loads the subtitles and resets the subtitle delay.
|
||||
|
||||
For a Jellyfin video, SubMiner reads the original file from your server and saves the subtitles to `~/.cache/subminer/generated-subs/`, since a stream has no folder of its own. Each Jellyfin generation replaces the previous file there. Playback keeps going while it works. If Jellyfin is transcoding, SubMiner transcribes the audio track that playback started with.
|
||||
|
||||
For a YouTube video, SubMiner downloads its audio first and pauses the video until the subtitles load. The subtitles are temporary, and the audio is deleted when generation ends or you switch videos. See [YouTube integration](/youtube-integration#generate-subtitles-with-whisper).
|
||||
|
||||
Change the shortcut with `shortcuts.openSubtitleGeneration`.
|
||||
@@ -46,7 +48,7 @@ subminer generate-subs episode.mkv --model-path /path/to/ggml-small.bin
|
||||
| `--audio-stream <index>` | Pick an audio stream by its absolute FFmpeg index |
|
||||
| `--output <path>` | Write to this SRT path. Existing files are never overwritten. |
|
||||
|
||||
With a file argument, SubMiner uses the audio stream tagged Japanese, or the first stream. `Ctrl+C` cancels.
|
||||
With a file argument, SubMiner uses the audio stream tagged Japanese, or the first stream. `Ctrl+C` cancels. The launcher only works with local files. Use the overlay for Jellyfin and YouTube.
|
||||
|
||||
## Choosing a model
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ Update checks and startup launcher migration share a serialized update-state sto
|
||||
- `src/main/boot/` owns boot-phase assembly seams so `src/main.ts` can stay focused on lifecycle coordination and startup-path selection.
|
||||
- `src/main/runtime/linux-overlay-mode-runtime.ts` owns Linux fullscreen mode state and window replacement. App cleanup cancels pending replacements; `main.ts` supplies window creation and subtitle refresh hooks.
|
||||
- `src/core/services/` owns focused runtime services plus pure or side-effect-bounded logic.
|
||||
- `src/core/services/subtitle-generation*.ts` shares local whisper.cpp transcription, safe model downloads, and progress between the launcher and Electron. Optional dialogue mode retains both Silero-detected speech and other audible sections, omits confidently silent gaps, decodes passages independently, and restores original media timing. `src/main/runtime/subtitle-generation-runtime.ts` owns the overlay job lifecycle and only loads completed subtitles into the same local media; `src/shared/subtitle-generation*.ts` owns configuration, the multilingual model catalog, and IPC contracts. The overlay runtime retains a session model selection, validates picker requests through IPC, and keeps external model paths authoritative.
|
||||
- `src/core/services/subtitle-generation*.ts` shares local whisper.cpp transcription, safe model downloads, and progress between the launcher and Electron. Optional dialogue mode retains both Silero-detected speech and other audible sections, omits confidently silent gaps, decodes passages independently, and restores original media timing. `src/main/runtime/subtitle-generation-runtime.ts` owns the overlay job lifecycle and only loads completed subtitles into the same media. For Jellyfin playback it reads the item's original-file stream (`toJellyfinOriginalStreamUrl`) into one reused output folder, where each run best-effort deletes earlier files and writes a uniquely named SRT; transcoded playback uses Jellyfin's audio index and only external references, since mpv's track indexes describe the transcode. Subprocess errors redact URL tokens; `src/shared/subtitle-generation*.ts` owns configuration, the multilingual model catalog, and IPC contracts. The overlay runtime retains a session model selection, validates picker requests through IPC, and keeps external model paths authoritative.
|
||||
- Subtitle model recommendations use bounded `nvidia-smi` and Whisper CUDA discovery probes in `subtitle-generation-acceleration.ts`. The overlay runtime caches results by executable path for 30 seconds and exposes acceleration status through the existing status IPC. Recommendations do not alter model selection or transcription arguments.
|
||||
- `subtitle-generation-reference.ts` ranks mpv's loaded text subtitle tracks, excludes signs/songs and forced references, and extracts timing hints with FFmpeg. The overlay and launcher snapshot references only for matching media, including active subtitle delays. Hints guide long-passage cuts with or without VAD; they never limit audio coverage or replace Whisper timestamps.
|
||||
- `src/main/runtime/subtitle-selection.ts` reads and validates mpv subtitle tracks and applies primary/secondary selections. Its opt-in session shortcut opens the shared overlay modal window, with renderer focus and subtitle suppression handled by the modal registry.
|
||||
|
||||
@@ -1935,6 +1935,27 @@ describe('stats server API routes', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/mine-card widens halfwidth katakana from older stats lines', async () => {
|
||||
await withMiningEnv({}, async (env) => {
|
||||
const app = createStatsApp(createMockTracker(), {
|
||||
addYomitanNote: async () => 777,
|
||||
ankiConnectConfig: ankiConfig(env, { fields: { sentence: 'Sentence' } }),
|
||||
});
|
||||
|
||||
const { status, body } = await mineCard(app, 'word', {
|
||||
sourcePath: env.sourcePath,
|
||||
sentence: 'アンタがアータルたん?',
|
||||
word: 'アンタ',
|
||||
});
|
||||
|
||||
assert.equal(status, 200, JSON.stringify(body));
|
||||
assert.equal(
|
||||
sentNote(env.requests, 'updateNoteFields').fields.Sentence,
|
||||
'<b>アンタ</b>がアータルたん?',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/mine-card leaves word card selection text to Yomitan glossary', async () => {
|
||||
await withMiningEnv({}, async (env) => {
|
||||
const app = createStatsApp(createMockTracker(), {
|
||||
|
||||
@@ -3936,6 +3936,83 @@ test('getWordOccurrences maps a normalized word back to anime, video, and subtit
|
||||
}
|
||||
});
|
||||
|
||||
test('getWordOccurrences widens halfwidth katakana in lines recorded before ingest widening', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/kamikatsu-s01e08.mkv', {
|
||||
canonicalTitle: 'Episode 8',
|
||||
sourcePath: '/tmp/KamiKatsu S01E08.mkv',
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
const { sessionId } = startSessionRecord(db, videoId, 1_000_000);
|
||||
const wordResult = db
|
||||
.prepare(
|
||||
`INSERT INTO imm_words (
|
||||
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run('アンタ', 'アンタ', 'あんた', 'noun', '名詞', '代名詞', '', 1_000, 1_000, 1);
|
||||
const lineResult = db
|
||||
.prepare(
|
||||
`INSERT INTO imm_subtitle_lines (
|
||||
session_id, event_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(sessionId, null, videoId, null, 1, 0, 1000, 'アンタがアータルたん?', 1_000, 1_000);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
|
||||
VALUES (?, ?, ?)`,
|
||||
).run(Number(lineResult.lastInsertRowid), Number(wordResult.lastInsertRowid), 1);
|
||||
|
||||
const rows = getWordOccurrences(db, 'アンタ', 'アンタ', 'あんた', 10);
|
||||
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.text),
|
||||
['アンタがアータルたん?'],
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('searchSubtitleSentences matches halfwidth and fullwidth katakana in either direction', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/kamikatsu-s01e08.mkv', {
|
||||
canonicalTitle: 'Episode 8',
|
||||
sourcePath: '/tmp/KamiKatsu S01E08.mkv',
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
const { sessionId } = startSessionRecord(db, videoId, 1_000_000);
|
||||
const insertLine = db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines (
|
||||
session_id, event_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
);
|
||||
insertLine.run(sessionId, null, videoId, null, 1, 0, 1000, 'アンタがアータルたん?', 1_000, 1_000);
|
||||
insertLine.run(sessionId, null, videoId, null, 2, 1000, 2000, 'アンタ、ダキニ', 2_000, 2_000);
|
||||
|
||||
for (const query of ['アンタ', 'アンタ']) {
|
||||
assert.deepEqual(
|
||||
searchSubtitleSentences(db, query, 10).map((row) => row.text),
|
||||
['アンタ、ダキニ', 'アンタがアータルたん?'],
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('searchSubtitleSentences searches known subtitle lines and returns media context', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = openTestDb(dbPath);
|
||||
|
||||
@@ -18,6 +18,7 @@ import type {
|
||||
} from './types';
|
||||
import { fromDbTimestamp, toDbTimestamp } from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
import { narrowFullwidthKana, widenHalfwidthKana } from '../../utils/halfwidth-kana';
|
||||
import {
|
||||
areLexicalDailyRollupsReady,
|
||||
getLexicalDailyRollups,
|
||||
@@ -382,6 +383,13 @@ export function getKanjiStats(db: DatabaseSync, limit = 100): KanjiStatsRow[] {
|
||||
return stmt.all(limit) as KanjiStatsRow[];
|
||||
}
|
||||
|
||||
// Lines recorded before subtitle text was widened at ingest can still hold halfwidth
|
||||
// katakana; widen them on the way out so the dashboard and cards mined from it match
|
||||
// newer rows. Display-only: word identity columns stay as stored since they are keys.
|
||||
function widenLineText<T extends { text: string }>(row: T): T {
|
||||
return { ...row, text: widenHalfwidthKana(row.text) };
|
||||
}
|
||||
|
||||
export function getWordOccurrences(
|
||||
db: DatabaseSync,
|
||||
headword: string,
|
||||
@@ -390,7 +398,7 @@ export function getWordOccurrences(
|
||||
limit = 100,
|
||||
offset = 0,
|
||||
): WordOccurrenceRow[] {
|
||||
return db
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
@@ -418,6 +426,7 @@ export function getWordOccurrences(
|
||||
`,
|
||||
)
|
||||
.all(headword, word, reading, limit, offset) as unknown as WordOccurrenceRow[];
|
||||
return rows.map(widenLineText);
|
||||
}
|
||||
|
||||
export function getKanjiOccurrences(
|
||||
@@ -426,7 +435,7 @@ export function getKanjiOccurrences(
|
||||
limit = 100,
|
||||
offset = 0,
|
||||
): KanjiOccurrenceRow[] {
|
||||
return db
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
@@ -454,6 +463,7 @@ export function getKanjiOccurrences(
|
||||
`,
|
||||
)
|
||||
.all(kanji, limit, offset) as unknown as KanjiOccurrenceRow[];
|
||||
return rows.map(widenLineText);
|
||||
}
|
||||
|
||||
export function searchSubtitleSentences(
|
||||
@@ -470,6 +480,14 @@ export function searchSubtitleSentences(
|
||||
const params: string[] = [];
|
||||
for (const term of terms) {
|
||||
const likeTerm = `%${escapeLikeTerm(term)}%`;
|
||||
// Lines recorded before ingest widening can hold halfwidth katakana, so subtitle
|
||||
// text is matched against the term's widened and narrowed forms as well.
|
||||
const widenedTerm = widenHalfwidthKana(term);
|
||||
const textLikeTerms = uniqueNonEmptyTerms([
|
||||
term,
|
||||
widenedTerm,
|
||||
narrowFullwidthKana(widenedTerm),
|
||||
]).map((variant) => `%${escapeLikeTerm(variant)}%`);
|
||||
const headwords = getHeadwordCandidatesForSentenceSearchTerm(term, options);
|
||||
const headwordClause =
|
||||
headwords.length > 0
|
||||
@@ -485,16 +503,16 @@ export function searchSubtitleSentences(
|
||||
: '';
|
||||
clauses.push(`
|
||||
(
|
||||
l.text LIKE ? ESCAPE '\\'
|
||||
${textLikeTerms.map(() => "l.text LIKE ? ESCAPE '\\'").join(' OR ')}
|
||||
OR v.canonical_title LIKE ? ESCAPE '\\'
|
||||
OR COALESCE(a.canonical_title, '') LIKE ? ESCAPE '\\'
|
||||
${headwordClause}
|
||||
)
|
||||
`);
|
||||
params.push(likeTerm, likeTerm, likeTerm, ...headwords);
|
||||
params.push(...textLikeTerms, likeTerm, likeTerm, ...headwords);
|
||||
}
|
||||
|
||||
return db
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
@@ -518,6 +536,7 @@ export function searchSubtitleSentences(
|
||||
`,
|
||||
)
|
||||
.all(...params, resolvedLimit) as unknown as SentenceSearchResultRow[];
|
||||
return rows.map(widenLineText);
|
||||
}
|
||||
|
||||
export function getSessionEvents(
|
||||
|
||||
@@ -130,6 +130,7 @@ export {
|
||||
listSubtitleTracks as listJellyfinSubtitleTracksRuntime,
|
||||
resolvePlaybackPlan as resolveJellyfinPlaybackPlanRuntime,
|
||||
ticksToSeconds as jellyfinTicksToSecondsRuntime,
|
||||
toJellyfinOriginalStreamUrl,
|
||||
} from './jellyfin';
|
||||
export { JellyfinRemoteSessionService } from './jellyfin-remote';
|
||||
export {
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
listSubtitleTracks,
|
||||
resolvePlaybackPlan,
|
||||
ticksToSeconds,
|
||||
toJellyfinOriginalStreamUrl,
|
||||
} from './jellyfin';
|
||||
|
||||
const clientInfo = {
|
||||
@@ -862,3 +863,26 @@ test('resolvePlaybackPlan replaces a legacy api_key on the server transcoding ur
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test('toJellyfinOriginalStreamUrl maps direct and transcoded playback to the original file', () => {
|
||||
assert.equal(
|
||||
toJellyfinOriginalStreamUrl(
|
||||
'https://media.example/jf/videos/item-1/master.m3u8?ApiKey=token&MediaSourceId=src&VideoCodec=h264&StartTimeTicks=9',
|
||||
'item-1',
|
||||
),
|
||||
'https://media.example/jf/Videos/item-1/stream?static=true&ApiKey=token&MediaSourceId=src',
|
||||
);
|
||||
assert.equal(
|
||||
toJellyfinOriginalStreamUrl(
|
||||
'http://jf.local:8096/Videos/item-1/stream?static=true&api_key=legacy&AudioStreamIndex=2',
|
||||
'item-1',
|
||||
),
|
||||
'http://jf.local:8096/Videos/item-1/stream?static=true&ApiKey=legacy',
|
||||
);
|
||||
assert.equal(
|
||||
toJellyfinOriginalStreamUrl('https://jf/Videos/other/stream?ApiKey=token', 'item-1'),
|
||||
null,
|
||||
);
|
||||
assert.equal(toJellyfinOriginalStreamUrl('https://jf/Videos/item-1/stream', 'item-1'), null);
|
||||
assert.equal(toJellyfinOriginalStreamUrl('/media/item-1.mkv', 'item-1'), null);
|
||||
});
|
||||
@@ -254,6 +254,29 @@ function createDirectPlayUrl(
|
||||
return `${session.serverUrl}/Videos/${itemId}/stream?${query.toString()}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Maps a playing stream URL of `itemId`, direct or transcoded, to its original file. Returns null
|
||||
* when the URL is not that item's stream or carries no access token.
|
||||
*/
|
||||
export function toJellyfinOriginalStreamUrl(playbackUrl: string, itemId: string): string | null {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(playbackUrl);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const at = url.pathname.toLowerCase().indexOf(`/videos/${itemId.toLowerCase()}/`);
|
||||
if (!/^https?:$/.test(url.protocol) || !itemId || at < 0) return null;
|
||||
const param = (pattern: RegExp) =>
|
||||
[...url.searchParams].find(([key, value]) => pattern.test(key) && value)?.[1];
|
||||
const accessToken = param(/^api_?key$/i);
|
||||
if (!accessToken) return null;
|
||||
const query = new URLSearchParams({ static: 'true', ApiKey: accessToken });
|
||||
const mediaSourceId = param(/^mediasourceid$/i);
|
||||
if (mediaSourceId) query.set('MediaSourceId', mediaSourceId);
|
||||
return `${url.origin}${url.pathname.slice(0, at)}/Videos/${itemId}/stream?${query.toString()}`;
|
||||
}
|
||||
|
||||
function createTranscodeUrl(
|
||||
session: JellyfinAuthSession,
|
||||
itemId: string,
|
||||
|
||||
@@ -131,6 +131,18 @@ test('dispatchMpvProtocolMessage emits subtitle text on property change', async
|
||||
assert.deepEqual(state.events, [{ text: '字幕', isOverlayVisible: false }]);
|
||||
});
|
||||
|
||||
test('dispatchMpvProtocolMessage widens halfwidth katakana in subtitle text', async () => {
|
||||
const { deps, state } = createDeps();
|
||||
|
||||
await dispatchMpvProtocolMessage(
|
||||
{ event: 'property-change', name: 'sub-text', data: 'アンタがアータルたん?' },
|
||||
deps,
|
||||
);
|
||||
|
||||
assert.equal(state.subText, 'アンタがアータルたん?');
|
||||
assert.deepEqual(state.events, [{ text: 'アンタがアータルたん?', isOverlayVisible: false }]);
|
||||
});
|
||||
|
||||
test('dispatchMpvProtocolMessage emits subtitle track changes', async () => {
|
||||
const { deps, state } = createDeps({
|
||||
emitSubtitleTrackChange: (payload) => state.events.push(payload),
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { MpvSubtitleRenderMetrics } from '../../types';
|
||||
import { sanitizeMediaTitle } from '../../shared/media-identity';
|
||||
import { widenHalfwidthKana } from '../utils/halfwidth-kana';
|
||||
|
||||
export type MpvMessage = {
|
||||
event?: string;
|
||||
@@ -244,7 +245,7 @@ export async function dispatchMpvProtocolMessage(
|
||||
): Promise<void> {
|
||||
if (msg.event === 'property-change') {
|
||||
if (msg.name === 'sub-text') {
|
||||
const nextSubText = (msg.data as string) || '';
|
||||
const nextSubText = widenHalfwidthKana((msg.data as string) || '');
|
||||
const overlayVisible = deps.isVisibleOverlayVisible();
|
||||
deps.emitSubtitleChange({
|
||||
text: nextSubText,
|
||||
@@ -458,7 +459,7 @@ export async function dispatchMpvProtocolMessage(
|
||||
}>,
|
||||
);
|
||||
} else if (msg.request_id === MPV_REQUEST_ID_SUBTEXT) {
|
||||
const nextSubText = (msg.data as string) || '';
|
||||
const nextSubText = widenHalfwidthKana((msg.data as string) || '');
|
||||
deps.setCurrentSubText(nextSubText);
|
||||
deps.emitSubtitleChange({
|
||||
text: nextSubText,
|
||||
|
||||
@@ -8,6 +8,7 @@ import { clampMediaEndTime } from '../../../anki-integration/media-duration.js';
|
||||
import { MediaGenerator } from '../../../media-generator.js';
|
||||
import { statsJson } from '../../../types/stats-http-contract.js';
|
||||
import { STATS_MINING_TAG } from '../../../shared/anki-source.js';
|
||||
import { widenHalfwidthKana } from '../../utils/halfwidth-kana.js';
|
||||
import {
|
||||
resolveRetimedSecondarySubtitleTextFromSidecar,
|
||||
resolveSecondarySubtitleTextFromSidecar,
|
||||
@@ -36,8 +37,11 @@ export function registerStatsMiningRoutes(app: Hono, options?: StatsMiningRouteO
|
||||
const sourcePath = typeof body?.sourcePath === 'string' ? body.sourcePath.trim() : '';
|
||||
const startMs = typeof body?.startMs === 'number' ? body.startMs : NaN;
|
||||
const endMs = typeof body?.endMs === 'number' ? body.endMs : NaN;
|
||||
const sentence = typeof body?.sentence === 'string' ? body.sentence.trim() : '';
|
||||
const word = typeof body?.word === 'string' ? body.word.trim() : '';
|
||||
// Older stats rows can carry halfwidth katakana; cards get the fullwidth form that
|
||||
// live mining produces, and the word is widened alongside so highlighting still hits.
|
||||
const sentence =
|
||||
typeof body?.sentence === 'string' ? widenHalfwidthKana(body.sentence.trim()) : '';
|
||||
const word = typeof body?.word === 'string' ? widenHalfwidthKana(body.word.trim()) : '';
|
||||
const bodySecondaryText =
|
||||
typeof body?.secondaryText === 'string' ? body.secondaryText.trim() : '';
|
||||
const videoTitle = typeof body?.videoTitle === 'string' ? body.videoTitle.trim() : '';
|
||||
|
||||
@@ -55,6 +55,36 @@ test('parseSrtCues strips HTML-like markup while preserving line breaks', () =>
|
||||
assert.equal(cues[0]!.text, 'これは\nテストです');
|
||||
});
|
||||
|
||||
test('parseSubtitleCues widens halfwidth katakana in cue text', () => {
|
||||
const cues = parseSubtitleCues(
|
||||
'1\n00:00:01,000 --> 00:00:02,000\n(トーカ)ダキニお姉ちゃん\n',
|
||||
'ep.srt',
|
||||
);
|
||||
|
||||
assert.equal(cues[0]?.text, '(トーカ)ダキニお姉ちゃん');
|
||||
});
|
||||
|
||||
test('parseSubtitleCues merges cues that differ only in halfwidth katakana', () => {
|
||||
const cues = parseSubtitleCues(
|
||||
[
|
||||
'1',
|
||||
'00:00:01,000 --> 00:00:02,000',
|
||||
'山ガク',
|
||||
'',
|
||||
'2',
|
||||
'00:00:01,000 --> 00:00:02,000',
|
||||
'山ガク',
|
||||
'',
|
||||
].join('\n'),
|
||||
'ep.srt',
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
cues.map((cue) => cue.text),
|
||||
['山ガク'],
|
||||
);
|
||||
});
|
||||
|
||||
test('parseSrtCues handles hours in timestamps', () => {
|
||||
const content = ['1', '01:30:00,000 --> 01:30:05,000', 'テスト', ''].join('\n');
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
type AssOverrideCommand,
|
||||
} from './ass-text';
|
||||
import { hasAssAnimationEvidence, mergeDuplicateCues } from './subtitle-cue-dedup';
|
||||
import { widenHalfwidthKana } from '../utils/halfwidth-kana';
|
||||
|
||||
/** Vertical third of the screen a cue is authored to occupy. */
|
||||
export type AssVerticalBand = 'top' | 'middle' | 'bottom';
|
||||
@@ -104,7 +105,7 @@ function decodeSubtitleCueText(text: string): string {
|
||||
}
|
||||
|
||||
function sanitizeSubtitleCueText(text: string): string {
|
||||
return decodeSubtitleCueText(text).trim();
|
||||
return widenHalfwidthKana(decodeSubtitleCueText(text)).trim();
|
||||
}
|
||||
|
||||
function sanitizeAssCueText(text: string): string {
|
||||
@@ -2655,10 +2656,16 @@ function parseAnnotatedAssCues(content: string): AnnotatedSubtitleCue[] {
|
||||
removeAssFontTextureEvents(parseAnnotatedAssEvents(content, placement)),
|
||||
placement.playResY,
|
||||
);
|
||||
return mergeAssCaptionRows(
|
||||
const cues = mergeAssCaptionRows(
|
||||
recoverFragmentOnlyAssLines(recoverCanonicalAssEvents(events)),
|
||||
placement.playResY,
|
||||
);
|
||||
// Widened after reconstruction, which works glyph by glyph, but before duplicate
|
||||
// merging, so ガク and ガク authored for the same span collapse into one cue.
|
||||
for (const cue of cues) {
|
||||
cue.text = widenHalfwidthKana(cue.text);
|
||||
}
|
||||
return cues;
|
||||
}
|
||||
|
||||
export function parseAssCues(content: string): SubtitleCue[] {
|
||||
|
||||
@@ -3,6 +3,11 @@ import { expandSubtitleGenerationPath } from './subtitle-generation-files';
|
||||
|
||||
const OUTPUT_LIMIT = 64 * 1024;
|
||||
|
||||
// ffmpeg echoes input URLs in its errors; streamed media carries access tokens in the query.
|
||||
function redactUrlCredentials(text: string): string {
|
||||
return text.replace(/([?&](?:api_?key|token|access_token)=)[^&#\s'"]+/gi, '$1<redacted>');
|
||||
}
|
||||
|
||||
// Keep partial lines between chunks: ffmpeg and whisper both report progress on stderr.
|
||||
export function runSubtitleGenerationProcess(input: {
|
||||
command: string;
|
||||
@@ -63,7 +68,11 @@ export function runSubtitleGenerationProcess(input: {
|
||||
cleanup();
|
||||
if (input.signal?.aborted) reject(new Error('Subtitle generation cancelled.'));
|
||||
else if (code !== 0) {
|
||||
reject(new Error(`${input.command} exited with status ${code}: ${stderr.trim()}`));
|
||||
reject(
|
||||
new Error(
|
||||
`${input.command} exited with status ${code}: ${redactUrlCredentials(stderr.trim())}`,
|
||||
),
|
||||
);
|
||||
} else resolve(stdout);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -378,7 +378,22 @@ test('empty executable paths find tools on PATH and explicit overrides take prec
|
||||
}
|
||||
}));
|
||||
|
||||
test('generation rejects remote media, missing models, and missing tools before starting a subprocess', () =>
|
||||
test('generation reads an HTTP stream as-is and saves to the requested output', () =>
|
||||
fixture(async (directory) => {
|
||||
const input = await generationFixture(directory);
|
||||
const mediaPath = 'https://jellyfin.example/Videos/item/stream?static=true&ApiKey=secret';
|
||||
const outputPath = path.join(directory, 'out', 'stream.ja.srt');
|
||||
await mkdir(path.dirname(outputPath));
|
||||
assert.equal(await generateJapaneseSubtitles({ ...input, mediaPath, outputPath }), outputPath);
|
||||
const calls: string[][] = (await readFile(input.callsPath, 'utf8'))
|
||||
.trim()
|
||||
.split('\n')
|
||||
.map((line) => JSON.parse(line));
|
||||
assert.equal(calls[0]?.at(-1), mediaPath);
|
||||
assert.equal(calls[1]?.[calls[1].indexOf('-i') + 1], mediaPath);
|
||||
}));
|
||||
|
||||
test('generation rejects remote media without an output path, missing models, and missing tools before starting a subprocess', () =>
|
||||
fixture(async (directory) => {
|
||||
const input = await generationFixture(directory);
|
||||
await assert.rejects(
|
||||
@@ -473,6 +488,17 @@ test('process cancellation terminates work and bounds diagnostic output', () =>
|
||||
error.message.length < 66000 &&
|
||||
error.message.includes('status 7'),
|
||||
);
|
||||
const unauthorized = await executable(
|
||||
directory,
|
||||
'unauthorized',
|
||||
"process.stderr.write('https://host/Videos/1/stream?ApiKey=se+cr/et==&static=true: Server returned 401');process.exitCode=1;",
|
||||
);
|
||||
await assert.rejects(
|
||||
runSubtitleGenerationProcess({ command: unauthorized, args: [] }),
|
||||
(error: unknown) =>
|
||||
error instanceof Error &&
|
||||
error.message.includes('?ApiKey=<redacted>&static=true: Server returned 401'),
|
||||
);
|
||||
}));
|
||||
|
||||
test('download uses the pinned model revision and removes files that fail integrity', () =>
|
||||
|
||||
@@ -191,6 +191,7 @@ export async function requireSubtitleGenerationSetup(
|
||||
export async function generateJapaneseSubtitles(input: {
|
||||
config: SubtitleGenerationConfig;
|
||||
modelDirectory: string;
|
||||
/** A local file, or an http(s) stream when `outputPath` is set. */
|
||||
mediaPath: string;
|
||||
audioStreamIndex?: number;
|
||||
references?: readonly SubtitleGenerationReference[];
|
||||
@@ -201,10 +202,12 @@ export async function generateJapaneseSubtitles(input: {
|
||||
signal?: AbortSignal;
|
||||
}): Promise<string> {
|
||||
input.signal?.throwIfAborted();
|
||||
if (/^[a-z][a-z\d+.-]*:\/\//i.test(input.mediaPath))
|
||||
// HTTP streams are read by ffmpeg directly but have no directory to save beside.
|
||||
const remote = /^https?:\/\//i.test(input.mediaPath);
|
||||
if (remote ? !input.outputPath : /^[a-z][a-z\d+.-]*:\/\//i.test(input.mediaPath))
|
||||
throw new Error('Subtitle generation requires a local media file.');
|
||||
const mediaPath = path.resolve(input.mediaPath);
|
||||
if (!(await stat(mediaPath)).isFile())
|
||||
const mediaPath = remote ? input.mediaPath : path.resolve(input.mediaPath);
|
||||
if (!remote && !(await stat(mediaPath)).isFile())
|
||||
throw new Error('Subtitle generation requires a local media file.');
|
||||
if (input.outputPath) await ensureAvailableOutput(path.resolve(input.outputPath));
|
||||
await ensureWritableDirectory(
|
||||
|
||||
@@ -1,23 +1,12 @@
|
||||
import { MergedToken, PartOfSpeech } from '../../../types';
|
||||
import { widenHalfwidthKana } from '../../utils/halfwidth-kana';
|
||||
|
||||
const KATAKANA_TO_HIRAGANA_OFFSET = 0x60;
|
||||
const KATAKANA_CODEPOINT_START = 0x30a1;
|
||||
const KATAKANA_CODEPOINT_END = 0x30f6;
|
||||
|
||||
// No `u` flag: the range is entirely BMP so it changes nothing here, and
|
||||
// Bun's unicode-mode matcher mis-handles this class next to certain ligatures.
|
||||
const HALFWIDTH_KANA_RUN = /[\uff66-\uff9f]+/g;
|
||||
|
||||
// NFKC over the halfwidth kana only, never the whole string: it composes the
|
||||
// voiced pairs (カ + ゙) into single characters so ガク compares equal to ガク
|
||||
// instead of counting one character longer than the word it spells, but run
|
||||
// over everything it would also rewrite unrelated text (① → 1, ㍑ → リットル).
|
||||
function composeHalfwidthKana(text: string): string {
|
||||
return text.replace(HALFWIDTH_KANA_RUN, (run) => run.normalize('NFKC'));
|
||||
}
|
||||
|
||||
export function normalizeKana(text: string): string {
|
||||
const raw = composeHalfwidthKana(text).trim();
|
||||
const raw = widenHalfwidthKana(text).trim();
|
||||
if (!raw) {
|
||||
return '';
|
||||
}
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { narrowFullwidthKana, widenHalfwidthKana } from './halfwidth-kana';
|
||||
|
||||
test('widenHalfwidthKana widens halfwidth katakana and composes voiced marks', () => {
|
||||
assert.equal(
|
||||
widenHalfwidthKana('(トーカ)ダキニお姉ちゃん アンタがアータルたん。'),
|
||||
'(トーカ)ダキニお姉ちゃん アンタがアータルたん。',
|
||||
);
|
||||
});
|
||||
|
||||
test('widenHalfwidthKana leaves other compatibility characters alone', () => {
|
||||
assert.equal(widenHalfwidthKana('見て見てっ!①㍑AB'), '見て見てっ!①㍑AB');
|
||||
});
|
||||
|
||||
test('widenHalfwidthKana folds a lone halfwidth voiced mark into the kana before it', () => {
|
||||
assert.equal(widenHalfwidthKana('山ガク パン'), '山ガク パン');
|
||||
assert.equal(widenHalfwidthKana('ア゙'), 'ア\u3099');
|
||||
});
|
||||
|
||||
test('narrowFullwidthKana inverts widening and leaves other text alone', () => {
|
||||
assert.equal(narrowFullwidthKana('ダキニお姉ちゃん アンタ。ヴ'), 'ダキニお姉ちゃん アンタ。ヴ');
|
||||
assert.equal(narrowFullwidthKana('漢字ひらがな!'), '漢字ひらがな!');
|
||||
});
|
||||
@@ -0,0 +1,49 @@
|
||||
// No `u` flag: the range is entirely BMP so it changes nothing here, and
|
||||
// Bun's unicode-mode matcher mis-handles this class next to certain ligatures.
|
||||
// The optional leading character is captured so a run opening with a voiced
|
||||
// mark (ガ) can fold the mark into the fullwidth kana before it.
|
||||
const HALFWIDTH_KANA_RUN = /([^。-゚]?)([。-゚]+)/g;
|
||||
const LEADING_COMBINING_VOICED_MARK = /^[゙゚]/;
|
||||
|
||||
/**
|
||||
* Rewrites halfwidth katakana and its punctuation (アンタ, ダキニ, 。「」、・) to the fullwidth
|
||||
* forms. Broadcast-derived subtitles write katakana halfwidth to save space; subtitle
|
||||
* text is normalized with this where it enters the app (live mpv text, parsed cues) so
|
||||
* display, lookups, mining, and stats all see one form.
|
||||
*
|
||||
* NFKC runs over the halfwidth runs only, never the whole string: it composes the voiced
|
||||
* pairs (カ + ゙) into single characters, but over everything it would also rewrite
|
||||
* unrelated text (① → 1, ! → !, ㍑ → リットル).
|
||||
*/
|
||||
export function widenHalfwidthKana(text: string): string {
|
||||
return text.replace(HALFWIDTH_KANA_RUN, (_match, previous: string, run: string) => {
|
||||
const widened = run.normalize('NFKC');
|
||||
if (!previous || !LEADING_COMBINING_VOICED_MARK.test(widened)) {
|
||||
return previous + widened;
|
||||
}
|
||||
// A lone ゙ widens to a combining mark; NFC composes it with the kana before it
|
||||
// (カ + ゛ → ガ) and leaves pairs that have no composed form untouched.
|
||||
return (previous + widened[0]).normalize('NFC') + widened.slice(1);
|
||||
});
|
||||
}
|
||||
|
||||
// Inverse of widenHalfwidthKana per character, derived from it so the two stay in sync.
|
||||
// Voiced pairs (ガ) are included because they widen to a single character (ガ).
|
||||
const HALFWIDTH_BY_FULLWIDTH = new Map<string, string>();
|
||||
for (let code = 0xff61; code <= 0xff9f; code += 1) {
|
||||
const halfwidth = String.fromCharCode(code);
|
||||
for (const form of [halfwidth, `${halfwidth}゙`, `${halfwidth}゚`]) {
|
||||
const widened = widenHalfwidthKana(form);
|
||||
if (widened.length === 1 && !HALFWIDTH_BY_FULLWIDTH.has(widened)) {
|
||||
HALFWIDTH_BY_FULLWIDTH.set(widened, form);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Rewrites fullwidth katakana and punctuation to halfwidth forms (アンタ → アンタ). Used to
|
||||
* search text stored before subtitle ingest widened it; display paths should widen instead.
|
||||
*/
|
||||
export function narrowFullwidthKana(text: string): string {
|
||||
return Array.from(text, (char) => HALFWIDTH_BY_FULLWIDTH.get(char) ?? char).join('');
|
||||
}
|
||||
+12
@@ -319,6 +319,7 @@ import {
|
||||
replayCurrentSubtitleRuntime,
|
||||
resolveSanitizedSubtitleSeekCommand,
|
||||
resolveJellyfinPlaybackPlanRuntime,
|
||||
toJellyfinOriginalStreamUrl,
|
||||
runStartupBootstrapRuntime,
|
||||
saveSubtitlePosition as saveSubtitlePositionCore,
|
||||
clearYomitanParserCachesForWindow,
|
||||
@@ -6924,6 +6925,17 @@ const subtitleGenerationRuntime = createSubtitleGenerationRuntime({
|
||||
}),
|
||||
resolveYoutubeSourceUrl: (mediaPath) =>
|
||||
youtubePlaybackRuntime.getYoutubeSourceUrlForStream(mediaPath),
|
||||
resolveJellyfinMedia: (mediaPath) => {
|
||||
const playback = activeJellyfinRemotePlayback;
|
||||
const sourceUrl = playback && toJellyfinOriginalStreamUrl(mediaPath, playback.itemId);
|
||||
return sourceUrl
|
||||
? {
|
||||
sourceUrl,
|
||||
transcoded: playback.playMethod === 'Transcode',
|
||||
audioStreamIndex: playback.audioStreamIndex ?? null,
|
||||
}
|
||||
: null;
|
||||
},
|
||||
getMpvClient: () => appState.mpvClient,
|
||||
onProgress: (progress) => {
|
||||
for (const window of [overlayManager.getMainWindow(), overlayManager.getModalWindow()]) {
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
removeAssControlDebrisLines,
|
||||
removeLiveGlyphFragmentLines,
|
||||
} from '../../core/services/ass-text';
|
||||
import { widenHalfwidthKana } from '../../core/utils/halfwidth-kana';
|
||||
|
||||
// Slack on top of each cue's recorded animation envelope, for time-pos observation
|
||||
// staleness and small user sub-delay offsets. The envelope itself covers how far
|
||||
@@ -26,11 +27,14 @@ function cuesUseAssSyntax(cues: readonly SubtitleCue[] | null | undefined): bool
|
||||
);
|
||||
}
|
||||
|
||||
// Live text also arrives from direct `sub-text` reads that bypass the mpv protocol
|
||||
// handler, so it is widened here too to match the parsed cues.
|
||||
function decodedLiveText(
|
||||
liveText: string,
|
||||
cues: readonly SubtitleCue[] | null | undefined,
|
||||
): string {
|
||||
return cuesUseAssSyntax(cues) ? removeAssControlDebrisLines(liveText) : liveText;
|
||||
const text = widenHalfwidthKana(liveText);
|
||||
return cuesUseAssSyntax(cues) ? removeAssControlDebrisLines(text) : text;
|
||||
}
|
||||
|
||||
function animationSpan(cue: SubtitleCue): { start: number; end: number } {
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { DEFAULT_SUBTITLE_GENERATION_CONFIG } from '../../shared/subtitle-generation';
|
||||
import {
|
||||
createSubtitleGenerationRuntime,
|
||||
type SubtitleGenerationJellyfinMedia,
|
||||
type SubtitleGenerationRuntimeDeps,
|
||||
} from './subtitle-generation-runtime';
|
||||
|
||||
@@ -506,3 +508,101 @@ test('switching to another resolved YouTube stream cancels the modal job', async
|
||||
|
||||
assert.deepEqual(await job, { ok: false, message: 'Cancelled.' });
|
||||
});
|
||||
|
||||
function jellyfinFixture(
|
||||
jellyfin: Omit<SubtitleGenerationJellyfinMedia, 'sourceUrl'>,
|
||||
overrides: Partial<SubtitleGenerationRuntimeDeps> = {},
|
||||
) {
|
||||
const playing = 'https://jf.example/Videos/item/master.m3u8?ApiKey=secret';
|
||||
const subject = fixture({
|
||||
resolveJellyfinMedia: (mediaPath) =>
|
||||
mediaPath === playing
|
||||
? {
|
||||
sourceUrl: 'https://jf.example/Videos/item/stream?static=true&ApiKey=secret',
|
||||
...jellyfin,
|
||||
}
|
||||
: null,
|
||||
createStreamOutputDirectory: async () => '/tmp/generated',
|
||||
...overrides,
|
||||
});
|
||||
subject.client.requestProperty = async (name) =>
|
||||
name === 'path'
|
||||
? playing
|
||||
: name === 'media-title'
|
||||
? 'Show S01E02'
|
||||
: name === 'track-list'
|
||||
? [
|
||||
{ type: 'audio', selected: true, 'ff-index': 0 },
|
||||
{ type: 'sub', lang: 'eng', title: 'English Full', 'ff-index': 4 },
|
||||
{
|
||||
type: 'sub',
|
||||
lang: 'eng',
|
||||
title: 'English',
|
||||
external: true,
|
||||
'external-filename': '/tmp/jellyfin-subs/track-5.srt',
|
||||
},
|
||||
]
|
||||
: null;
|
||||
return subject;
|
||||
}
|
||||
|
||||
test('a direct-play Jellyfin stream generates from its original file and shows by title', async () => {
|
||||
let outputPath = '';
|
||||
const subject = jellyfinFixture(
|
||||
{ transcoded: false, audioStreamIndex: 2 },
|
||||
{
|
||||
generate: async (input) => {
|
||||
assert.equal(
|
||||
input.mediaPath,
|
||||
'https://jf.example/Videos/item/stream?static=true&ApiKey=secret',
|
||||
);
|
||||
outputPath = input.outputPath ?? '';
|
||||
assert.equal(path.dirname(outputPath), path.join('/tmp', 'generated'));
|
||||
assert.match(path.basename(outputPath), /^jellyfin-\d+\.ja\.generated\.srt$/);
|
||||
assert.equal(input.audioStreamIndex, 0);
|
||||
assert.deepEqual(
|
||||
input.references?.map((reference) => reference.source.kind),
|
||||
['embedded', 'external'],
|
||||
);
|
||||
return outputPath;
|
||||
},
|
||||
},
|
||||
);
|
||||
assert.equal((await subject.runtime.getStatus()).mediaPath, 'Jellyfin: Show S01E02');
|
||||
assert.equal((await subject.runtime.start()).ok, true);
|
||||
assert.deepEqual(subject.commands[0], [
|
||||
'sub-add',
|
||||
outputPath,
|
||||
'select',
|
||||
'Generated Japanese',
|
||||
'ja',
|
||||
]);
|
||||
});
|
||||
|
||||
test('a transcoded Jellyfin stream uses the Jellyfin audio index and only external references', async () => {
|
||||
const subject = jellyfinFixture(
|
||||
{ transcoded: true, audioStreamIndex: 2 },
|
||||
{
|
||||
generate: async (input) => {
|
||||
assert.equal(input.audioStreamIndex, 2);
|
||||
assert.deepEqual(
|
||||
input.references?.map((reference) => reference.source.kind),
|
||||
['external'],
|
||||
);
|
||||
return '/tmp/generated/jellyfin.ja.generated.srt';
|
||||
},
|
||||
},
|
||||
);
|
||||
assert.equal((await subject.runtime.start()).ok, true);
|
||||
});
|
||||
|
||||
test('unrecognized streams cannot generate', async () => {
|
||||
const subject = jellyfinFixture(
|
||||
{ transcoded: false, audioStreamIndex: null },
|
||||
{ resolveJellyfinMedia: () => null },
|
||||
);
|
||||
assert.equal((await subject.runtime.getStatus()).mediaPath, null);
|
||||
const result = await subject.runtime.start();
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.message, /Jellyfin stream/);
|
||||
});
|
||||
@@ -1,3 +1,5 @@
|
||||
import { mkdir, readdir, rm } from 'node:fs/promises';
|
||||
import { homedir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { readSubtitleGenerationReferences } from '../../core/services/subtitle-generation-reference';
|
||||
import { detectSubtitleGenerationAcceleration } from '../../core/services/subtitle-generation-acceleration';
|
||||
@@ -29,6 +31,22 @@ interface GenerationMpvClient {
|
||||
request: (command: unknown[]) => Promise<{ error?: string }>;
|
||||
}
|
||||
|
||||
/** A Jellyfin stream playing in mpv, as subtitle generation reads it. */
|
||||
export interface SubtitleGenerationJellyfinMedia {
|
||||
/** The item's original file. It carries the access token, so it never reaches status text. */
|
||||
sourceUrl: string;
|
||||
/**
|
||||
* Transcoded playback has its own stream layout, so mpv's track indexes do not address the
|
||||
* original file. Generation then uses Jellyfin's audio index and skips embedded references.
|
||||
*/
|
||||
transcoded: boolean;
|
||||
audioStreamIndex: number | null;
|
||||
}
|
||||
|
||||
type GenerationMedia =
|
||||
| { kind: 'local'; path: string }
|
||||
| { kind: 'jellyfin'; path: string; jellyfin: SubtitleGenerationJellyfinMedia };
|
||||
|
||||
export interface SubtitleGenerationRuntimeDeps {
|
||||
getConfig: () => SubtitleGenerationConfig;
|
||||
getModelDirectory: () => string;
|
||||
@@ -56,11 +74,27 @@ export interface SubtitleGenerationRuntimeDeps {
|
||||
* resolved direct stream in mpv, so mpv's path alone does not identify the video.
|
||||
*/
|
||||
resolveYoutubeSourceUrl?: (mediaPath: string) => string | null;
|
||||
/** Identifies mpv's path as Jellyfin playback. Without it, streams cannot generate. */
|
||||
resolveJellyfinMedia?: (mediaPath: string) => SubtitleGenerationJellyfinMedia | null;
|
||||
/**
|
||||
* Directory for subtitles of streamed media, which has no folder of its own. Called once per
|
||||
* stream generation, so it may delete what an earlier run wrote there.
|
||||
*/
|
||||
createStreamOutputDirectory?: () => Promise<string>;
|
||||
}
|
||||
|
||||
async function currentLocalMedia(client: GenerationMpvClient | null): Promise<string | null> {
|
||||
if (!client?.connected) return null;
|
||||
const media = await client.requestProperty('path');
|
||||
// One reused folder keeps stream subtitles from piling up: each run deletes earlier files.
|
||||
// mpv has already parsed a loaded track, and a file still locked (Windows) waits for a later run.
|
||||
async function reuseStreamOutputDirectory(): Promise<string> {
|
||||
const directory = path.join(homedir(), '.cache', 'subminer', 'generated-subs');
|
||||
await mkdir(directory, { recursive: true });
|
||||
for (const name of await readdir(directory))
|
||||
await rm(path.join(directory, name), { recursive: true, force: true }).catch(() => undefined);
|
||||
return directory;
|
||||
}
|
||||
|
||||
// Resolves mpv's `path` value to an absolute local file, or null for URLs.
|
||||
async function localMediaPath(client: GenerationMpvClient, media: unknown): Promise<string | null> {
|
||||
if (typeof media !== 'string' || !media || /^[a-z][a-z\d+.-]*:\/\//i.test(media)) return null;
|
||||
if (path.isAbsolute(media)) return path.normalize(media);
|
||||
const directory = await client.requestProperty('working-directory');
|
||||
@@ -139,6 +173,22 @@ export function createSubtitleGenerationRuntime(deps: SubtitleGenerationRuntimeD
|
||||
function youtubeSource(mediaPath: string | null | undefined): string | null | undefined {
|
||||
return (mediaPath && deps.resolveYoutubeSourceUrl?.(mediaPath)) || mediaPath;
|
||||
}
|
||||
async function currentMedia(client: GenerationMpvClient | null): Promise<GenerationMedia | null> {
|
||||
if (!client?.connected) return null;
|
||||
const media = await client.requestProperty('path');
|
||||
if (typeof media === 'string' && /^https?:\/\//i.test(media)) {
|
||||
const jellyfin = deps.resolveJellyfinMedia?.(media);
|
||||
return jellyfin ? { kind: 'jellyfin', path: media, jellyfin } : null;
|
||||
}
|
||||
const local = await localMediaPath(client, media);
|
||||
return local ? { kind: 'local', path: local } : null;
|
||||
}
|
||||
// Stream URLs carry access tokens, so Jellyfin media shows by title.
|
||||
async function describeMedia(client: GenerationMpvClient, media: GenerationMedia) {
|
||||
if (media.kind === 'local') return media.path;
|
||||
const title = await client.requestProperty('media-title').catch(() => null);
|
||||
return `Jellyfin: ${typeof title === 'string' && title.trim() ? title.trim() : 'current item'}`;
|
||||
}
|
||||
async function currentYoutubeMedia(client: GenerationMpvClient | null): Promise<string | null> {
|
||||
if (!client?.connected) return null;
|
||||
const media = await client.requestProperty('path');
|
||||
@@ -196,9 +246,10 @@ export function createSubtitleGenerationRuntime(deps: SubtitleGenerationRuntimeD
|
||||
deps.getModelDirectory(),
|
||||
);
|
||||
const client = deps.getMpvClient();
|
||||
const media = client ? await currentMedia(client).catch(() => null) : null;
|
||||
const mediaPath =
|
||||
youtubeJobUrl ??
|
||||
(await currentLocalMedia(client).catch(() => null)) ??
|
||||
(client && media ? await describeMedia(client, media) : null) ??
|
||||
(deps.generateYoutube ? await currentYoutubeMedia(client).catch(() => null) : null);
|
||||
return {
|
||||
model,
|
||||
@@ -350,21 +401,33 @@ export function createSubtitleGenerationRuntime(deps: SubtitleGenerationRuntimeD
|
||||
const youtubeUrl = deps.generateYoutube ? await currentYoutubeMedia(client) : null;
|
||||
if (client && youtubeUrl && deps.generateYoutube)
|
||||
return await startYoutube(client, youtubeUrl, config, signal, deps.generateYoutube);
|
||||
const mediaPath = await currentLocalMedia(client);
|
||||
if (!client || !mediaPath)
|
||||
throw new Error('Open a local video or audio file, or a YouTube video, in mpv first.');
|
||||
const media = await currentMedia(client);
|
||||
if (!client || !media)
|
||||
throw new Error(
|
||||
'Open a local video or audio file, a Jellyfin stream, or a YouTube video in mpv first.',
|
||||
);
|
||||
const tracks = await client.requestProperty('track-list');
|
||||
const audioStreamIndex = selectedAudioIndex(tracks);
|
||||
const references = await readSubtitleGenerationReferences(tracks, (name) =>
|
||||
client.requestProperty(name),
|
||||
);
|
||||
if ((await currentLocalMedia(client)) !== mediaPath)
|
||||
const transcoded = media.kind === 'jellyfin' && media.jellyfin.transcoded;
|
||||
const audioStreamIndex = transcoded
|
||||
? (media.jellyfin.audioStreamIndex ?? undefined)
|
||||
: selectedAudioIndex(tracks);
|
||||
const references = (
|
||||
await readSubtitleGenerationReferences(tracks, (name) => client.requestProperty(name))
|
||||
).filter((reference) => !transcoded || reference.source.kind === 'external');
|
||||
if ((await currentMedia(client))?.path !== media.path)
|
||||
throw new Error('The current media changed. Start generation again.');
|
||||
signal.throwIfAborted();
|
||||
const outputPath = await (deps.generate ?? generateJapaneseSubtitles)({
|
||||
config,
|
||||
modelDirectory: deps.getModelDirectory(),
|
||||
mediaPath,
|
||||
mediaPath: media.kind === 'jellyfin' ? media.jellyfin.sourceUrl : media.path,
|
||||
outputPath:
|
||||
media.kind === 'jellyfin'
|
||||
? path.join(
|
||||
await (deps.createStreamOutputDirectory ?? reuseStreamOutputDirectory)(),
|
||||
`jellyfin-${Date.now()}.ja.generated.srt`,
|
||||
)
|
||||
: undefined,
|
||||
audioStreamIndex,
|
||||
references,
|
||||
onProgress: report,
|
||||
@@ -372,8 +435,8 @@ export function createSubtitleGenerationRuntime(deps: SubtitleGenerationRuntimeD
|
||||
});
|
||||
// Saving succeeds even if playback changes or disconnects during the job.
|
||||
try {
|
||||
const playingMedia = await currentLocalMedia(client);
|
||||
if (!signal.aborted && deps.getMpvClient() === client && playingMedia === mediaPath) {
|
||||
const playingMedia = (await currentMedia(client))?.path;
|
||||
if (!signal.aborted && deps.getMpvClient() === client && playingMedia === media.path) {
|
||||
await loadGeneratedSubtitles(client, outputPath);
|
||||
return {
|
||||
ok: true,
|
||||
|
||||
@@ -256,6 +256,18 @@ test('youtube flow announces manual picker opening before probing tracks', async
|
||||
assertNoProblems(harness);
|
||||
});
|
||||
|
||||
test('youtube flow widens halfwidth katakana in the refreshed subtitle', async () => {
|
||||
const harness = createFlowHarness({
|
||||
mpv: { subText: 'アンタがアータルたん?' },
|
||||
probeTracks: [primaryTrack],
|
||||
pick: { primaryTrackId: primaryTrack.id, secondaryTrackId: null },
|
||||
});
|
||||
|
||||
await harness.runtime.openManualPicker({ url: 'https://example.com' });
|
||||
|
||||
assert.deepEqual(harness.refreshedSubtitles, ['アンタがアータルたん?']);
|
||||
});
|
||||
|
||||
test('youtube flow can open a manual picker session and load the selected subtitles', async () => {
|
||||
const harness = createFlowHarness({
|
||||
probeTracks: [primaryTrack, secondaryTrack],
|
||||
|
||||
@@ -21,6 +21,7 @@ import {
|
||||
} from '../../core/services/youtube/generate';
|
||||
import { resolveSubtitleSourcePath } from './subtitle-prefetch-source';
|
||||
import { extractYoutubeVideoId } from './youtube-playback';
|
||||
import { widenHalfwidthKana } from '../../core/utils/halfwidth-kana';
|
||||
|
||||
type YoutubeFlowOpenPicker = (payload: YoutubePickerOpenPayload) => Promise<boolean>;
|
||||
|
||||
@@ -457,7 +458,7 @@ async function injectDownloadedSubtitles(
|
||||
|
||||
const currentSubText = await deps.requestMpvProperty('sub-text').catch(() => null);
|
||||
if (typeof currentSubText === 'string' && currentSubText.trim().length > 0) {
|
||||
deps.refreshCurrentSubtitle(currentSubText);
|
||||
deps.refreshCurrentSubtitle(widenHalfwidthKana(currentSubText));
|
||||
}
|
||||
|
||||
deps.showMpvOsd(secondaryTrack ? 'Primary and secondary subtitles loaded.' : 'Subtitles loaded.');
|
||||
|
||||
@@ -88,7 +88,7 @@ export function createSubtitleGenerationModal(
|
||||
const vad = snapshot ? describeGenerationVad(snapshot.vad) : null;
|
||||
const tools = snapshot ? describeGenerationTools(snapshot.tools) : null;
|
||||
const readyMessage = !snapshot?.mediaPath
|
||||
? 'Open a local file or a YouTube video to generate subtitles.'
|
||||
? 'Open a local file, a Jellyfin stream, or a YouTube video to generate subtitles.'
|
||||
: !tools?.ready
|
||||
? 'Install the missing tools or set their paths in Settings, then click Check again.'
|
||||
: !model?.ready
|
||||
@@ -97,7 +97,8 @@ export function createSubtitleGenerationModal(
|
||||
? 'Download the speech detection model or uncheck Focus on spoken dialogue.'
|
||||
: 'Ready when you are.';
|
||||
dom.media.textContent =
|
||||
snapshot?.mediaPath ?? 'Open a local file or a YouTube video in the player first.';
|
||||
snapshot?.mediaPath ??
|
||||
'Open a local file, a Jellyfin stream, or a YouTube video in the player first.';
|
||||
dom.tools.textContent = tools?.text ?? 'Checking local tools...';
|
||||
dom.model.textContent = model?.text ?? 'Checking local models...';
|
||||
dom.modelPicker.classList.toggle('hidden', !snapshot || Boolean(snapshot.externalModelPath));
|
||||
|
||||
Reference in new issue
Block a user