mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-08-05 07:21:34 -07:00
Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
c305bf34c2
|
|||
|
64534299ed
|
@@ -0,0 +1,5 @@
|
|||||||
|
type: fixed
|
||||||
|
area: subtitles
|
||||||
|
|
||||||
|
- Heavily typeset ASS scripts (karaoke OP/ED, sign work) no longer fill the subtitle sidebar with garbage. Vector drawing runs (`\p1` … `\p0`) are no longer shown as subtitle text (e.g. `m 20 0 b 10 0 0 10 0 20 …`), and duplicate events in a parsed subtitle file collapse into one cue: identical text over an identical span (layered "shadow" copies), and per-frame animation bursts. An ASS burst has to prove itself with authoring evidence — a temporal tag (`\t`, `\move`, karaoke timing), an animated `Effect` column, or override values that change from event to event — plus one shared style and actor, so three rapid `えっ` reactions from three characters, or a sign repeated with the same static `\clip`, stay separate. SRT and VTT carry no such metadata, so there the run has to be at least five contiguous events all shorter than 0.1s, which is where ASS-to-SRT conversion leaves karaoke frames.
|
||||||
|
- Subtitle text is now decoded from ASS exactly once, where it enters the app, and matches how mpv renders the same line (including `\N`, `\n`, `\h`, unclosed `{`, and the fact that `\{` is not an escape). Renderer, timing tracker, tokenizer and the tokenization cache take that decoded text as-is instead of each re-deriving it, so one authored line can no longer produce two different cache keys, and a cue that normalizes to nothing is no longer stored as subtitle text or cached under an empty key.
|
||||||
@@ -64,18 +64,23 @@ External subtitle files only (SRT, VTT, ASS). Embedded subtitle tracks are out o
|
|||||||
A cue parser extracts both timing and text content from subtitle files for prefetching.
|
A cue parser extracts both timing and text content from subtitle files for prefetching.
|
||||||
|
|
||||||
**Parsed cue structure:**
|
**Parsed cue structure:**
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface SubtitleCue {
|
interface SubtitleCue {
|
||||||
startTime: number; // seconds
|
startTime: number; // seconds
|
||||||
endTime: number; // seconds
|
endTime: number; // seconds
|
||||||
text: string; // raw subtitle text
|
text: string; // plain text, decoded from the source format
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
**Supported formats:**
|
**Supported formats:**
|
||||||
|
|
||||||
- SRT/VTT: Regex-based parsing of timing lines + text content between timing blocks.
|
- SRT/VTT: Regex-based parsing of timing lines + text content between timing blocks.
|
||||||
- ASS: Parse `[Events]` section, extract `Dialogue:` lines, split on the first 9 commas only (ASS v4+ has 10 fields; the last field is Text which can itself contain commas). Strip ASS override tags (`{\...}`) from the text before storing.
|
- ASS: Parse `[Events]` section, extract `Dialogue:` lines, read the field order from the `Format:` row, and take everything after the Text field index as the text (Text can itself contain commas).
|
||||||
ASS text fields contain inline override tags like `{\b1}`, `{\an8}`, `{\fad(200,300)}`. The cue parser strips these during extraction so the tokenizer receives clean text.
|
|
||||||
|
**ASS decoding.** The parser is where ASS text is decoded, once, via `assToPlainText()` in `src/core/services/ass-text.ts`. That decoder mirrors mpv's `ass_to_plaintext` so a cue read from a file reads identically to the same line arriving live on `sub-text`: `{...}` override blocks are markup, `\pN … \p0` vector drawing runs are dropped rather than shown as text, `\N`/`\n`/`\h` are the only escapes (`\{`, `\}` and `\\` are not), and an unclosed `{` is rendered verbatim. Every layer downstream — renderer, timing tracker, tokenizer, tokenization cache keys — receives plain text and uses `normalizePlainSubtitleText()` for whitespace only, so nothing decodes the same string twice and one authored line always maps to one cache key.
|
||||||
|
|
||||||
|
**Duplicate collapsing.** Typeset scripts emit one `Dialogue:` event per animation frame, plus layered copies of the same line. The parser collapses identical text over an identical span unconditionally, and collapses contiguous same-text runs of at least three events when the run looks like an animation. For ASS that means shared style and actor plus authoring evidence: a temporal tag (`\t`, `\move`, `\k`/`\kf`/`\ko`/`\K`, or anything wrapped in `\t(...)`), an animated `Effect` column (`Karaoke`, `Banner`, `Scroll`), or override values that change across the run. Static tags shared by every event (`\pos`, an identical `\clip`) are not evidence. SRT/VTT carry no such metadata, so there collapsing needs at least five contiguous events all under 0.1s — the frame timing left behind by ASS-to-SRT conversion. The parser keeps this authoring metadata (style, actor, layer, `Effect`, parsed override commands, source order) private; `parseSubtitleCues()` returns only `SubtitleCue`.
|
||||||
|
|
||||||
#### Prefetch Service Lifecycle
|
#### Prefetch Service Lifecycle
|
||||||
|
|
||||||
@@ -153,6 +158,7 @@ tokens (already have frequencyRank values from parser-level applyFrequencyRanks)
|
|||||||
### Dependency Analysis
|
### Dependency Analysis
|
||||||
|
|
||||||
All annotations either depend on MeCab POS data or benefit from running after it:
|
All annotations either depend on MeCab POS data or benefit from running after it:
|
||||||
|
|
||||||
- **Known word marking:** Needs base tokens (surface/headword). No POS dependency, but no reason to run separately.
|
- **Known word marking:** Needs base tokens (surface/headword). No POS dependency, but no reason to run separately.
|
||||||
- **Frequency filtering:** Uses `pos1Exclusions` and `pos2Exclusions` to clear frequency ranks on excluded tokens (particles, noise). Depends on MeCab POS data.
|
- **Frequency filtering:** Uses `pos1Exclusions` and `pos2Exclusions` to clear frequency ranks on excluded tokens (particles, noise). Depends on MeCab POS data.
|
||||||
- **JLPT marking:** Uses `shouldIgnoreJlptForMecabPos1` to filter. Depends on MeCab POS data.
|
- **JLPT marking:** Uses `shouldIgnoreJlptForMecabPos1` to filter. Depends on MeCab POS data.
|
||||||
@@ -169,18 +175,14 @@ function annotateTokens(tokens, deps, options): MergedToken[] {
|
|||||||
|
|
||||||
// Single pass: known word + frequency filtering + JLPT computed together
|
// Single pass: known word + frequency filtering + JLPT computed together
|
||||||
const annotated = tokens.map((token) => {
|
const annotated = tokens.map((token) => {
|
||||||
const isKnown = nPlusOneEnabled
|
const isKnown = nPlusOneEnabled ? token.isKnown || computeIsKnown(token, deps) : false;
|
||||||
? token.isKnown || computeIsKnown(token, deps)
|
|
||||||
: false;
|
|
||||||
|
|
||||||
// Filter frequency rank using POS exclusions (rank values already set at parser level)
|
// Filter frequency rank using POS exclusions (rank values already set at parser level)
|
||||||
const frequencyRank = frequencyEnabled
|
const frequencyRank = frequencyEnabled
|
||||||
? filterFrequencyRank(token, pos1Exclusions, pos2Exclusions)
|
? filterFrequencyRank(token, pos1Exclusions, pos2Exclusions)
|
||||||
: undefined;
|
: undefined;
|
||||||
|
|
||||||
const jlptLevel = jlptEnabled
|
const jlptLevel = jlptEnabled ? computeJlptLevel(token, deps.getJlptLevel) : undefined;
|
||||||
? computeJlptLevel(token, deps.getJlptLevel)
|
|
||||||
: undefined;
|
|
||||||
|
|
||||||
return { ...token, isKnown, frequencyRank, jlptLevel };
|
return { ...token, isKnown, frequencyRank, jlptLevel };
|
||||||
});
|
});
|
||||||
@@ -221,6 +223,7 @@ Replace `document.createElement('span')` calls in the renderer with `templateSpa
|
|||||||
### Current Behavior
|
### Current Behavior
|
||||||
|
|
||||||
In `renderWithTokens` (`subtitle-render.ts`), each render cycle:
|
In `renderWithTokens` (`subtitle-render.ts`), each render cycle:
|
||||||
|
|
||||||
1. Clears DOM with `innerHTML = ''`
|
1. Clears DOM with `innerHTML = ''`
|
||||||
2. Creates a `DocumentFragment`
|
2. Creates a `DocumentFragment`
|
||||||
3. Calls `document.createElement('span')` for each token (~10-15 per subtitle)
|
3. Calls `document.createElement('span')` for each token (~10-15 per subtitle)
|
||||||
@@ -257,7 +260,7 @@ Full recycling (collecting old nodes, clearing attributes, reusing them) require
|
|||||||
## Combined Impact Summary
|
## Combined Impact Summary
|
||||||
|
|
||||||
| Scenario | Before | After | Improvement |
|
| Scenario | Before | After | Improvement |
|
||||||
|----------|--------|-------|-------------|
|
| --------------------------------- | ---------- | ---------- | ----------- |
|
||||||
| Normal playback (prefetch-warmed) | ~200-320ms | ~30-50ms | ~80-85% |
|
| Normal playback (prefetch-warmed) | ~200-320ms | ~30-50ms | ~80-85% |
|
||||||
| Cache hit (repeated subtitle) | ~72ms | ~55-65ms | ~10-20% |
|
| Cache hit (repeated subtitle) | ~72ms | ~55-65ms | ~10-20% |
|
||||||
| Cache miss (immediate seek) | ~200-320ms | ~150-260ms | ~20-25% |
|
| Cache miss (immediate seek) | ~200-320ms | ~150-260ms | ~20-25% |
|
||||||
@@ -267,16 +270,19 @@ Full recycling (collecting old nodes, clearing attributes, reusing them) require
|
|||||||
## Files Summary
|
## Files Summary
|
||||||
|
|
||||||
### New Files
|
### New Files
|
||||||
|
|
||||||
- `src/core/services/subtitle-prefetch.ts`
|
- `src/core/services/subtitle-prefetch.ts`
|
||||||
- `src/core/services/subtitle-cue-parser.ts`
|
- `src/core/services/subtitle-cue-parser.ts`
|
||||||
|
|
||||||
### Modified Files
|
### Modified Files
|
||||||
|
|
||||||
- `src/core/services/subtitle-processing-controller.ts` (expose `preCacheTokenization`)
|
- `src/core/services/subtitle-processing-controller.ts` (expose `preCacheTokenization`)
|
||||||
- `src/core/services/tokenizer/annotation-stage.ts` (batched single-pass)
|
- `src/core/services/tokenizer/annotation-stage.ts` (batched single-pass)
|
||||||
- `src/renderer/subtitle-render.ts` (template cloneNode)
|
- `src/renderer/subtitle-render.ts` (template cloneNode)
|
||||||
- `src/main.ts` (wire up prefetch service)
|
- `src/main.ts` (wire up prefetch service)
|
||||||
|
|
||||||
### Test Files
|
### Test Files
|
||||||
|
|
||||||
- New tests for subtitle cue parser (SRT, VTT, ASS formats)
|
- New tests for subtitle cue parser (SRT, VTT, ASS formats)
|
||||||
- New tests for subtitle prefetch service (priority window, seek, pause/resume)
|
- New tests for subtitle prefetch service (priority window, seek, pause/resume)
|
||||||
- Updated tests for annotation stage (same behavior, new implementation)
|
- Updated tests for annotation stage (same behavior, new implementation)
|
||||||
|
|||||||
@@ -0,0 +1,193 @@
|
|||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import {
|
||||||
|
assOverrideSignature,
|
||||||
|
assToPlainText,
|
||||||
|
collectAssOverrideCommands,
|
||||||
|
extractAssOverrideBlocks,
|
||||||
|
hasAssTemporalOverride,
|
||||||
|
isAnimatedAssEffectKind,
|
||||||
|
isAssTemporalCommand,
|
||||||
|
normalizePlainSubtitleText,
|
||||||
|
parseAssEffectField,
|
||||||
|
} from './ass-text';
|
||||||
|
|
||||||
|
test('assToPlainText drops vector drawing runs', () => {
|
||||||
|
assert.equal(
|
||||||
|
assToPlainText(
|
||||||
|
'{\\an5\\pos(730,1042)\\p1\\blur1}m 20 0 b 10 0 0 10 0 20 b 0 31 10 40 20 40 {\\p0}',
|
||||||
|
),
|
||||||
|
'',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText keeps text around drawing runs on the same event', () => {
|
||||||
|
assert.equal(
|
||||||
|
assToPlainText('{\\p1}m 0 0 l 10 10{\\p0}本文{\\p1}m 5 5 l 6 6{\\p0}続き'),
|
||||||
|
'本文続き',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText leaves \\pos alone when no drawing mode is active', () => {
|
||||||
|
assert.equal(assToPlainText('{\\pos(960,1068)\\bord3}位置指定'), '位置指定');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText does not read \\pos as a drawing tag', () => {
|
||||||
|
assert.equal(assToPlainText('{\\p1\\pos(1,2)}m 0 0 l 5 5'), '');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText resolves line-break and space escapes', () => {
|
||||||
|
assert.equal(assToPlainText('一行目\\N二行目'), '一行目\n二行目');
|
||||||
|
assert.equal(assToPlainText('一行目\\n二行目'), '一行目\n二行目');
|
||||||
|
assert.equal(assToPlainText('一行目\\N二行目', ' '), '一行目 二行目');
|
||||||
|
assert.equal(assToPlainText('間\\h隔'), '間 隔');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText matches mpv on brace and backslash sequences', () => {
|
||||||
|
// mpv has no `\{` / `\}` / `\\` escapes: the backslashes are literal text and the
|
||||||
|
// braces still open and close an override block.
|
||||||
|
assert.equal(assToPlainText('\\{注\\}'), '\\');
|
||||||
|
assert.equal(assToPlainText('\\\\N'), '\\\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText renders an unclosed override block verbatim', () => {
|
||||||
|
// mpv shows the stray brace; guessing where the block ended can eat a whole line.
|
||||||
|
assert.equal(assToPlainText('本文{\\pos(1,2)'), '本文{\\pos(1,2)');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText is idempotent', () => {
|
||||||
|
const samples = [
|
||||||
|
'{\\an5\\p1}m 0 0 l 5 5{\\p0}本文',
|
||||||
|
'\\{注\\}',
|
||||||
|
'\\\\N',
|
||||||
|
'本文{\\pos(1,2)',
|
||||||
|
'一行目\\N二行目\\h終わり',
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const sample of samples) {
|
||||||
|
const once = assToPlainText(sample);
|
||||||
|
assert.equal(assToPlainText(once), once, sample);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assToPlainText normalizes CRLF before converting', () => {
|
||||||
|
assert.equal(assToPlainText('一行目\r\n二行目'), '一行目\n二行目');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('normalizePlainSubtitleText settles whitespace without decoding ASS', () => {
|
||||||
|
// A brace reaching this layer is literal text mpv chose to show, not markup.
|
||||||
|
assert.equal(normalizePlainSubtitleText('本文{\\pos(1,2)'), '本文{\\pos(1,2)');
|
||||||
|
assert.equal(normalizePlainSubtitleText('一行目\\N二行目'), '一行目\n二行目');
|
||||||
|
assert.equal(
|
||||||
|
normalizePlainSubtitleText('一行目\\N二行目', { collapseLineBreaks: true }),
|
||||||
|
'一行目 二行目',
|
||||||
|
);
|
||||||
|
assert.equal(normalizePlainSubtitleText(' 余白 ', { trim: false }), ' 余白 ');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('normalizePlainSubtitleText is idempotent', () => {
|
||||||
|
for (const sample of ['一行目\\N二行目', '間\\h隔', '本文{\\pos(1,2)', ' 余白 ']) {
|
||||||
|
const once = normalizePlainSubtitleText(sample);
|
||||||
|
assert.equal(normalizePlainSubtitleText(once), once, sample);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('extractAssOverrideBlocks returns block contents', () => {
|
||||||
|
assert.deepEqual(extractAssOverrideBlocks('{\\an8}上{\\fad(200,200)}下'), [
|
||||||
|
'\\an8',
|
||||||
|
'\\fad(200,200)',
|
||||||
|
]);
|
||||||
|
assert.deepEqual(extractAssOverrideBlocks('括弧なし'), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('collectAssOverrideCommands captures names and arguments from blocks only', () => {
|
||||||
|
const commands = collectAssOverrideCommands('{\\pos(1,2)\\1c&HFFFFFF&\\kf30}歌詞');
|
||||||
|
|
||||||
|
assert.deepEqual(commands, [
|
||||||
|
{ name: 'pos', args: '1,2', animated: false },
|
||||||
|
{ name: '1c', args: '&HFFFFFF&', animated: false },
|
||||||
|
{ name: 'kf', args: '30', animated: false },
|
||||||
|
]);
|
||||||
|
|
||||||
|
// A `\pos(...)` sitting in visible text is not typesetting markup.
|
||||||
|
assert.deepEqual(collectAssOverrideCommands('\\pos(730,1042) と書いてある'), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('collectAssOverrideCommands marks tags animated by a wrapping \\t', () => {
|
||||||
|
const commands = collectAssOverrideCommands('{\\clip(0,0,10,10)\\t(0,500,\\frz30)}文字');
|
||||||
|
|
||||||
|
assert.deepEqual(
|
||||||
|
commands.map((command) => [command.name, command.animated]),
|
||||||
|
[
|
||||||
|
['clip', false],
|
||||||
|
['t', false],
|
||||||
|
['frz', true],
|
||||||
|
],
|
||||||
|
);
|
||||||
|
assert.equal(hasAssTemporalOverride(commands), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('collectAssOverrideCommands survives pathologically nested \\t tags', () => {
|
||||||
|
const depth = 200000;
|
||||||
|
const block = `{${'\\t(0,500,'.repeat(depth)}\\frz30${')'.repeat(depth)}}文字`;
|
||||||
|
|
||||||
|
const commands = collectAssOverrideCommands(block);
|
||||||
|
|
||||||
|
// Recursion stops at the nesting cap; the outer tags are still reported, and nothing
|
||||||
|
// blows the call stack.
|
||||||
|
assert.equal(commands[0]!.name, 't');
|
||||||
|
assert.equal(hasAssTemporalOverride(commands), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('hasAssTemporalOverride ignores static placement and shape tags', () => {
|
||||||
|
assert.equal(
|
||||||
|
hasAssTemporalOverride(collectAssOverrideCommands('{\\pos(1,2)\\clip(m 1 1)\\blur2}文字')),
|
||||||
|
false,
|
||||||
|
);
|
||||||
|
assert.equal(hasAssTemporalOverride(collectAssOverrideCommands('{\\move(1,2,3,4)}文字')), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('isAssTemporalCommand covers only intrinsically animated tags', () => {
|
||||||
|
for (const command of ['t', 'move', 'k', 'kf', 'ko', 'K']) {
|
||||||
|
assert.equal(isAssTemporalCommand(command), true, command);
|
||||||
|
}
|
||||||
|
for (const command of ['clip', 'iclip', 'frz', 'fscx', 'blur', 'be', 'pos', 'fad']) {
|
||||||
|
assert.equal(isAssTemporalCommand(command), false, command);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('assOverrideSignature distinguishes events by their override values', () => {
|
||||||
|
const first = assOverrideSignature(collectAssOverrideCommands('{\\clip(m 1 1)}歌詞'));
|
||||||
|
const second = assOverrideSignature(collectAssOverrideCommands('{\\clip(m 2 2)}歌詞'));
|
||||||
|
const repeat = assOverrideSignature(collectAssOverrideCommands('{\\clip(m 1 1)}別の行'));
|
||||||
|
|
||||||
|
assert.notEqual(first, second);
|
||||||
|
assert.equal(first, repeat);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseAssEffectField classifies the event-level Effect column', () => {
|
||||||
|
assert.equal(parseAssEffectField(''), 'none');
|
||||||
|
assert.equal(parseAssEffectField(' '), 'none');
|
||||||
|
assert.equal(parseAssEffectField('Banner;20;1;0'), 'banner');
|
||||||
|
assert.equal(parseAssEffectField('Scroll up;0;0;30;10'), 'scroll');
|
||||||
|
assert.equal(parseAssEffectField('Scroll down;0;0;30;10'), 'scroll');
|
||||||
|
assert.equal(parseAssEffectField('Karaoke'), 'karaoke');
|
||||||
|
assert.equal(parseAssEffectField('fx-template'), 'other');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseAssEffectField matches stock effect names exactly', () => {
|
||||||
|
// Custom effect names that merely start with a stock name are not stock effects.
|
||||||
|
assert.equal(parseAssEffectField('scrolling-credit'), 'other');
|
||||||
|
assert.equal(parseAssEffectField('bannerfx;1'), 'other');
|
||||||
|
assert.equal(parseAssEffectField('karaoke-template'), 'other');
|
||||||
|
assert.equal(parseAssEffectField('Scroll'), 'other');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('isAnimatedAssEffectKind covers the stock animated effects only', () => {
|
||||||
|
assert.equal(isAnimatedAssEffectKind('karaoke'), true);
|
||||||
|
assert.equal(isAnimatedAssEffectKind('banner'), true);
|
||||||
|
assert.equal(isAnimatedAssEffectKind('scroll'), true);
|
||||||
|
// Typesetting groups put static template names in this column too.
|
||||||
|
assert.equal(isAnimatedAssEffectKind('other'), false);
|
||||||
|
assert.equal(isAnimatedAssEffectKind('none'), false);
|
||||||
|
});
|
||||||
@@ -0,0 +1,280 @@
|
|||||||
|
/*
|
||||||
|
* ASS/SSA text handling, split into two deliberately distinct contracts:
|
||||||
|
*
|
||||||
|
* assToPlainText() raw ASS event text -> plain text. Ingestion only.
|
||||||
|
* normalizePlainSubtitleText() already-decoded text -> display/lookup form.
|
||||||
|
*
|
||||||
|
* Subtitle text is decoded from ASS exactly once, at the point it enters the app: the
|
||||||
|
* file cue parser does it for sidecar/embedded scripts, and mpv does it for live text
|
||||||
|
* (`sub-text` is already run through mpv's own `ass_to_plaintext`). Everything
|
||||||
|
* downstream -- renderer, timing tracker, tokenizer, tokenization cache keys -- gets
|
||||||
|
* plain text and only normalizes whitespace, so no layer decodes the same string twice.
|
||||||
|
*
|
||||||
|
* assToPlainText mirrors mpv's `ass_to_plaintext` rather than inventing its own rules,
|
||||||
|
* so a cue parsed from a file reads the same as the same line arriving live:
|
||||||
|
* - `{...}` override blocks are markup
|
||||||
|
* - `\pN ... \p0` runs are vector paths, not dialogue
|
||||||
|
* - `\N`, `\n` and `\h` are the only escapes; `\{`, `\}` and `\\` are NOT escapes,
|
||||||
|
* so `\{注\}` decodes to a lone backslash exactly as mpv renders it
|
||||||
|
* - an unclosed `{` is rendered verbatim instead of swallowing the rest of the line
|
||||||
|
* Because the decoder never emits an escape or a closed brace, running it twice is a
|
||||||
|
* no-op -- but downstream code should still use normalizePlainSubtitleText.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** What `\N` and `\n` become. */
|
||||||
|
export type AssLineBreak = '\n' | ' ';
|
||||||
|
|
||||||
|
// `\p<n>` with n > 0 switches libass into vector-drawing mode: everything until the
|
||||||
|
// next `\p0` is a path (`m 20 0 b 10 0 ...`), not dialogue. The negative lookahead keeps
|
||||||
|
// `\pos(...)` from being read as a drawing tag.
|
||||||
|
const ASS_DRAWING_SCALE_PATTERN = /\\p(?![a-zA-Z])(\d*)/g;
|
||||||
|
|
||||||
|
function readDrawingScale(block: string): number | null {
|
||||||
|
ASS_DRAWING_SCALE_PATTERN.lastIndex = 0;
|
||||||
|
let scale: number | null = null;
|
||||||
|
let match: RegExpExecArray | null;
|
||||||
|
// Drawing mode is whatever the last `\p` tag in this block set it to.
|
||||||
|
while ((match = ASS_DRAWING_SCALE_PATTERN.exec(block)) !== null) {
|
||||||
|
scale = match[1] ? Number(match[1]) : 0;
|
||||||
|
}
|
||||||
|
return scale;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolve `\N`, `\n` and `\h`. The only text-level escapes libass recognises. */
|
||||||
|
function resolveWhitespaceEscapes(text: string, lineBreak: AssLineBreak): string {
|
||||||
|
return text.replace(/\\([Nnh])/g, (_match, escaped: string) =>
|
||||||
|
escaped === 'h' ? ' ' : lineBreak,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Strip `{...}` override blocks and the drawing runs they enable. */
|
||||||
|
function stripAssMarkup(raw: string): string {
|
||||||
|
let out = '';
|
||||||
|
let cursor = 0;
|
||||||
|
let drawing = false;
|
||||||
|
|
||||||
|
while (cursor < raw.length) {
|
||||||
|
if (raw[cursor] !== '{') {
|
||||||
|
if (!drawing) {
|
||||||
|
out += raw[cursor];
|
||||||
|
}
|
||||||
|
cursor += 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const close = raw.indexOf('}', cursor + 1);
|
||||||
|
if (close === -1) {
|
||||||
|
// mpv shows an unclosed `{` and everything after it. Guessing where the block was
|
||||||
|
// meant to end can eat a whole line of dialogue.
|
||||||
|
if (!drawing) {
|
||||||
|
out += raw.slice(cursor);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
const scale = readDrawingScale(raw.slice(cursor, close + 1));
|
||||||
|
if (scale !== null) {
|
||||||
|
drawing = scale > 0;
|
||||||
|
}
|
||||||
|
cursor = close + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decode a raw ASS/SSA event text field. Call this once, where the text enters the app;
|
||||||
|
* downstream layers take the result as plain text.
|
||||||
|
*/
|
||||||
|
export function assToPlainText(text: string, lineBreak: AssLineBreak = '\n'): string {
|
||||||
|
if (!text) return '';
|
||||||
|
return resolveWhitespaceEscapes(stripAssMarkup(text.replace(/\r\n/g, '\n')), lineBreak);
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface NormalizePlainSubtitleTextOptions {
|
||||||
|
/** Fold every line break into a single space. */
|
||||||
|
collapseLineBreaks?: boolean;
|
||||||
|
trim?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whitespace normalization for text that has already been decoded -- by mpv for live
|
||||||
|
* subtitles, by the cue parser for files. Override blocks and drawing runs are none of
|
||||||
|
* this function's business; a `{` that reaches here is literal text mpv chose to show.
|
||||||
|
*
|
||||||
|
* `\N`/`\n`/`\h` are still folded, because subtitle sources outside the ASS path (asbplayer
|
||||||
|
* and other websocket clients) forward them raw and the display layer has to cope.
|
||||||
|
*/
|
||||||
|
export function normalizePlainSubtitleText(
|
||||||
|
text: string,
|
||||||
|
options: NormalizePlainSubtitleTextOptions = {},
|
||||||
|
): string {
|
||||||
|
if (!text) return '';
|
||||||
|
const { collapseLineBreaks = false, trim = true } = options;
|
||||||
|
|
||||||
|
let normalized = resolveWhitespaceEscapes(
|
||||||
|
text.replace(/\r\n/g, '\n'),
|
||||||
|
collapseLineBreaks ? ' ' : '\n',
|
||||||
|
);
|
||||||
|
if (collapseLineBreaks) {
|
||||||
|
normalized = normalized.replace(/\n/g, ' ').replace(/\s+/g, ' ');
|
||||||
|
}
|
||||||
|
|
||||||
|
return trim ? normalized.trim() : normalized;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The contents of each `{...}` block, without the braces. */
|
||||||
|
export function extractAssOverrideBlocks(text: string): string[] {
|
||||||
|
const blocks: string[] = [];
|
||||||
|
let cursor = 0;
|
||||||
|
|
||||||
|
while (cursor < text.length) {
|
||||||
|
const open = text.indexOf('{', cursor);
|
||||||
|
if (open === -1) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
const close = text.indexOf('}', open + 1);
|
||||||
|
if (close === -1) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
blocks.push(text.slice(open + 1, close));
|
||||||
|
cursor = close + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
return blocks;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface AssOverrideCommand {
|
||||||
|
/** Tag name without the backslash, e.g. `pos`, `kf`, `1c`. */
|
||||||
|
name: string;
|
||||||
|
/** Everything the tag was given, e.g. `960,1068` for `\pos(960,1068)`. */
|
||||||
|
args: string;
|
||||||
|
/** Nested inside a `\t(...)` argument, so its value is animated over the event. */
|
||||||
|
animated: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ASS_OVERRIDE_NAME_PATTERN = /[1-4]?[a-zA-Z]+/y;
|
||||||
|
|
||||||
|
function readCommandArgs(block: string, start: number): { args: string; next: number } {
|
||||||
|
if (block[start] === '(') {
|
||||||
|
let depth = 0;
|
||||||
|
for (let i = start; i < block.length; i += 1) {
|
||||||
|
if (block[i] === '(') depth += 1;
|
||||||
|
else if (block[i] === ')') {
|
||||||
|
depth -= 1;
|
||||||
|
if (depth === 0) {
|
||||||
|
return { args: block.slice(start + 1, i), next: i + 1 };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { args: block.slice(start + 1), next: block.length };
|
||||||
|
}
|
||||||
|
|
||||||
|
const nextTag = block.indexOf('\\', start);
|
||||||
|
const end = nextTag === -1 ? block.length : nextTag;
|
||||||
|
return { args: block.slice(start, end), next: end };
|
||||||
|
}
|
||||||
|
|
||||||
|
// `\t(...)` can wrap another `\t(...)`, and nothing in the format stops an author (or a
|
||||||
|
// malformed file) from nesting them thousands deep. Real typesetting never goes past one
|
||||||
|
// or two levels, so stop recursing well before the call stack is at risk.
|
||||||
|
const MAX_ANIMATION_NESTING_DEPTH = 8;
|
||||||
|
|
||||||
|
function parseOverrideBlock(
|
||||||
|
block: string,
|
||||||
|
animated: boolean,
|
||||||
|
into: AssOverrideCommand[],
|
||||||
|
depth = 0,
|
||||||
|
): void {
|
||||||
|
let cursor = 0;
|
||||||
|
|
||||||
|
while (cursor < block.length) {
|
||||||
|
if (block[cursor] !== '\\') {
|
||||||
|
cursor += 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
ASS_OVERRIDE_NAME_PATTERN.lastIndex = cursor + 1;
|
||||||
|
const nameMatch = ASS_OVERRIDE_NAME_PATTERN.exec(block);
|
||||||
|
if (!nameMatch) {
|
||||||
|
cursor += 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const name = nameMatch[0];
|
||||||
|
const { args, next } = readCommandArgs(block, cursor + 1 + name.length);
|
||||||
|
into.push({ name, args: args.trim(), animated });
|
||||||
|
// `\t(0,500,\frz30)` animates whatever it wraps, so record the inner tags too.
|
||||||
|
if (name === 't' && args.includes('\\') && depth < MAX_ANIMATION_NESTING_DEPTH) {
|
||||||
|
parseOverrideBlock(args, true, into, depth + 1);
|
||||||
|
}
|
||||||
|
cursor = next;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Override commands with their arguments, in source order. Only `{...}` blocks are
|
||||||
|
* inspected, so a `\pos(...)` sitting in visible text is never mistaken for markup.
|
||||||
|
*/
|
||||||
|
export function collectAssOverrideCommands(text: string): AssOverrideCommand[] {
|
||||||
|
const commands: AssOverrideCommand[] = [];
|
||||||
|
for (const block of extractAssOverrideBlocks(text)) {
|
||||||
|
parseOverrideBlock(block, false, commands);
|
||||||
|
}
|
||||||
|
return commands;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tags that are animated by definition: `\t` interpolates, `\move` travels, and the
|
||||||
|
// karaoke tags advance a highlight across the event's own duration. Everything else --
|
||||||
|
// `\pos`, `\clip`, `\frz`, `\blur`, `\fad` -- is a static value for the event, so its
|
||||||
|
// presence says nothing about whether neighbouring events form one animation.
|
||||||
|
const ASS_TEMPORAL_COMMANDS = new Set(['t', 'move', 'k', 'kf', 'ko', 'K']);
|
||||||
|
|
||||||
|
export function isAssTemporalCommand(name: string): boolean {
|
||||||
|
return ASS_TEMPORAL_COMMANDS.has(name);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** True when the event animates on its own, or animates a static tag through `\t(...)`. */
|
||||||
|
export function hasAssTemporalOverride(commands: readonly AssOverrideCommand[]): boolean {
|
||||||
|
return commands.some((command) => command.animated || isAssTemporalCommand(command.name));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Canonical form of an event's override values, for comparing consecutive events. Two
|
||||||
|
* events with the same signature were typeset identically, so neither is a frame of an
|
||||||
|
* animation the other belongs to.
|
||||||
|
*/
|
||||||
|
export function assOverrideSignature(commands: readonly AssOverrideCommand[]): string {
|
||||||
|
return commands.map((command) => `${command.name}(${command.args})`).join('|');
|
||||||
|
}
|
||||||
|
|
||||||
|
export type AssEffectKind = 'none' | 'banner' | 'scroll' | 'karaoke' | 'other';
|
||||||
|
|
||||||
|
// The stock effects, matched exactly. Typesetting groups put their own template names in
|
||||||
|
// this column -- `scrolling-credit` is a static sign, not libass's `Scroll up` -- so a
|
||||||
|
// prefix match would hand out animation evidence to arbitrary custom effects.
|
||||||
|
const STOCK_ASS_EFFECTS = new Map<string, AssEffectKind>([
|
||||||
|
['banner', 'banner'],
|
||||||
|
['scroll up', 'scroll'],
|
||||||
|
['scroll down', 'scroll'],
|
||||||
|
['karaoke', 'karaoke'],
|
||||||
|
]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The event-level `Effect` column. The stock values (`Banner;...`, `Scroll up;...`,
|
||||||
|
* `Scroll down;...`, `Karaoke`) all animate; anything else is a custom name and lands in
|
||||||
|
* `other`.
|
||||||
|
*/
|
||||||
|
export function parseAssEffectField(raw: string): AssEffectKind {
|
||||||
|
const value = raw.trim().toLowerCase();
|
||||||
|
if (!value) return 'none';
|
||||||
|
|
||||||
|
const name = value.split(';', 1)[0]!.trim();
|
||||||
|
return STOCK_ASS_EFFECTS.get(name) ?? 'other';
|
||||||
|
}
|
||||||
|
|
||||||
|
const ANIMATED_ASS_EFFECT_KINDS = new Set<AssEffectKind>(['banner', 'scroll', 'karaoke']);
|
||||||
|
|
||||||
|
export function isAnimatedAssEffectKind(kind: AssEffectKind): boolean {
|
||||||
|
return ANIMATED_ASS_EFFECT_KINDS.has(kind);
|
||||||
|
}
|
||||||
@@ -0,0 +1,185 @@
|
|||||||
|
/*
|
||||||
|
* Duplicate/animation-burst collapsing for parsed subtitle cues.
|
||||||
|
*
|
||||||
|
* Split out of the cue parser so the parsing rules and the "is this run one animation?"
|
||||||
|
* heuristics can be read -- and tested -- on their own. The parser owns the cue shape;
|
||||||
|
* this module only decides which cues survive.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { hasAssTemporalOverride, isAnimatedAssEffectKind } from './ass-text';
|
||||||
|
import type {
|
||||||
|
AnnotatedSubtitleCue,
|
||||||
|
SubtitleCue,
|
||||||
|
SubtitleSourceFormat,
|
||||||
|
} from './subtitle-cue-parser';
|
||||||
|
|
||||||
|
// Back-to-back frames of the same animation are authored flush against each other; a
|
||||||
|
// tiny tolerance absorbs the centisecond rounding of the ASS timestamp format.
|
||||||
|
const DUPLICATE_CUE_GAP_TOLERANCE_SECONDS = 0.05;
|
||||||
|
// A burst is a *sequence*. Two adjacent events are two events, not an animation --
|
||||||
|
// characters do repeat each other, and a repeated line can legitimately be short.
|
||||||
|
const MIN_BURST_EVENTS = 3;
|
||||||
|
// Real dialogue holds on screen for about a second, so a run with a couple of much
|
||||||
|
// shorter events among them looks like frames. Used only alongside authoring evidence.
|
||||||
|
const ANIMATION_FRAME_MAX_SECONDS = 0.3;
|
||||||
|
// A karaoke run usually ends on a long "hold" frame, so not every event is short.
|
||||||
|
const MIN_TAGGED_BURST_FRAMES = 2;
|
||||||
|
// SRT and VTT carry no authoring metadata at all, so timing is the only signal available
|
||||||
|
// -- which makes it the easiest one to get wrong. ASS->SRT conversion leaves frames at
|
||||||
|
// ~0.04s, well under any real utterance, and a burst leaves many of them behind. Both
|
||||||
|
// bounds are deliberately far stricter than the ASS path: a run of ordinary short lines
|
||||||
|
// (`えっ` traded between characters) must not clear them.
|
||||||
|
const TIMING_ONLY_FRAME_MAX_SECONDS = 0.1;
|
||||||
|
const MIN_TIMING_ONLY_FRAMES = 5;
|
||||||
|
|
||||||
|
function cueKey(cue: SubtitleCue): string {
|
||||||
|
return `${cue.startTime}|${cue.endTime}|${cue.text}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Identical text over an identical span is redundant however it was authored -- most
|
||||||
|
* often a layered ASS event stacking a shadow copy under the visible one.
|
||||||
|
*/
|
||||||
|
function collapseExactDuplicates(cues: AnnotatedSubtitleCue[]): AnnotatedSubtitleCue[] {
|
||||||
|
const seen = new Set<string>();
|
||||||
|
return cues.filter((cue) => {
|
||||||
|
const key = cueKey(cue);
|
||||||
|
if (seen.has(key)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
seen.add(key);
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function countFramesShorterThan(run: AnnotatedSubtitleCue[], maxSeconds: number): number {
|
||||||
|
return run.filter((cue) => cue.endTime - cue.startTime < maxSeconds).length;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evidence that a run of ASS events is one animation rather than several authored lines.
|
||||||
|
* A static tag says nothing on its own -- three events sharing one `\clip(...)` are three
|
||||||
|
* signs -- so the tag has to be temporal by nature (`\t`, `\move`, karaoke timing, or
|
||||||
|
* anything wrapped in `\t(...)`), an animated `Effect` column, or a value that actually
|
||||||
|
* changes from event to event, which is how per-frame typesetting is authored.
|
||||||
|
*/
|
||||||
|
export function hasAssAnimationEvidence(run: AnnotatedSubtitleCue[]): boolean {
|
||||||
|
if (run.every((cue) => hasAssTemporalOverride(cue.overrides))) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (run.every((cue) => isAnimatedAssEffectKind(cue.effectKind))) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const [first] = run;
|
||||||
|
const everyEventTypeset = run.every((cue) => cue.overrides.length > 0);
|
||||||
|
const signatureChanges = run.some((cue) => cue.overrideSignature !== first!.overrideSignature);
|
||||||
|
return everyEventTypeset && signatureChanges;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isAnimationBurst(
|
||||||
|
run: AnnotatedSubtitleCue[],
|
||||||
|
format: SubtitleSourceFormat,
|
||||||
|
): boolean {
|
||||||
|
if (run.length < MIN_BURST_EVENTS) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (format === 'srt') {
|
||||||
|
return (
|
||||||
|
run.length >= MIN_TIMING_ONLY_FRAMES &&
|
||||||
|
countFramesShorterThan(run, TIMING_ONLY_FRAME_MAX_SECONDS) === run.length
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (countFramesShorterThan(run, ANIMATION_FRAME_MAX_SECONDS) < MIN_TAGGED_BURST_FRAMES) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// One animation belongs to one styled, one named source line. Two characters trading
|
||||||
|
// the same short word are two styles or two actors, and never merge.
|
||||||
|
const [first] = run;
|
||||||
|
if (run.some((cue) => cue.style !== first!.style || cue.name !== first!.name)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
return hasAssAnimationEvidence(run);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Karaoke and sign typesetting emits one Dialogue event per animation frame, all carrying
|
||||||
|
* the same visible text over a contiguous span. Collapse each such run into a single cue.
|
||||||
|
*
|
||||||
|
* Only runs that look like animation collapse. Two ordinary lines that happen to repeat
|
||||||
|
* -- several characters each saying `おはよう` in turn, a positioned sign redrawn with a
|
||||||
|
* different fade -- stay separate, because merging them would destroy real mineable lines.
|
||||||
|
*/
|
||||||
|
function collapseAnimationBursts(
|
||||||
|
cues: AnnotatedSubtitleCue[],
|
||||||
|
format: SubtitleSourceFormat,
|
||||||
|
): AnnotatedSubtitleCue[] {
|
||||||
|
const indicesByText = new Map<string, number[]>();
|
||||||
|
cues.forEach((cue, index) => {
|
||||||
|
const bucket = indicesByText.get(cue.text);
|
||||||
|
if (bucket) {
|
||||||
|
bucket.push(index);
|
||||||
|
} else {
|
||||||
|
indicesByText.set(cue.text, [index]);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const dropped = new Set<number>();
|
||||||
|
const extendedEnd = new Map<number, number>();
|
||||||
|
|
||||||
|
for (const indices of indicesByText.values()) {
|
||||||
|
if (indices.length < MIN_BURST_EVENTS) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
let runStart = 0;
|
||||||
|
while (runStart < indices.length) {
|
||||||
|
let runEnd = runStart;
|
||||||
|
let chainEnd = cues[indices[runStart]!]!.endTime;
|
||||||
|
|
||||||
|
while (runEnd + 1 < indices.length) {
|
||||||
|
const next = cues[indices[runEnd + 1]!]!;
|
||||||
|
if (next.startTime > chainEnd + DUPLICATE_CUE_GAP_TOLERANCE_SECONDS) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
chainEnd = Math.max(chainEnd, next.endTime);
|
||||||
|
runEnd += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const run = indices.slice(runStart, runEnd + 1).map((index) => cues[index]!);
|
||||||
|
if (isAnimationBurst(run, format)) {
|
||||||
|
for (let i = runStart + 1; i <= runEnd; i += 1) {
|
||||||
|
dropped.add(indices[i]!);
|
||||||
|
}
|
||||||
|
extendedEnd.set(indices[runStart]!, chainEnd);
|
||||||
|
}
|
||||||
|
|
||||||
|
runStart = runEnd + 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dropped.size === 0) {
|
||||||
|
return cues;
|
||||||
|
}
|
||||||
|
|
||||||
|
const merged: AnnotatedSubtitleCue[] = [];
|
||||||
|
cues.forEach((cue, index) => {
|
||||||
|
if (dropped.has(index)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const end = extendedEnd.get(index);
|
||||||
|
merged.push(end !== undefined && end > cue.endTime ? { ...cue, endTime: end } : cue);
|
||||||
|
});
|
||||||
|
return merged;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function mergeDuplicateCues(
|
||||||
|
cues: AnnotatedSubtitleCue[],
|
||||||
|
format: SubtitleSourceFormat,
|
||||||
|
): AnnotatedSubtitleCue[] {
|
||||||
|
return collapseAnimationBursts(collapseExactDuplicates(cues), format);
|
||||||
|
}
|
||||||
@@ -91,6 +91,17 @@ test('parseSrtCues skips malformed timing lines gracefully', () => {
|
|||||||
assert.equal(cues[0]!.text, '有効');
|
assert.equal(cues[0]!.text, '有効');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues strips complete brace blocks from SRT and VTT text', () => {
|
||||||
|
const content = ['1', '00:00:01,000 --> 00:00:02,000', '彼は{謎}と言った', ''].join('\n');
|
||||||
|
|
||||||
|
for (const filename of ['test.srt', 'test.vtt']) {
|
||||||
|
const cues = parseSubtitleCues(content, filename);
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1, filename);
|
||||||
|
assert.equal(cues[0]!.text, '彼はと言った', filename);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
test('parseAssCues parses basic ASS dialogue lines', () => {
|
test('parseAssCues parses basic ASS dialogue lines', () => {
|
||||||
const content = [
|
const content = [
|
||||||
'[Script Info]',
|
'[Script Info]',
|
||||||
@@ -137,7 +148,9 @@ test('parseAssCues handles text containing commas', () => {
|
|||||||
assert.equal(cues[0]!.text, 'はい、そうです、ね');
|
assert.equal(cues[0]!.text, 'はい、そうです、ね');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('parseAssCues handles \\N line breaks', () => {
|
test('parseAssCues decodes \\N line breaks into real newlines', () => {
|
||||||
|
// ASS is decoded once, here at ingestion, so cue text matches what mpv hands over for
|
||||||
|
// the same line played live.
|
||||||
const content = [
|
const content = [
|
||||||
'[Events]',
|
'[Events]',
|
||||||
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
@@ -146,7 +159,7 @@ test('parseAssCues handles \\N line breaks', () => {
|
|||||||
|
|
||||||
const cues = parseAssCues(content);
|
const cues = parseAssCues(content);
|
||||||
|
|
||||||
assert.equal(cues[0]!.text, '一行目\\N二行目');
|
assert.equal(cues[0]!.text, '一行目\n二行目');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('parseAssCues strips HTML-like markup while preserving ASS line breaks', () => {
|
test('parseAssCues strips HTML-like markup while preserving ASS line breaks', () => {
|
||||||
@@ -158,7 +171,46 @@ test('parseAssCues strips HTML-like markup while preserving ASS line breaks', ()
|
|||||||
|
|
||||||
const cues = parseAssCues(content);
|
const cues = parseAssCues(content);
|
||||||
|
|
||||||
assert.equal(cues[0]!.text, '一行目\\N二行目');
|
assert.equal(cues[0]!.text, '一行目\n二行目');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseAssCues drops vector drawing runs enabled by \\p', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 1,0:00:01.00,0:00:04.00,Default,,0,0,0,,{\\an5\\pos(730,1042)\\p1\\blur1}m 20 0 b 10 0 0 10 0 20 b 0 31 10 40 20 40 {\\p0}',
|
||||||
|
'Dialogue: 0,0:00:05.00,0:00:08.00,Default,,0,0,0,,これは字幕',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseAssCues(content);
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.text, 'これは字幕');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseAssCues keeps text that follows a \\p0 reset on the same line', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:04.00,Default,,0,0,0,,{\\p1}m 0 0 l 10 10{\\p0}本文{\\p1}m 5 5 l 6 6{\\p0}続き',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseAssCues(content);
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.text, '本文続き');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseAssCues leaves \\pos untouched when no drawing mode is active', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:04.00,Default,,0,0,0,,{\\pos(960,1068)\\bord3}位置指定',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseAssCues(content);
|
||||||
|
|
||||||
|
assert.equal(cues[0]!.text, '位置指定');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('parseAssCues returns empty for content without Events section', () => {
|
test('parseAssCues returns empty for content without Events section', () => {
|
||||||
@@ -258,6 +310,344 @@ test('parseSubtitleCues returns cues sorted by start time', () => {
|
|||||||
assert.equal(cues[1]!.text, '二番目');
|
assert.equal(cues[1]!.text, '二番目');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues collapses per-frame karaoke duplicates into one cue', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.05,OP_JP,,0,0,0,,{\\clip(m 1 1)}過ぎ去ってしまう瞬間を',
|
||||||
|
'Dialogue: 0,0:00:01.05,0:00:01.09,OP_JP,,0,0,0,,{\\clip(m 2 2)}過ぎ去ってしまう瞬間を',
|
||||||
|
'Dialogue: 0,0:00:01.09,0:00:03.55,OP_JP,,0,0,0,,{\\clip(m 3 3)}過ぎ去ってしまう瞬間を',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.startTime, 1.0);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.55);
|
||||||
|
assert.equal(cues[0]!.text, '過ぎ去ってしまう瞬間を');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps back-to-back plain dialogue repeats separate', () => {
|
||||||
|
// Several characters greeting in turn: distinct utterances that happen to abut.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:04:05.67,0:04:06.82,Dial_JP,,0,0,0,,おはよう',
|
||||||
|
'Dialogue: 0,0:04:06.82,0:04:07.56,Dial_JP,,0,0,0,,おはよう',
|
||||||
|
'Dialogue: 0,0:04:07.56,0:04:08.78,Dial_JP,,0,0,0,,おはよう',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
assert.equal(cues[0]!.endTime, 246.82);
|
||||||
|
assert.equal(cues[2]!.startTime, 247.56);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues collapses exact duplicate cues even without effect tags', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:04.00,Default,,0,0,0,,重なった行',
|
||||||
|
'Dialogue: 1,0:00:01.00,0:00:04.00,Default,,0,0,0,,重なった行',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues collapses tag-less animation frames in converted SRT', () => {
|
||||||
|
// ASS -> SRT conversion drops override tags, so only the ~0.04s frame timing remains.
|
||||||
|
const lines = ['1', '00:00:07,870 --> 00:00:07,910', 'Kaguya Wants to be Confessed to', ''];
|
||||||
|
for (let i = 1; i < 8; i++) {
|
||||||
|
const start = 7910 + (i - 1) * 40;
|
||||||
|
const end = start + 40;
|
||||||
|
const at = (ms: number) =>
|
||||||
|
`00:00:0${Math.floor(ms / 1000)},${String(ms % 1000).padStart(3, '0')}`;
|
||||||
|
lines.push(String(i + 1), `${at(start)} --> ${at(end)}`, 'Kaguya Wants to be Confessed to', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(lines.join('\n'), 'test.srt');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.startTime, 7.87);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps identical lines that recur far apart', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:02.00,Default,,0,0,0,,なんで',
|
||||||
|
'Dialogue: 0,0:05:00.00,0:05:01.00,Default,,0,0,0,,なんで',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 2);
|
||||||
|
assert.equal(cues[0]!.startTime, 1.0);
|
||||||
|
assert.equal(cues[1]!.startTime, 300.0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps two positioned signs that repeat the same text', () => {
|
||||||
|
// Both carry override tags, but `\pos` and `\fad` are static placement, not animation.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:01:00.00,0:01:03.00,Sign,,0,0,0,,{\\pos(960,120)\\fad(200,200)}第一話',
|
||||||
|
'Dialogue: 0,0:01:03.00,0:01:06.00,Sign,,0,0,0,,{\\pos(960,900)\\fad(200,200)}第一話',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 2);
|
||||||
|
assert.equal(cues[1]!.startTime, 63.0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps a run of ordinary positioned lines separate', () => {
|
||||||
|
// Three events is a sequence, but none of them runs at animation-frame speed.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:01:00.00,0:01:02.00,Sign,,0,0,0,,{\\pos(960,120)\\fad(100,100)}止まれ',
|
||||||
|
'Dialogue: 0,0:01:02.00,0:01:04.00,Sign,,0,0,0,,{\\pos(960,120)\\fad(100,100)}止まれ',
|
||||||
|
'Dialogue: 0,0:01:04.00,0:01:06.00,Sign,,0,0,0,,{\\pos(960,120)\\fad(100,100)}止まれ',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps a short repeated SRT pair without burst evidence', () => {
|
||||||
|
const content = [
|
||||||
|
'1',
|
||||||
|
'00:00:01,000 --> 00:00:01,200',
|
||||||
|
'えっ',
|
||||||
|
'',
|
||||||
|
'2',
|
||||||
|
'00:00:01,200 --> 00:00:01,400',
|
||||||
|
'えっ',
|
||||||
|
'',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.srt');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues collapses a burst marked only by the Effect column', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.05,OP_JP,,0,0,0,Karaoke,歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.05,0:00:01.09,OP_JP,,0,0,0,Karaoke,歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.09,0:00:03.55,OP_JP,,0,0,0,Karaoke,歌詞',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.55);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps a second karaoke burst that starts after a gap', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.05,OP_JP,,0,0,0,,{\\clip(m 1 1)}リフレイン',
|
||||||
|
'Dialogue: 0,0:00:01.05,0:00:01.09,OP_JP,,0,0,0,,{\\clip(m 2 2)}リフレイン',
|
||||||
|
'Dialogue: 0,0:00:01.09,0:00:03.00,OP_JP,,0,0,0,,{\\clip(m 3 3)}リフレイン',
|
||||||
|
'Dialogue: 0,0:00:20.00,0:00:20.05,OP_JP,,0,0,0,,{\\clip(m 1 1)}リフレイン',
|
||||||
|
'Dialogue: 0,0:00:20.05,0:00:20.09,OP_JP,,0,0,0,,{\\clip(m 2 2)}リフレイン',
|
||||||
|
'Dialogue: 0,0:00:20.09,0:00:22.00,OP_JP,,0,0,0,,{\\clip(m 3 3)}リフレイン',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 2);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.0);
|
||||||
|
assert.equal(cues[1]!.startTime, 20.0);
|
||||||
|
assert.equal(cues[1]!.endTime, 22.0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues does not merge a burst into unrelated dialogue between frames', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.05,OP_JP,,0,0,0,,{\\clip(m 1 1)}歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.02,0:00:03.00,Dial_JP,,0,0,0,,別のセリフ',
|
||||||
|
'Dialogue: 0,0:00:01.05,0:00:01.09,OP_JP,,0,0,0,,{\\clip(m 2 2)}歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.09,0:00:03.55,OP_JP,,0,0,0,,{\\clip(m 3 3)}歌詞',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 2);
|
||||||
|
assert.deepEqual(
|
||||||
|
cues.map((cue) => cue.text),
|
||||||
|
['歌詞', '別のセリフ'],
|
||||||
|
);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.55);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps rapid ASS lines from different actors separate', () => {
|
||||||
|
// Three 200ms `えっ` reactions traded between characters. Fast, adjacent and identical,
|
||||||
|
// but authored as three lines: different styles and different actors.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Dial_A,アリス,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Dial_B,ボブ,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:01.60,Dial_C,キャロル,0,0,0,,えっ',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues reads the speaker column when it is spelled Actor', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Actor, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Dial_JP,アリス,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Dial_JP,ボブ,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:01.60,Dial_JP,キャロル,0,0,0,,えっ',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues does not treat a custom Effect name as animation', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Sign,,0,0,0,scrolling-credit,制作',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Sign,,0,0,0,scrolling-credit,制作',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:01.60,Sign,,0,0,0,scrolling-credit,制作',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps rapid ASS lines that share a style but not an actor', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Dial_JP,アリス,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Dial_JP,ボブ,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:01.60,Dial_JP,キャロル,0,0,0,,えっ',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps untagged rapid ASS repeats separate', () => {
|
||||||
|
// No overrides at all: timing-only evidence is an SRT/VTT fallback and must not apply
|
||||||
|
// to ASS, where the absence of typesetting is itself evidence of plain dialogue.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.05,Dial_JP,,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.05,0:00:01.10,Dial_JP,,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.10,0:00:01.15,Dial_JP,,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.15,0:00:01.20,Dial_JP,,0,0,0,,えっ',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.25,Dial_JP,,0,0,0,,えっ',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 5);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps repeated signs sharing one static clip', () => {
|
||||||
|
// `\clip` is a static shape for the event. Three events with the identical clip were
|
||||||
|
// typeset the same way, so none of them is a frame of the others.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Sign,,0,0,0,,{\\clip(0,0,100,100)}注意',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Sign,,0,0,0,,{\\clip(0,0,100,100)}注意',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:01.60,Sign,,0,0,0,,{\\clip(0,0,100,100)}注意',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 3);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues collapses a sign animated through \\t', () => {
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Sign,,0,0,0,,{\\pos(10,10)\\t(0,200,\\frz30)}回る',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Sign,,0,0,0,,{\\pos(10,10)\\t(0,200,\\frz30)}回る',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:03.00,Sign,,0,0,0,,{\\pos(10,10)\\t(0,200,\\frz30)}回る',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.ass');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps a short repeated SRT run above the frame threshold', () => {
|
||||||
|
// Five contiguous 200ms cues: a sequence, but nowhere near animation-frame speed.
|
||||||
|
const lines: string[] = [];
|
||||||
|
for (let i = 0; i < 5; i++) {
|
||||||
|
const start = 1000 + i * 200;
|
||||||
|
const at = (ms: number) =>
|
||||||
|
`00:00:0${Math.floor(ms / 1000)},${String(ms % 1000).padStart(3, '0')}`;
|
||||||
|
lines.push(String(i + 1), `${at(start)} --> ${at(start + 200)}`, 'えっ', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(lines.join('\n'), 'test.srt');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 5);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues keeps a short SRT frame run below the minimum length', () => {
|
||||||
|
// Four 40ms frames: frame-speed, but too few to tell an animation from an artefact.
|
||||||
|
const lines: string[] = [];
|
||||||
|
for (let i = 0; i < 4; i++) {
|
||||||
|
const start = 7870 + i * 40;
|
||||||
|
const at = (ms: number) =>
|
||||||
|
`00:00:0${Math.floor(ms / 1000)},${String(ms % 1000).padStart(3, '0')}`;
|
||||||
|
lines.push(String(i + 1), `${at(start)} --> ${at(start + 40)}`, 'タイトル', '');
|
||||||
|
}
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(lines.join('\n'), 'test.srt');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 4);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('parseSubtitleCues applies ASS burst rules to ASS content behind an .srt filename', () => {
|
||||||
|
// The extension lies, so the SRT parser finds nothing and the content-sniffing fallback
|
||||||
|
// takes over -- which has to carry the `ass` source format with it, or the far stricter
|
||||||
|
// timing-only thresholds would let this karaoke burst through as three cues.
|
||||||
|
const content = [
|
||||||
|
'[Events]',
|
||||||
|
'Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text',
|
||||||
|
'Dialogue: 0,0:00:01.00,0:00:01.20,Karaoke,,0,0,0,,{\\k20}歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.20,0:00:01.40,Karaoke,,0,0,0,,{\\k20}歌詞',
|
||||||
|
'Dialogue: 0,0:00:01.40,0:00:03.00,Karaoke,,0,0,0,,{\\k20}歌詞',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const cues = parseSubtitleCues(content, 'test.srt');
|
||||||
|
|
||||||
|
assert.equal(cues.length, 1);
|
||||||
|
assert.equal(cues[0]!.startTime, 1.0);
|
||||||
|
assert.equal(cues[0]!.endTime, 3.0);
|
||||||
|
assert.equal(cues[0]!.text, '歌詞');
|
||||||
|
});
|
||||||
|
|
||||||
test('parseSubtitleCues detects subtitle formats from remote URLs', () => {
|
test('parseSubtitleCues detects subtitle formats from remote URLs', () => {
|
||||||
const assContent = [
|
const assContent = [
|
||||||
'[Events]',
|
'[Events]',
|
||||||
|
|||||||
@@ -1,9 +1,46 @@
|
|||||||
|
import {
|
||||||
|
assOverrideSignature,
|
||||||
|
assToPlainText,
|
||||||
|
collectAssOverrideCommands,
|
||||||
|
parseAssEffectField,
|
||||||
|
type AssEffectKind,
|
||||||
|
type AssOverrideCommand,
|
||||||
|
} from './ass-text';
|
||||||
|
import { mergeDuplicateCues } from './subtitle-cue-dedup';
|
||||||
|
|
||||||
export interface SubtitleCue {
|
export interface SubtitleCue {
|
||||||
startTime: number;
|
startTime: number;
|
||||||
endTime: number;
|
endTime: number;
|
||||||
text: string;
|
text: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Everything the parser knows about a source event, shared only with the dedup engine.
|
||||||
|
* Deduplication needs the authoring context -- which style the line belongs to, which
|
||||||
|
* override commands it carries, whether the `Effect` column was set -- to tell a karaoke
|
||||||
|
* burst apart from two characters saying the same word in turn. None of it is meaningful
|
||||||
|
* outside the parser, so the public API stays `{startTime, endTime, text}`.
|
||||||
|
*/
|
||||||
|
export interface AnnotatedSubtitleCue extends SubtitleCue {
|
||||||
|
/** Text exactly as authored, override blocks and all. */
|
||||||
|
rawText: string;
|
||||||
|
style: string;
|
||||||
|
layer: number;
|
||||||
|
/** ASS `Name`/`Actor` column. */
|
||||||
|
name: string;
|
||||||
|
/** ASS `Effect` column, verbatim. */
|
||||||
|
effect: string;
|
||||||
|
effectKind: AssEffectKind;
|
||||||
|
/** Override commands found in `{...}` blocks, with their arguments. */
|
||||||
|
overrides: readonly AssOverrideCommand[];
|
||||||
|
/** Canonical form of `overrides`, for spotting values that change across a run. */
|
||||||
|
overrideSignature: string;
|
||||||
|
/** Position in the source file, so sorting by time stays deterministic across layers. */
|
||||||
|
order: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type SubtitleSourceFormat = 'ass' | 'srt';
|
||||||
|
|
||||||
const HTML_SUBTITLE_TAG_PATTERN = /<\/?[A-Za-z][^>\n]*>/g;
|
const HTML_SUBTITLE_TAG_PATTERN = /<\/?[A-Za-z][^>\n]*>/g;
|
||||||
|
|
||||||
const SRT_TIMING_PATTERN =
|
const SRT_TIMING_PATTERN =
|
||||||
@@ -23,12 +60,21 @@ function parseTimestamp(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The single ASS decode for the file path: cues leave the parser as plain text with real
|
||||||
|
* line breaks, matching what mpv hands over for the same line played live. No layer
|
||||||
|
* downstream decodes ASS again.
|
||||||
|
*/
|
||||||
function sanitizeSubtitleCueText(text: string): string {
|
function sanitizeSubtitleCueText(text: string): string {
|
||||||
return text.replace(ASS_OVERRIDE_TAG_PATTERN, '').replace(HTML_SUBTITLE_TAG_PATTERN, '').trim();
|
return assToPlainText(text, '\n').replace(HTML_SUBTITLE_TAG_PATTERN, '').trim();
|
||||||
}
|
}
|
||||||
|
|
||||||
export function parseSrtCues(content: string): SubtitleCue[] {
|
function toPublicCues(cues: AnnotatedSubtitleCue[]): SubtitleCue[] {
|
||||||
const cues: SubtitleCue[] = [];
|
return cues.map(({ startTime, endTime, text }) => ({ startTime, endTime, text }));
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseAnnotatedSrtCues(content: string): AnnotatedSubtitleCue[] {
|
||||||
|
const cues: AnnotatedSubtitleCue[] = [];
|
||||||
const lines = content.split(/\r?\n/);
|
const lines = content.split(/\r?\n/);
|
||||||
let i = 0;
|
let i = 0;
|
||||||
|
|
||||||
@@ -60,20 +106,39 @@ export function parseSrtCues(content: string): SubtitleCue[] {
|
|||||||
i += 1;
|
i += 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
const text = sanitizeSubtitleCueText(textLines.join('\n'));
|
const rawText = textLines.join('\n');
|
||||||
|
const text = sanitizeSubtitleCueText(rawText);
|
||||||
if (text) {
|
if (text) {
|
||||||
cues.push({ startTime, endTime, text });
|
cues.push({
|
||||||
|
startTime,
|
||||||
|
endTime,
|
||||||
|
text,
|
||||||
|
rawText,
|
||||||
|
style: '',
|
||||||
|
layer: 0,
|
||||||
|
name: '',
|
||||||
|
effect: '',
|
||||||
|
effectKind: 'none',
|
||||||
|
// SRT and VTT carry no authoring metadata, and the dedup engine never reads
|
||||||
|
// overrides for those formats -- collecting them would be parsing for nobody.
|
||||||
|
overrides: [],
|
||||||
|
overrideSignature: '',
|
||||||
|
order: cues.length,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return cues;
|
return cues;
|
||||||
}
|
}
|
||||||
|
|
||||||
const ASS_OVERRIDE_TAG_PATTERN = /\{[^}]*\}/g;
|
export function parseSrtCues(content: string): SubtitleCue[] {
|
||||||
|
return toPublicCues(parseAnnotatedSrtCues(content));
|
||||||
|
}
|
||||||
|
|
||||||
const ASS_TIMING_PATTERN = /^(\d+):(\d{2}):(\d{2})\.(\d{1,2})$/;
|
const ASS_TIMING_PATTERN = /^(\d+):(\d{2}):(\d{2})\.(\d{1,2})$/;
|
||||||
const ASS_FORMAT_PREFIX = 'Format:';
|
const ASS_FORMAT_PREFIX = 'Format:';
|
||||||
const ASS_DIALOGUE_PREFIX = 'Dialogue:';
|
const ASS_DIALOGUE_PREFIX = 'Dialogue:';
|
||||||
|
const ASS_NAME_FIELD_ALIASES = ['name', 'actor'];
|
||||||
|
|
||||||
function parseAssTimestamp(raw: string): number | null {
|
function parseAssTimestamp(raw: string): number | null {
|
||||||
const match = ASS_TIMING_PATTERN.exec(raw.trim());
|
const match = ASS_TIMING_PATTERN.exec(raw.trim());
|
||||||
@@ -87,13 +152,43 @@ function parseAssTimestamp(raw: string): number | null {
|
|||||||
return hours * 3600 + minutes * 60 + seconds + centiseconds / 100;
|
return hours * 3600 + minutes * 60 + seconds + centiseconds / 100;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function parseAssCues(content: string): SubtitleCue[] {
|
function readField(fields: string[], index: number): string {
|
||||||
const cues: SubtitleCue[] = [];
|
return index >= 0 && index < fields.length ? fields[index]!.trim() : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function findFieldIndex(formatFields: string[], aliases: string[]): number {
|
||||||
|
for (const alias of aliases) {
|
||||||
|
const index = formatFields.indexOf(alias);
|
||||||
|
if (index >= 0) {
|
||||||
|
return index;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseAnnotatedAssCues(content: string): AnnotatedSubtitleCue[] {
|
||||||
|
const cues: AnnotatedSubtitleCue[] = [];
|
||||||
const lines = content.split(/\r?\n/);
|
const lines = content.split(/\r?\n/);
|
||||||
let inEventsSection = false;
|
let inEventsSection = false;
|
||||||
let startFieldIndex = -1;
|
const fieldIndex = {
|
||||||
let endFieldIndex = -1;
|
start: -1,
|
||||||
let textFieldIndex = -1;
|
end: -1,
|
||||||
|
text: -1,
|
||||||
|
style: -1,
|
||||||
|
layer: -1,
|
||||||
|
name: -1,
|
||||||
|
effect: -1,
|
||||||
|
};
|
||||||
|
|
||||||
|
const resetFieldIndex = () => {
|
||||||
|
fieldIndex.start = -1;
|
||||||
|
fieldIndex.end = -1;
|
||||||
|
fieldIndex.text = -1;
|
||||||
|
fieldIndex.style = -1;
|
||||||
|
fieldIndex.layer = -1;
|
||||||
|
fieldIndex.name = -1;
|
||||||
|
fieldIndex.effect = -1;
|
||||||
|
};
|
||||||
|
|
||||||
for (const line of lines) {
|
for (const line of lines) {
|
||||||
const trimmed = line.trim();
|
const trimmed = line.trim();
|
||||||
@@ -101,9 +196,7 @@ export function parseAssCues(content: string): SubtitleCue[] {
|
|||||||
if (trimmed.startsWith('[') && trimmed.endsWith(']')) {
|
if (trimmed.startsWith('[') && trimmed.endsWith(']')) {
|
||||||
inEventsSection = trimmed.toLowerCase() === '[events]';
|
inEventsSection = trimmed.toLowerCase() === '[events]';
|
||||||
if (!inEventsSection) {
|
if (!inEventsSection) {
|
||||||
startFieldIndex = -1;
|
resetFieldIndex();
|
||||||
endFieldIndex = -1;
|
|
||||||
textFieldIndex = -1;
|
|
||||||
}
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
@@ -117,9 +210,15 @@ export function parseAssCues(content: string): SubtitleCue[] {
|
|||||||
.slice(ASS_FORMAT_PREFIX.length)
|
.slice(ASS_FORMAT_PREFIX.length)
|
||||||
.split(',')
|
.split(',')
|
||||||
.map((field) => field.trim().toLowerCase());
|
.map((field) => field.trim().toLowerCase());
|
||||||
startFieldIndex = formatFields.indexOf('start');
|
fieldIndex.start = formatFields.indexOf('start');
|
||||||
endFieldIndex = formatFields.indexOf('end');
|
fieldIndex.end = formatFields.indexOf('end');
|
||||||
textFieldIndex = formatFields.indexOf('text');
|
fieldIndex.text = formatFields.indexOf('text');
|
||||||
|
fieldIndex.style = formatFields.indexOf('style');
|
||||||
|
fieldIndex.layer = formatFields.indexOf('layer');
|
||||||
|
// Aegisub writes the speaker column as `Actor`; the v4+ spec calls it `Name`.
|
||||||
|
// Missing it costs the burst check its speaker guard, so both spellings count.
|
||||||
|
fieldIndex.name = findFieldIndex(formatFields, ASS_NAME_FIELD_ALIASES);
|
||||||
|
fieldIndex.effect = formatFields.indexOf('effect');
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -127,34 +226,57 @@ export function parseAssCues(content: string): SubtitleCue[] {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (startFieldIndex < 0 || endFieldIndex < 0 || textFieldIndex < 0) {
|
if (fieldIndex.start < 0 || fieldIndex.end < 0 || fieldIndex.text < 0) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const fields = trimmed.slice(ASS_DIALOGUE_PREFIX.length).split(',');
|
const fields = trimmed.slice(ASS_DIALOGUE_PREFIX.length).split(',');
|
||||||
if (
|
if (
|
||||||
startFieldIndex >= fields.length ||
|
fieldIndex.start >= fields.length ||
|
||||||
endFieldIndex >= fields.length ||
|
fieldIndex.end >= fields.length ||
|
||||||
textFieldIndex >= fields.length
|
fieldIndex.text >= fields.length
|
||||||
) {
|
) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const startTime = parseAssTimestamp(fields[startFieldIndex]!);
|
const startTime = parseAssTimestamp(fields[fieldIndex.start]!);
|
||||||
const endTime = parseAssTimestamp(fields[endFieldIndex]!);
|
const endTime = parseAssTimestamp(fields[fieldIndex.end]!);
|
||||||
if (startTime === null || endTime === null) {
|
if (startTime === null || endTime === null) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
const text = sanitizeSubtitleCueText(fields.slice(textFieldIndex).join(','));
|
const rawText = fields.slice(fieldIndex.text).join(',');
|
||||||
if (text) {
|
const text = sanitizeSubtitleCueText(rawText);
|
||||||
cues.push({ startTime, endTime, text });
|
if (!text) {
|
||||||
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const effect = readField(fields, fieldIndex.effect);
|
||||||
|
const layer = Number(readField(fields, fieldIndex.layer));
|
||||||
|
const overrides = collectAssOverrideCommands(rawText);
|
||||||
|
cues.push({
|
||||||
|
startTime,
|
||||||
|
endTime,
|
||||||
|
text,
|
||||||
|
rawText,
|
||||||
|
style: readField(fields, fieldIndex.style),
|
||||||
|
layer: Number.isFinite(layer) ? layer : 0,
|
||||||
|
name: readField(fields, fieldIndex.name),
|
||||||
|
effect,
|
||||||
|
effectKind: parseAssEffectField(effect),
|
||||||
|
overrides,
|
||||||
|
overrideSignature: assOverrideSignature(overrides),
|
||||||
|
order: cues.length,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
return cues;
|
return cues;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function parseAssCues(content: string): SubtitleCue[] {
|
||||||
|
return toPublicCues(parseAnnotatedAssCues(content));
|
||||||
|
}
|
||||||
|
|
||||||
function detectSubtitleFormat(source: string): 'srt' | 'vtt' | 'ass' | 'ssa' | null {
|
function detectSubtitleFormat(source: string): 'srt' | 'vtt' | 'ass' | 'ssa' | null {
|
||||||
const [normalizedSource = source] =
|
const [normalizedSource = source] =
|
||||||
(() => {
|
(() => {
|
||||||
@@ -173,27 +295,31 @@ function detectSubtitleFormat(source: string): 'srt' | 'vtt' | 'ass' | 'ssa' | n
|
|||||||
|
|
||||||
export function parseSubtitleCues(content: string, filename: string): SubtitleCue[] {
|
export function parseSubtitleCues(content: string, filename: string): SubtitleCue[] {
|
||||||
const format = detectSubtitleFormat(filename);
|
const format = detectSubtitleFormat(filename);
|
||||||
let cues: SubtitleCue[];
|
let cues: AnnotatedSubtitleCue[];
|
||||||
|
let sourceFormat: SubtitleSourceFormat = 'srt';
|
||||||
|
|
||||||
switch (format) {
|
switch (format) {
|
||||||
case 'srt':
|
case 'srt':
|
||||||
case 'vtt':
|
case 'vtt':
|
||||||
cues = parseSrtCues(content);
|
cues = parseAnnotatedSrtCues(content);
|
||||||
break;
|
break;
|
||||||
case 'ass':
|
case 'ass':
|
||||||
case 'ssa':
|
case 'ssa':
|
||||||
cues = parseAssCues(content);
|
cues = parseAnnotatedAssCues(content);
|
||||||
|
sourceFormat = 'ass';
|
||||||
break;
|
break;
|
||||||
default:
|
default:
|
||||||
cues = [];
|
cues = [];
|
||||||
}
|
}
|
||||||
|
|
||||||
if (cues.length === 0) {
|
if (cues.length === 0) {
|
||||||
const assCues = parseAssCues(content);
|
const assCues = parseAnnotatedAssCues(content);
|
||||||
const srtCues = parseSrtCues(content);
|
const srtCues = parseAnnotatedSrtCues(content);
|
||||||
cues = assCues.length >= srtCues.length ? assCues : srtCues;
|
const preferAss = assCues.length >= srtCues.length;
|
||||||
|
cues = preferAss ? assCues : srtCues;
|
||||||
|
sourceFormat = preferAss && assCues.length > 0 ? 'ass' : 'srt';
|
||||||
}
|
}
|
||||||
|
|
||||||
cues.sort((a, b) => a.startTime - b.startTime);
|
cues.sort((a, b) => a.startTime - b.startTime || a.endTime - b.endTime || a.order - b.order);
|
||||||
return cues;
|
return toPublicCues(mergeDuplicateCues(cues, sourceFormat));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -115,6 +115,21 @@ test('subtitle processing does not emit plain payload for cached lines', async (
|
|||||||
assert.deepEqual(emitted, [{ text: '字幕', tokens: [] }]);
|
assert.deepEqual(emitted, [{ text: '字幕', tokens: [] }]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('text that normalizes to nothing is never cached', () => {
|
||||||
|
const controller = createSubtitleProcessingController({
|
||||||
|
tokenizeSubtitle: async (text) => ({ text, tokens: [] }),
|
||||||
|
emitSubtitle: () => {},
|
||||||
|
});
|
||||||
|
|
||||||
|
// Two different inputs both reduce to an empty key; sharing one entry would serve the
|
||||||
|
// first one's tokens for the second.
|
||||||
|
controller.preCacheTokenization(' ', { text: ' ', tokens: [] });
|
||||||
|
|
||||||
|
assert.equal(controller.hasCachedSubtitle(' '), false);
|
||||||
|
assert.equal(controller.hasCachedSubtitle('\\n'), false);
|
||||||
|
assert.equal(controller.consumeCachedSubtitle('\\n'), null);
|
||||||
|
});
|
||||||
|
|
||||||
test('subtitle processing shows plain line while tokenization is still pending', async () => {
|
test('subtitle processing shows plain line while tokenization is still pending', async () => {
|
||||||
const emitted: SubtitleData[] = [];
|
const emitted: SubtitleData[] = [];
|
||||||
let resolveTokenization: ((value: SubtitleData | null) => void) | undefined;
|
let resolveTokenization: ((value: SubtitleData | null) => void) | undefined;
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import type { SubtitleData } from '../../types';
|
import type { SubtitleData } from '../../types';
|
||||||
|
import { normalizePlainSubtitleText } from './ass-text';
|
||||||
|
|
||||||
export interface SubtitleProcessingControllerDeps {
|
export interface SubtitleProcessingControllerDeps {
|
||||||
tokenizeSubtitle: (text: string) => Promise<SubtitleData | null>;
|
tokenizeSubtitle: (text: string) => Promise<SubtitleData | null>;
|
||||||
@@ -25,8 +26,15 @@ export interface SubtitleProcessingController {
|
|||||||
hasCachedSubtitle: (text: string) => boolean;
|
hasCachedSubtitle: (text: string) => boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Prefetched cues and live mpv text are both already decoded from ASS, so the key only
|
||||||
|
* has to settle whitespace for one authored line to resolve to one entry.
|
||||||
|
*
|
||||||
|
* An empty key is not a line: it is whatever normalization reduced to nothing. Callers
|
||||||
|
* must skip the cache for it rather than let every such input share one entry.
|
||||||
|
*/
|
||||||
export function normalizeSubtitleCacheKey(text: string): string {
|
export function normalizeSubtitleCacheKey(text: string): string {
|
||||||
return text.replace(/\r\n/g, '\n').replace(/\\N/g, '\n').replace(/\\n/g, '\n').trim();
|
return normalizePlainSubtitleText(text);
|
||||||
}
|
}
|
||||||
|
|
||||||
export function createSubtitleProcessingController(
|
export function createSubtitleProcessingController(
|
||||||
@@ -50,6 +58,9 @@ export function createSubtitleProcessingController(
|
|||||||
|
|
||||||
const getCachedTokenization = (text: string): SubtitleData | null => {
|
const getCachedTokenization = (text: string): SubtitleData | null => {
|
||||||
const cacheKey = normalizeSubtitleCacheKey(text);
|
const cacheKey = normalizeSubtitleCacheKey(text);
|
||||||
|
if (!cacheKey) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
const cached = tokenizationCache.get(cacheKey);
|
const cached = tokenizationCache.get(cacheKey);
|
||||||
if (!cached) {
|
if (!cached) {
|
||||||
return null;
|
return null;
|
||||||
@@ -61,7 +72,11 @@ export function createSubtitleProcessingController(
|
|||||||
};
|
};
|
||||||
|
|
||||||
const setCachedTokenization = (text: string, payload: SubtitleData): void => {
|
const setCachedTokenization = (text: string, payload: SubtitleData): void => {
|
||||||
tokenizationCache.set(normalizeSubtitleCacheKey(text), payload);
|
const cacheKey = normalizeSubtitleCacheKey(text);
|
||||||
|
if (!cacheKey) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
tokenizationCache.set(cacheKey, payload);
|
||||||
while (tokenizationCache.size > SUBTITLE_TOKENIZATION_CACHE_LIMIT) {
|
while (tokenizationCache.size > SUBTITLE_TOKENIZATION_CACHE_LIMIT) {
|
||||||
const firstKey = tokenizationCache.keys().next().value;
|
const firstKey = tokenizationCache.keys().next().value;
|
||||||
if (firstKey !== undefined) {
|
if (firstKey !== undefined) {
|
||||||
@@ -219,7 +234,8 @@ export function createSubtitleProcessingController(
|
|||||||
return cached;
|
return cached;
|
||||||
},
|
},
|
||||||
hasCachedSubtitle: (text: string) => {
|
hasCachedSubtitle: (text: string) => {
|
||||||
return tokenizationCache.has(normalizeSubtitleCacheKey(text));
|
const cacheKey = normalizeSubtitleCacheKey(text);
|
||||||
|
return cacheKey.length > 0 && tokenizationCache.has(cacheKey);
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1651,9 +1651,11 @@ test('tokenizeSubtitle clears JLPT level from standalone Yomitan particle token'
|
|||||||
assert.equal(result.tokens?.[0]?.jlptLevel, undefined);
|
assert.equal(result.tokens?.[0]?.jlptLevel, undefined);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('tokenizeSubtitle returns null tokens for empty normalized text', async () => {
|
test('tokenizeSubtitle returns the normalized text when it comes out empty', async () => {
|
||||||
|
// Handing back the original would push whatever normalization dropped into app state
|
||||||
|
// as if it were subtitle text.
|
||||||
const result = await tokenizeSubtitle(' \\n ', makeDeps());
|
const result = await tokenizeSubtitle(' \\n ', makeDeps());
|
||||||
assert.deepEqual(result, { text: ' \\n ', tokens: null });
|
assert.deepEqual(result, { text: '', tokens: null });
|
||||||
});
|
});
|
||||||
|
|
||||||
test('tokenizeSubtitle normalizes newlines before Yomitan parse request', async () => {
|
test('tokenizeSubtitle normalizes newlines before Yomitan parse request', async () => {
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ import {
|
|||||||
} from './tokenizer/yomitan-parser-runtime';
|
} from './tokenizer/yomitan-parser-runtime';
|
||||||
import type { YomitanTermFrequency } from './tokenizer/yomitan-parser-runtime';
|
import type { YomitanTermFrequency } from './tokenizer/yomitan-parser-runtime';
|
||||||
import { isKanaChar } from './tokenizer/token-classification';
|
import { isKanaChar } from './tokenizer/token-classification';
|
||||||
|
import { normalizePlainSubtitleText } from './ass-text';
|
||||||
|
|
||||||
const logger = createLogger('main:tokenizer');
|
const logger = createLogger('main:tokenizer');
|
||||||
|
|
||||||
@@ -858,14 +859,14 @@ export async function tokenizeSubtitle(
|
|||||||
text: string,
|
text: string,
|
||||||
deps: TokenizerServiceDeps,
|
deps: TokenizerServiceDeps,
|
||||||
): Promise<SubtitleData> {
|
): Promise<SubtitleData> {
|
||||||
const displayText = text
|
const displayText = normalizePlainSubtitleText(text);
|
||||||
.replace(/\r\n/g, '\n')
|
|
||||||
.replace(/\\N/g, '\n')
|
|
||||||
.replace(/\\n/g, '\n')
|
|
||||||
.trim();
|
|
||||||
|
|
||||||
|
// ASS decoding already happened upstream (cue parser for files, mpv for live text), so
|
||||||
|
// all this drops is whitespace -- but a whitespace-only line still normalizes to empty.
|
||||||
|
// Return the normalized form anyway: handing back the original would put a blank line
|
||||||
|
// into application state as if it were subtitle text.
|
||||||
if (!displayText) {
|
if (!displayText) {
|
||||||
return { text, tokens: null };
|
return { text: displayText, tokens: null };
|
||||||
}
|
}
|
||||||
|
|
||||||
const tokenizeText = displayText
|
const tokenizeText = displayText
|
||||||
|
|||||||
@@ -1004,6 +1004,24 @@ test('normalizeSubtitle collapses explicit line breaks when collapseLineBreaks i
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('normalizeSubtitle leaves already-decoded text alone', () => {
|
||||||
|
// Primary subtitle text is decoded from ASS once, upstream: by mpv for live lines and
|
||||||
|
// by the cue parser for prefetched ones. A brace that survives that is literal text.
|
||||||
|
assert.equal(normalizeSubtitle('本文{\\pos(1,2)'), '本文{\\pos(1,2)');
|
||||||
|
assert.equal(normalizeSubtitle(' 余白 ', false), ' 余白 ');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('prepareSecondarySubtitleLines drops ASS vector drawing runs', () => {
|
||||||
|
assert.deepEqual(
|
||||||
|
prepareSecondarySubtitleLines(
|
||||||
|
'{\\an5\\pos(730,1042)\\p1\\blur1}m 20 0 b 10 0 0 10 0 20 b 0 31 10 40 20 40 {\\p0}',
|
||||||
|
),
|
||||||
|
[],
|
||||||
|
);
|
||||||
|
assert.deepEqual(prepareSecondarySubtitleLines('{\\p1}m 0 0 l 10 10{\\p0}本文'), ['本文']);
|
||||||
|
assert.deepEqual(prepareSecondarySubtitleLines('{\\pos(960,1068)\\bord3}位置指定'), ['位置指定']);
|
||||||
|
});
|
||||||
|
|
||||||
test('shouldRenderTokenizedSubtitle enables token rendering when tokens exist', () => {
|
test('shouldRenderTokenizedSubtitle enables token rendering when tokens exist', () => {
|
||||||
assert.equal(shouldRenderTokenizedSubtitle(5), true);
|
assert.equal(shouldRenderTokenizedSubtitle(5), true);
|
||||||
assert.equal(shouldRenderTokenizedSubtitle(0), false);
|
assert.equal(shouldRenderTokenizedSubtitle(0), false);
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import type {
|
|||||||
SubtitleData,
|
SubtitleData,
|
||||||
SubtitleRendererStyleConfig,
|
SubtitleRendererStyleConfig,
|
||||||
} from '../types';
|
} from '../types';
|
||||||
|
import { assToPlainText, normalizePlainSubtitleText } from '../core/services/ass-text.js';
|
||||||
import type { RendererContext } from './context';
|
import type { RendererContext } from './context';
|
||||||
import { PRIMARY_SUB_VISIBLE_ON_YOMITAN_POPUP_CLASS } from './yomitan-popup.js';
|
import { PRIMARY_SUB_VISIBLE_ON_YOMITAN_POPUP_CLASS } from './yomitan-popup.js';
|
||||||
|
|
||||||
@@ -42,17 +43,10 @@ function isWhitespaceOnly(value: string): boolean {
|
|||||||
return value.trim().length === 0;
|
return value.trim().length === 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Text reaching the overlay has already been decoded from ASS -- by mpv for live lines,
|
||||||
|
// by the cue parser for prefetched ones -- so this only settles line breaks.
|
||||||
export function normalizeSubtitle(text: string, trim = true, collapseLineBreaks = false): string {
|
export function normalizeSubtitle(text: string, trim = true, collapseLineBreaks = false): string {
|
||||||
if (!text) return '';
|
return normalizePlainSubtitleText(text, { trim, collapseLineBreaks });
|
||||||
|
|
||||||
let normalized = text.replace(/\\N/g, '\n').replace(/\\n/g, '\n');
|
|
||||||
normalized = normalized.replace(/\{[^}]*\}/g, '');
|
|
||||||
if (collapseLineBreaks) {
|
|
||||||
normalized = normalized.replace(/\n/g, ' ');
|
|
||||||
normalized = normalized.replace(/\s+/g, ' ');
|
|
||||||
}
|
|
||||||
|
|
||||||
return trim ? normalized.trim() : normalized;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const HEX_COLOR_PATTERN = /^#(?:[0-9a-fA-F]{3}|[0-9a-fA-F]{4}|[0-9a-fA-F]{6}|[0-9a-fA-F]{8})$/;
|
const HEX_COLOR_PATTERN = /^#(?:[0-9a-fA-F]{3}|[0-9a-fA-F]{4}|[0-9a-fA-F]{6}|[0-9a-fA-F]{8})$/;
|
||||||
@@ -672,7 +666,10 @@ function isKaraokeLikeLineSet(lines: string[]): boolean {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function prepareSecondarySubtitleLines(text: string): string[] {
|
export function prepareSecondarySubtitleLines(text: string): string[] {
|
||||||
const normalized = normalizeSubtitle(text, true, false);
|
// The one display-side ASS decode: secondary text also reaches the overlay from
|
||||||
|
// websocket clients that forward their source line untouched, so unlike the primary
|
||||||
|
// path it cannot assume mpv already decoded it.
|
||||||
|
const normalized = assToPlainText(text).trim();
|
||||||
|
|
||||||
if (!normalized) return [];
|
if (!normalized) return [];
|
||||||
|
|
||||||
|
|||||||
@@ -16,6 +16,8 @@
|
|||||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
import { normalizePlainSubtitleText } from './core/services/ass-text';
|
||||||
|
|
||||||
interface TimingEntry {
|
interface TimingEntry {
|
||||||
startTime: number;
|
startTime: number;
|
||||||
endTime: number;
|
endTime: number;
|
||||||
@@ -191,23 +193,14 @@ export class SubtitleTimingTracker {
|
|||||||
return costs[shorter.length] || 0;
|
return costs[shorter.length] || 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Both sides take text mpv has already decoded from ASS; only whitespace differs
|
||||||
|
// between the lookup key (single line) and the display form (line breaks kept).
|
||||||
private normalizeText(text: string): string {
|
private normalizeText(text: string): string {
|
||||||
return text
|
return normalizePlainSubtitleText(text, { collapseLineBreaks: true });
|
||||||
.replace(/\\N/g, ' ')
|
|
||||||
.replace(/\\n/g, ' ')
|
|
||||||
.replace(/\n/g, ' ')
|
|
||||||
.replace(/{[^}]*}/g, '')
|
|
||||||
.replace(/\s+/g, ' ')
|
|
||||||
.trim();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private prepareDisplayText(text: string): string {
|
private prepareDisplayText(text: string): string {
|
||||||
// Convert ASS/SSA newlines to real newlines, strip tags
|
return normalizePlainSubtitleText(text);
|
||||||
return text
|
|
||||||
.replace(/\\N/g, '\n')
|
|
||||||
.replace(/\\n/g, '\n')
|
|
||||||
.replace(/{[^}]*}/g, '')
|
|
||||||
.trim();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private startCleanup(): void {
|
private startCleanup(): void {
|
||||||
|
|||||||
Reference in New Issue
Block a user