sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-09-17 17:16:23 -07:00
Code Issues Packages Projects Releases Wiki Activity

fix(subtitles): retain uncertain audio during dialogue generation

Browse Source 
- Preserve audio rejected by VAD so dialogue under music is retained.
- Keep short passages intact and stitch overlapping cues despite punctuation differences.
This commit is contained in:
sudacode
2026-09-14 19:16:34 -07:00
parent d6cbacf8be
commit 02792232c2
15 changed files with 242 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs-site/public/config.example.jsonc
+1 -1
View File
@@ -19,7 +19,7 @@
"threads": 4, // Positive integer CPU thread count for whisper.cpp Japanese transcription.
"ffmpegPath": "", // Optional FFmpeg path override for audio extraction. Leave empty to find ffmpeg on PATH.
"ffprobePath": "", // Optional FFprobe path override for audio tracks and timing. Leave empty to find ffprobe on PATH.
"vadModelPath": "", // Path to a whisper.cpp Silero VAD model. Enables dialogue-focused generation from separate speech passages. Leave empty to transcribe the full audio, including songs.
"vadModelPath": "", // Path to a whisper.cpp Silero VAD model. Enables dialogue-focused generation while retaining uncertain audible sections, which may include songs. Leave empty to transcribe the full audio.
"vadPath": "" // Optional speech detector executable override. With vadModelPath configured, leave empty to find whisper-vad-speech-segments or vad-speech-segments on PATH.
}, // Generate timed Japanese subtitles from local audio using whisper.cpp.
docs-site/subtitle-generation.md
+3 -3
View File
@@ -25,11 +25,11 @@ You also need whisper.cpp's [speech segment detector](https://github.com/ggml-or
The checkbox choice lasts for the current SubMiner session, including closing and reopening the modal. To make dialogue mode your default, set `vadModelPath` in Settings to a [Silero GGML VAD model](https://huggingface.co/ggml-org/whisper-vad/tree/main). The modal downloads `ggml-silero-v6.2.0.bin` into the same `models/whisper/` directory as managed Whisper models. An existing configured VAD path takes precedence and checks the box initially. Unchecking it temporarily disables dialogue mode without changing that path. Downloading the model alone does not enable dialogue mode.
With speech detection configured, SubMiner transcribes short speech passages separately and restores each passage's position on the original audio timeline. Detection retains brief utterances and includes extra audio around speech to reduce clipped syllables. If the detector returns a long passage, SubMiner looks for quiet pauses near chunk boundaries. Adjacent chunks overlap slightly to provide context when speech continues through a cut. Matching subtitle cues in that overlap are combined; repeated dialogue at separate times remains separate.
With speech detection configured, SubMiner keeps detected speech and other audible sections for Whisper to evaluate. A low speech score alone does not discard audio, which helps retain dialogue mixed with music. Only confidently silent gaps outside detected speech are omitted, with extra audio retained around each passage to reduce clipped syllables.
SubMiner resets transcription context between passages and limits subtitle cues to the audio supplied for each chunk. A line cannot stretch across an omitted music break. Progress reports completed batches of dialogue passages. These adjustments do not replace Whisper's timestamp estimates or guarantee that every spoken line is recognized.
Passages that fit within Whisper's 30-second audio window stay intact. Longer passages split near quiet pauses, with a small overlap to provide context. Matching overlapping cues are combined even when punctuation differs; repeated dialogue at separate times remains separate. SubMiner resets transcription context between passages, limits subtitle cues to the supplied audio, and restores each passage's position on the original timeline. Progress reports completed batches of passages.
This mode prioritizes spoken dialogue over songs and background sounds. It can miss quiet speech or speech mixed with loud music, and recognition errors are still possible. Uncheck **Focus on spoken dialogue** to return to full-audio transcription for the session, or clear `vadModelPath` to change the default. A selected detector or model that fails stops generation with an error. Existing subtitles are preserved.
This mode favors retaining dialogue over excluding music, so songs and background sounds may also produce subtitles. It can take longer than transcribing only VAD-approved speech. Whisper can still miss or misrecognize dialogue, and its timestamps remain estimates. Uncheck **Focus on spoken dialogue** to return to full-audio transcription for the session, or clear `vadModelPath` to change the default. A selected detector or model that fails stops generation with an error. Existing subtitles are preserved.
## Choosing a model