mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-03-01 06:22:44 -08:00
feat(anki): add proxy transport and tokenizer annotation controls
This commit is contained in:
@@ -22,7 +22,7 @@ make docs-preview # Preview built site at http://localhost:4173
|
||||
|
||||
- [Configuration](/configuration) — Full config file reference and option details
|
||||
- [Keyboard Shortcuts](/shortcuts) — All global, overlay, mining, and plugin chord shortcuts in one place
|
||||
- [Anki Integration](/anki-integration) — AnkiConnect setup, field mapping, media generation, field grouping
|
||||
- [Anki Integration](/anki-integration) — AnkiConnect setup, proxy/polling transport, field mapping, media generation, field grouping
|
||||
- [Jellyfin Integration](/jellyfin-integration) — Optional Jellyfin auth, cast discovery, remote control, and playback launch
|
||||
- [Immersion Tracking](/immersion-tracking) — SQLite schema, retention/rollup policies, query templates, and extension points
|
||||
- [Performance & Tuning](/troubleshooting#performance-and-resource-impact) — Resource usage and practical low-impact profile
|
||||
|
||||
@@ -10,9 +10,14 @@ SubMiner uses the [AnkiConnect](https://ankiweb.net/shared/info/2055492159) add-
|
||||
|
||||
AnkiConnect listens on `http://127.0.0.1:8765` by default. If you changed the port in AnkiConnect's settings, update `ankiConnect.url` in your SubMiner config.
|
||||
|
||||
## How Polling Works
|
||||
## Auto-Enrichment Transport
|
||||
|
||||
SubMiner polls AnkiConnect at a regular interval (default: 3 seconds, configurable via `ankiConnect.pollingRate`) to detect new cards. When it finds a card that was added since the last poll:
|
||||
SubMiner supports two auto-enrichment transport modes:
|
||||
|
||||
1. `polling` (default): polls AnkiConnect at `ankiConnect.pollingRate` (default: 3s).
|
||||
2. `proxy` (optional): runs a local AnkiConnect-compatible proxy and enriches cards immediately after successful `addNote` / `addNotes` responses.
|
||||
|
||||
In both modes, the enrichment workflow is the same:
|
||||
|
||||
1. Checks if a duplicate expression already exists (for field grouping).
|
||||
2. Updates the sentence field with the current subtitle.
|
||||
@@ -20,7 +25,32 @@ SubMiner polls AnkiConnect at a regular interval (default: 3 seconds, configurab
|
||||
4. Fills the translation field from the secondary subtitle or AI.
|
||||
5. Writes metadata to the miscInfo field.
|
||||
|
||||
Polling uses the query `"deck:<your-deck>" added:1` to find recently added cards. If no deck is configured, it searches all decks.
|
||||
Polling mode uses the query `"deck:<your-deck>" added:1` to find recently added cards. If no deck is configured, it searches all decks.
|
||||
|
||||
### Proxy Mode Setup (Yomitan / Texthooker)
|
||||
|
||||
```jsonc
|
||||
"ankiConnect": {
|
||||
"url": "http://127.0.0.1:8765", // real AnkiConnect
|
||||
"proxy": {
|
||||
"enabled": true,
|
||||
"host": "127.0.0.1",
|
||||
"port": 8766,
|
||||
"upstreamUrl": "http://127.0.0.1:8765"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then point Yomitan/clients to `http://127.0.0.1:8766` instead of `8765`.
|
||||
|
||||
When SubMiner loads the bundled Yomitan extension, it also attempts to update the **default Yomitan profile** (`profiles[0].options.anki.server`) to the active SubMiner endpoint:
|
||||
|
||||
- proxy URL when `ankiConnect.proxy.enabled` is `true`
|
||||
- direct `ankiConnect.url` when proxy mode is disabled
|
||||
|
||||
To avoid clobbering custom setups, this auto-update only changes the default profile when its current server is blank or the stock Yomitan default (`http://127.0.0.1:8765`).
|
||||
|
||||
For browser-based Yomitan or other external clients (for example texthooker in a normal browser profile), set their Anki server to the same proxy URL separately.
|
||||
|
||||
## Field Mapping
|
||||
|
||||
@@ -214,6 +244,12 @@ When you mine the same word multiple times, SubMiner can merge the cards instead
|
||||
"enabled": true,
|
||||
"url": "http://127.0.0.1:8765",
|
||||
"pollingRate": 3000,
|
||||
"proxy": {
|
||||
"enabled": false,
|
||||
"host": "127.0.0.1",
|
||||
"port": 8766,
|
||||
"upstreamUrl": "http://127.0.0.1:8765"
|
||||
},
|
||||
"fields": {
|
||||
"audio": "ExpressionAudio",
|
||||
"image": "Picture",
|
||||
|
||||
@@ -47,6 +47,8 @@ Malformed config syntax (invalid JSON/JSONC) is startup-blocking: SubMiner shows
|
||||
|
||||
For valid JSON/JSONC with invalid option values, SubMiner uses warn-and-fallback behavior: it logs the bad key/value and continues with the default for that option.
|
||||
|
||||
On macOS, these validation warnings also open a native dialog with full details (desktop notification banners can truncate long messages).
|
||||
|
||||
### Hot-Reload Behavior
|
||||
|
||||
SubMiner watches the active config file (`config.jsonc` or `config.json`) while running and applies supported updates automatically.
|
||||
@@ -87,6 +89,7 @@ The configuration file includes several main sections:
|
||||
- [**Subtitle Style**](#subtitle-style) - Appearance customization
|
||||
- [**Texthooker**](#texthooker) - Control browser opening behavior
|
||||
- [**WebSocket Server**](#websocket-server) - Built-in subtitle broadcasting server
|
||||
- [**Startup Warmups**](#startup-warmups) - Control what preloads on startup vs first-use defer
|
||||
- [**Immersion Tracking**](#immersion-tracking) - Track subtitle sessions and mining activity in SQLite
|
||||
- [**YouTube Subtitle Generation**](#youtube-subtitle-generation) - Launcher defaults for yt-dlp + local whisper fallback
|
||||
|
||||
@@ -826,6 +829,32 @@ See `config.example.jsonc` for detailed configuration options.
|
||||
| `enabled` | `true`, `false`, `"auto"` | `"auto"` (default) disables if mpv_websocket is detected |
|
||||
| `port` | number | WebSocket server port (default: 6677) |
|
||||
|
||||
### Startup Warmups
|
||||
|
||||
Control which startup warmups run in the background versus deferring to first real usage:
|
||||
|
||||
```json
|
||||
{
|
||||
"startupWarmups": {
|
||||
"lowPowerMode": false,
|
||||
"mecab": true,
|
||||
"yomitanExtension": true,
|
||||
"subtitleDictionaries": true,
|
||||
"jellyfinRemoteSession": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ------------------------ | --------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `lowPowerMode` | `true`, `false` | Defer all warmups except Yomitan extension |
|
||||
| `mecab` | `true`, `false` | Warm up MeCab tokenizer at startup |
|
||||
| `yomitanExtension` | `true`, `false` | Warm up Yomitan extension at startup |
|
||||
| `subtitleDictionaries` | `true`, `false` | Warm up JLPT + frequency dictionaries at startup |
|
||||
| `jellyfinRemoteSession` | `true`, `false` | Warm up Jellyfin remote session at startup (still requires Jellyfin remote auto-connect settings) |
|
||||
|
||||
Defaults warm everything (`true` for all toggles, `lowPowerMode: false`). Setting a warmup toggle to `false` defers that work until first usage.
|
||||
|
||||
### Immersion Tracking
|
||||
|
||||
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions:
|
||||
|
||||
@@ -20,7 +20,7 @@ SubMiner prioritizes subtitle responsiveness over heavy initialization:
|
||||
1. The first subtitle render is **plain text first** (no tokenization wait).
|
||||
2. Tokenized enrichment (word spans, known-word flags, JLPT/frequency metadata) is applied right after parsing completes.
|
||||
3. Under rapid subtitle churn, SubMiner uses a **latest-only tokenization queue** so stale lines are dropped instead of building lag.
|
||||
4. MeCab, Yomitan extension load, and dictionary prewarm run as background warmups after overlay initialization.
|
||||
4. MeCab, Yomitan extension load, and dictionary prewarm run as background warmups after overlay initialization (configurable via `startupWarmups`, including low-power mode).
|
||||
|
||||
This keeps early playback snappy and avoids mpv-side sluggishness while startup work completes.
|
||||
|
||||
@@ -72,11 +72,13 @@ There are three ways to create cards, depending on your workflow.
|
||||
|
||||
### 1. Auto-Update from Yomitan
|
||||
|
||||
This is the most common flow. Yomitan creates a card in Anki, and SubMiner detects it via polling and enriches it automatically.
|
||||
This is the most common flow. Yomitan creates a card in Anki, and SubMiner enriches it automatically.
|
||||
|
||||
1. Click a word → Yomitan popup appears.
|
||||
2. Click the Anki icon in Yomitan to add the word.
|
||||
3. SubMiner detects the new card (polls AnkiConnect every 3 seconds by default).
|
||||
3. SubMiner receives or detects the new card:
|
||||
- **Proxy mode** (`ankiConnect.proxy.enabled: true`): immediate enrich after successful `addNote` / `addNotes`.
|
||||
- **Polling mode** (default): detects via AnkiConnect polling (`ankiConnect.pollingRate`, default 3 seconds).
|
||||
4. SubMiner updates the card with:
|
||||
- **Sentence**: The current subtitle line.
|
||||
- **Audio**: Extracted from the video using the subtitle's start/end timing (plus configurable padding).
|
||||
@@ -95,7 +97,7 @@ If you prefer a hands-on approach (animecards-style), you can copy the current s
|
||||
- For multiple lines: press `Ctrl/Cmd+Shift+C`, then a digit `1`–`9` to select how many recent subtitle lines to combine. The combined text is copied to the clipboard.
|
||||
3. Press `Ctrl/Cmd+V` to update the last-added card with the clipboard contents plus audio, image, and translation — the same fields auto-update would fill.
|
||||
|
||||
This is useful when auto-update polling is disabled or when you want explicit control over which subtitle line gets attached to the card.
|
||||
This is useful when auto-update is disabled or when you want explicit control over which subtitle line gets attached to the card.
|
||||
|
||||
| Shortcut | Action | Config key |
|
||||
| --------------------------- | ----------------------------------------- | ------------------------------------- |
|
||||
|
||||
@@ -12,13 +12,6 @@
|
||||
// ==========================================
|
||||
"auto_start_overlay": false, // When overlay connects to mpv, automatically show overlay and hide mpv subtitles. Values: true | false
|
||||
|
||||
// ==========================================
|
||||
// Visible Overlay Subtitle Binding
|
||||
// Control whether visible overlay toggles also toggle MPV subtitle visibility.
|
||||
// When enabled, visible overlay hides MPV subtitles; when disabled, MPV subtitles are left unchanged.
|
||||
// ==========================================
|
||||
"bind_visible_overlay_to_mpv_sub_visibility": true, // Link visible overlay toggles to MPV primary subtitle visibility. Values: true | false
|
||||
|
||||
// ==========================================
|
||||
// Texthooker Server
|
||||
// Control whether browser opens automatically for texthooker.
|
||||
@@ -179,6 +172,12 @@
|
||||
"enabled": false, // Enable AnkiConnect integration. Values: true | false
|
||||
"url": "http://127.0.0.1:8765", // Url setting.
|
||||
"pollingRate": 3000, // Polling interval in milliseconds.
|
||||
"proxy": {
|
||||
"enabled": false, // Enable local AnkiConnect-compatible proxy for push-based auto-enrichment. Values: true | false
|
||||
"host": "127.0.0.1", // Bind host for local AnkiConnect proxy.
|
||||
"port": 8766, // Bind port for local AnkiConnect proxy.
|
||||
"upstreamUrl": "http://127.0.0.1:8765" // Upstream AnkiConnect URL proxied by local AnkiConnect proxy.
|
||||
}, // Proxy setting.
|
||||
"tags": [
|
||||
"SubMiner"
|
||||
], // Tags to add to cards mined or updated by SubMiner. Provide an empty array to disable automatic tagging.
|
||||
|
||||
@@ -30,7 +30,7 @@ SubMiner retries the connection automatically with increasing delays (200 ms, 50
|
||||
- first subtitle parse/tokenization bursts
|
||||
- media generation (`ffmpeg` audio/image and AVIF paths)
|
||||
- media sync and subtitle tooling (`alass`, `ffsubsync`, `whisper` fallback path)
|
||||
- `ankiConnect` enrichment and frequent polling
|
||||
- `ankiConnect` enrichment (plus polling overhead when proxy mode is disabled)
|
||||
|
||||
### If playback feels sluggish
|
||||
|
||||
@@ -104,11 +104,17 @@ Logged when a malformed JSON line arrives from the mpv socket. Usually harmless
|
||||
|
||||
**"AnkiConnect: unable to connect"**
|
||||
|
||||
SubMiner polls AnkiConnect at `http://127.0.0.1:8765` (configurable via `ankiConnect.url`). This error means Anki is not running or the AnkiConnect add-on is not installed.
|
||||
SubMiner connects to the active Anki endpoint:
|
||||
|
||||
- `ankiConnect.url` (direct mode, default `http://127.0.0.1:8765`)
|
||||
- `http://<ankiConnect.proxy.host>:<ankiConnect.proxy.port>` (proxy mode)
|
||||
|
||||
This error means the active endpoint is unavailable, or (in proxy mode) the proxy cannot reach `ankiConnect.proxy.upstreamUrl`.
|
||||
|
||||
- Install the [AnkiConnect](https://ankiweb.net/shared/info/2055492159) add-on in Anki.
|
||||
- Make sure Anki is running before you start mining.
|
||||
- If you changed the AnkiConnect port, update `ankiConnect.url` in your config.
|
||||
- If you changed the AnkiConnect port, update `ankiConnect.url` (or `ankiConnect.proxy.upstreamUrl` if using proxy mode).
|
||||
- If using external Yomitan/browser clients, confirm they point to your SubMiner proxy URL.
|
||||
|
||||
SubMiner retries with exponential backoff (up to 5 s) and suppresses repeated error logs after 5 consecutive failures. When Anki comes back, you will see "AnkiConnect connection restored".
|
||||
|
||||
@@ -122,7 +128,7 @@ See [Anki Integration](/anki-integration) for the full field mapping reference.
|
||||
|
||||
Shown when SubMiner tries to update a card that no longer exists, or when AnkiConnect rejects the update. Common causes:
|
||||
|
||||
- The card was deleted in Anki between polling and update.
|
||||
- The card was deleted in Anki between creation and enrichment update.
|
||||
- The note type changed and a mapped field no longer exists.
|
||||
|
||||
## Overlay
|
||||
|
||||
Reference in New Issue
Block a user