feat(anki): add proxy transport and tokenizer annotation controls

This commit is contained in:
2026-02-27 21:25:26 -08:00
parent 34a0feae71
commit f8e961d105
26 changed files with 1453 additions and 60 deletions

View File

@@ -22,7 +22,7 @@ make docs-preview # Preview built site at http://localhost:4173
- [Configuration](/configuration) — Full config file reference and option details
- [Keyboard Shortcuts](/shortcuts) — All global, overlay, mining, and plugin chord shortcuts in one place
- [Anki Integration](/anki-integration) — AnkiConnect setup, field mapping, media generation, field grouping
- [Anki Integration](/anki-integration) — AnkiConnect setup, proxy/polling transport, field mapping, media generation, field grouping
- [Jellyfin Integration](/jellyfin-integration) — Optional Jellyfin auth, cast discovery, remote control, and playback launch
- [Immersion Tracking](/immersion-tracking) — SQLite schema, retention/rollup policies, query templates, and extension points
- [Performance & Tuning](/troubleshooting#performance-and-resource-impact) — Resource usage and practical low-impact profile

View File

@@ -10,9 +10,14 @@ SubMiner uses the [AnkiConnect](https://ankiweb.net/shared/info/2055492159) add-
AnkiConnect listens on `http://127.0.0.1:8765` by default. If you changed the port in AnkiConnect's settings, update `ankiConnect.url` in your SubMiner config.
## How Polling Works
## Auto-Enrichment Transport
SubMiner polls AnkiConnect at a regular interval (default: 3 seconds, configurable via `ankiConnect.pollingRate`) to detect new cards. When it finds a card that was added since the last poll:
SubMiner supports two auto-enrichment transport modes:
1. `polling` (default): polls AnkiConnect at `ankiConnect.pollingRate` (default: 3s).
2. `proxy` (optional): runs a local AnkiConnect-compatible proxy and enriches cards immediately after successful `addNote` / `addNotes` responses.
In both modes, the enrichment workflow is the same:
1. Checks if a duplicate expression already exists (for field grouping).
2. Updates the sentence field with the current subtitle.
@@ -20,7 +25,32 @@ SubMiner polls AnkiConnect at a regular interval (default: 3 seconds, configurab
4. Fills the translation field from the secondary subtitle or AI.
5. Writes metadata to the miscInfo field.
Polling uses the query `"deck:<your-deck>" added:1` to find recently added cards. If no deck is configured, it searches all decks.
Polling mode uses the query `"deck:<your-deck>" added:1` to find recently added cards. If no deck is configured, it searches all decks.
### Proxy Mode Setup (Yomitan / Texthooker)
```jsonc
"ankiConnect": {
"url": "http://127.0.0.1:8765", // real AnkiConnect
"proxy": {
"enabled": true,
"host": "127.0.0.1",
"port": 8766,
"upstreamUrl": "http://127.0.0.1:8765"
}
}
```
Then point Yomitan/clients to `http://127.0.0.1:8766` instead of `8765`.
When SubMiner loads the bundled Yomitan extension, it also attempts to update the **default Yomitan profile** (`profiles[0].options.anki.server`) to the active SubMiner endpoint:
- proxy URL when `ankiConnect.proxy.enabled` is `true`
- direct `ankiConnect.url` when proxy mode is disabled
To avoid clobbering custom setups, this auto-update only changes the default profile when its current server is blank or the stock Yomitan default (`http://127.0.0.1:8765`).
For browser-based Yomitan or other external clients (for example texthooker in a normal browser profile), set their Anki server to the same proxy URL separately.
## Field Mapping
@@ -214,6 +244,12 @@ When you mine the same word multiple times, SubMiner can merge the cards instead
"enabled": true,
"url": "http://127.0.0.1:8765",
"pollingRate": 3000,
"proxy": {
"enabled": false,
"host": "127.0.0.1",
"port": 8766,
"upstreamUrl": "http://127.0.0.1:8765"
},
"fields": {
"audio": "ExpressionAudio",
"image": "Picture",

View File

@@ -47,6 +47,8 @@ Malformed config syntax (invalid JSON/JSONC) is startup-blocking: SubMiner shows
For valid JSON/JSONC with invalid option values, SubMiner uses warn-and-fallback behavior: it logs the bad key/value and continues with the default for that option.
On macOS, these validation warnings also open a native dialog with full details (desktop notification banners can truncate long messages).
### Hot-Reload Behavior
SubMiner watches the active config file (`config.jsonc` or `config.json`) while running and applies supported updates automatically.
@@ -87,6 +89,7 @@ The configuration file includes several main sections:
- [**Subtitle Style**](#subtitle-style) - Appearance customization
- [**Texthooker**](#texthooker) - Control browser opening behavior
- [**WebSocket Server**](#websocket-server) - Built-in subtitle broadcasting server
- [**Startup Warmups**](#startup-warmups) - Control what preloads on startup vs first-use defer
- [**Immersion Tracking**](#immersion-tracking) - Track subtitle sessions and mining activity in SQLite
- [**YouTube Subtitle Generation**](#youtube-subtitle-generation) - Launcher defaults for yt-dlp + local whisper fallback
@@ -826,6 +829,32 @@ See `config.example.jsonc` for detailed configuration options.
| `enabled` | `true`, `false`, `"auto"` | `"auto"` (default) disables if mpv_websocket is detected |
| `port` | number | WebSocket server port (default: 6677) |
### Startup Warmups
Control which startup warmups run in the background versus deferring to first real usage:
```json
{
"startupWarmups": {
"lowPowerMode": false,
"mecab": true,
"yomitanExtension": true,
"subtitleDictionaries": true,
"jellyfinRemoteSession": true
}
}
```
| Option | Values | Description |
| ------------------------ | --------------- | ------------------------------------------------------------------------------------------------ |
| `lowPowerMode` | `true`, `false` | Defer all warmups except Yomitan extension |
| `mecab` | `true`, `false` | Warm up MeCab tokenizer at startup |
| `yomitanExtension` | `true`, `false` | Warm up Yomitan extension at startup |
| `subtitleDictionaries` | `true`, `false` | Warm up JLPT + frequency dictionaries at startup |
| `jellyfinRemoteSession` | `true`, `false` | Warm up Jellyfin remote session at startup (still requires Jellyfin remote auto-connect settings) |
Defaults warm everything (`true` for all toggles, `lowPowerMode: false`). Setting a warmup toggle to `false` defers that work until first usage.
### Immersion Tracking
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions:

View File

@@ -20,7 +20,7 @@ SubMiner prioritizes subtitle responsiveness over heavy initialization:
1. The first subtitle render is **plain text first** (no tokenization wait).
2. Tokenized enrichment (word spans, known-word flags, JLPT/frequency metadata) is applied right after parsing completes.
3. Under rapid subtitle churn, SubMiner uses a **latest-only tokenization queue** so stale lines are dropped instead of building lag.
4. MeCab, Yomitan extension load, and dictionary prewarm run as background warmups after overlay initialization.
4. MeCab, Yomitan extension load, and dictionary prewarm run as background warmups after overlay initialization (configurable via `startupWarmups`, including low-power mode).
This keeps early playback snappy and avoids mpv-side sluggishness while startup work completes.
@@ -72,11 +72,13 @@ There are three ways to create cards, depending on your workflow.
### 1. Auto-Update from Yomitan
This is the most common flow. Yomitan creates a card in Anki, and SubMiner detects it via polling and enriches it automatically.
This is the most common flow. Yomitan creates a card in Anki, and SubMiner enriches it automatically.
1. Click a word → Yomitan popup appears.
2. Click the Anki icon in Yomitan to add the word.
3. SubMiner detects the new card (polls AnkiConnect every 3 seconds by default).
3. SubMiner receives or detects the new card:
- **Proxy mode** (`ankiConnect.proxy.enabled: true`): immediate enrich after successful `addNote` / `addNotes`.
- **Polling mode** (default): detects via AnkiConnect polling (`ankiConnect.pollingRate`, default 3 seconds).
4. SubMiner updates the card with:
- **Sentence**: The current subtitle line.
- **Audio**: Extracted from the video using the subtitle's start/end timing (plus configurable padding).
@@ -95,7 +97,7 @@ If you prefer a hands-on approach (animecards-style), you can copy the current s
- For multiple lines: press `Ctrl/Cmd+Shift+C`, then a digit `1``9` to select how many recent subtitle lines to combine. The combined text is copied to the clipboard.
3. Press `Ctrl/Cmd+V` to update the last-added card with the clipboard contents plus audio, image, and translation — the same fields auto-update would fill.
This is useful when auto-update polling is disabled or when you want explicit control over which subtitle line gets attached to the card.
This is useful when auto-update is disabled or when you want explicit control over which subtitle line gets attached to the card.
| Shortcut | Action | Config key |
| --------------------------- | ----------------------------------------- | ------------------------------------- |

View File

@@ -12,13 +12,6 @@
// ==========================================
"auto_start_overlay": false, // When overlay connects to mpv, automatically show overlay and hide mpv subtitles. Values: true | false
// ==========================================
// Visible Overlay Subtitle Binding
// Control whether visible overlay toggles also toggle MPV subtitle visibility.
// When enabled, visible overlay hides MPV subtitles; when disabled, MPV subtitles are left unchanged.
// ==========================================
"bind_visible_overlay_to_mpv_sub_visibility": true, // Link visible overlay toggles to MPV primary subtitle visibility. Values: true | false
// ==========================================
// Texthooker Server
// Control whether browser opens automatically for texthooker.
@@ -179,6 +172,12 @@
"enabled": false, // Enable AnkiConnect integration. Values: true | false
"url": "http://127.0.0.1:8765", // Url setting.
"pollingRate": 3000, // Polling interval in milliseconds.
"proxy": {
"enabled": false, // Enable local AnkiConnect-compatible proxy for push-based auto-enrichment. Values: true | false
"host": "127.0.0.1", // Bind host for local AnkiConnect proxy.
"port": 8766, // Bind port for local AnkiConnect proxy.
"upstreamUrl": "http://127.0.0.1:8765" // Upstream AnkiConnect URL proxied by local AnkiConnect proxy.
}, // Proxy setting.
"tags": [
"SubMiner"
], // Tags to add to cards mined or updated by SubMiner. Provide an empty array to disable automatic tagging.

View File

@@ -30,7 +30,7 @@ SubMiner retries the connection automatically with increasing delays (200 ms, 50
- first subtitle parse/tokenization bursts
- media generation (`ffmpeg` audio/image and AVIF paths)
- media sync and subtitle tooling (`alass`, `ffsubsync`, `whisper` fallback path)
- `ankiConnect` enrichment and frequent polling
- `ankiConnect` enrichment (plus polling overhead when proxy mode is disabled)
### If playback feels sluggish
@@ -104,11 +104,17 @@ Logged when a malformed JSON line arrives from the mpv socket. Usually harmless
**"AnkiConnect: unable to connect"**
SubMiner polls AnkiConnect at `http://127.0.0.1:8765` (configurable via `ankiConnect.url`). This error means Anki is not running or the AnkiConnect add-on is not installed.
SubMiner connects to the active Anki endpoint:
- `ankiConnect.url` (direct mode, default `http://127.0.0.1:8765`)
- `http://<ankiConnect.proxy.host>:<ankiConnect.proxy.port>` (proxy mode)
This error means the active endpoint is unavailable, or (in proxy mode) the proxy cannot reach `ankiConnect.proxy.upstreamUrl`.
- Install the [AnkiConnect](https://ankiweb.net/shared/info/2055492159) add-on in Anki.
- Make sure Anki is running before you start mining.
- If you changed the AnkiConnect port, update `ankiConnect.url` in your config.
- If you changed the AnkiConnect port, update `ankiConnect.url` (or `ankiConnect.proxy.upstreamUrl` if using proxy mode).
- If using external Yomitan/browser clients, confirm they point to your SubMiner proxy URL.
SubMiner retries with exponential backoff (up to 5 s) and suppresses repeated error logs after 5 consecutive failures. When Anki comes back, you will see "AnkiConnect connection restored".
@@ -122,7 +128,7 @@ See [Anki Integration](/anki-integration) for the full field mapping reference.
Shown when SubMiner tries to update a card that no longer exists, or when AnkiConnect rejects the update. Common causes:
- The card was deleted in Anki between polling and update.
- The card was deleted in Anki between creation and enrichment update.
- The note type changed and a mapped field no longer exists.
## Overlay