mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-03-20 12:11:28 -07:00
docs: update documentation site for stats dashboard and immersion tracking
This commit is contained in:
@@ -2,6 +2,8 @@
|
||||
|
||||
In-repo VitePress documentation source for SubMiner.
|
||||
|
||||
Internal architecture/workflow source of truth lives in `docs/README.md` at the repo root. Keep `docs-site/` user-facing.
|
||||
|
||||
## Local development
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Architecture
|
||||
|
||||
This page is a contributor-facing architecture summary. Canonical internal architecture guidance lives in `docs/architecture/README.md` at the repo root.
|
||||
|
||||
SubMiner is split into three cooperating runtimes:
|
||||
|
||||
- Electron desktop app (`src/`) for overlay/UI/runtime orchestration.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
SubMiner can build a Yomitan-compatible character dictionary from AniList metadata so that character names in subtitles are recognized, highlighted, and enrichable with context — portraits, roles, voice actors, and biographical detail — without leaving the overlay.
|
||||
|
||||
The dictionary is generated per-media, merged across your recently-watched titles, and auto-imported into Yomitan. When a character name appears in a subtitle line, it gets highlighted and becomes clickable for a full profile lookup.
|
||||
The dictionary is generated per-media, merged across your recently-watched titles, and auto-imported into Yomitan. When a character name appears in a subtitle line, it gets highlighted and becomes available for hover-driven Yomitan profile lookup.
|
||||
|
||||
## How It Works
|
||||
|
||||
|
||||
@@ -117,6 +117,7 @@ The configuration file includes several main sections:
|
||||
- [**Jellyfin**](#jellyfin) - Optional Jellyfin auth, library listing, and playback launch
|
||||
- [**Discord Rich Presence**](#discord-rich-presence) - Optional Discord activity card updates
|
||||
- [**Immersion Tracking**](#immersion-tracking) - Track subtitle sessions and mining activity in SQLite
|
||||
- [**Stats Dashboard**](#stats-dashboard) - Local dashboard and overlay for immersion progress
|
||||
- [**YouTube Subtitle Generation**](#youtube-subtitle-generation) - Launcher defaults for yt-dlp + local whisper fallback
|
||||
|
||||
## Core Settings
|
||||
@@ -1144,7 +1145,7 @@ Troubleshooting:
|
||||
|
||||
### Immersion Tracking
|
||||
|
||||
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions:
|
||||
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions. This data also powers the stats dashboard:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -1190,7 +1191,34 @@ When `dbPath` is blank or omitted, SubMiner writes telemetry and session summari
|
||||
|
||||
Set `dbPath` only if you want to relocate the database (for backup, syncing, or inspection workflows). The database is created when tracking starts for the first time.
|
||||
|
||||
See [Immersion Tracking Storage](/immersion-tracking) for schema details, query templates, retention/rollup behavior, backend portability notes, and the dedicated SQLite verification command.
|
||||
See [Immersion Tracking Storage](/immersion-tracking) for schema details, query templates, dashboard access, retention/rollup behavior, backend portability notes, and the dedicated SQLite verification command.
|
||||
|
||||
### Stats Dashboard
|
||||
|
||||
Configure the local stats UI served from SubMiner and the in-app stats overlay toggle:
|
||||
|
||||
```json
|
||||
{
|
||||
"stats": {
|
||||
"toggleKey": "Backquote",
|
||||
"serverPort": 5175,
|
||||
"autoStartServer": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Values | Description |
|
||||
| ----------------- | ----------------- | --------------------------------------------------------------------------- |
|
||||
| `toggleKey` | Electron key code | Overlay-local key code used to toggle the stats overlay. Default `Backquote`. |
|
||||
| `serverPort` | integer | Localhost port for the browser stats UI. Default `5175`. |
|
||||
| `autoStartServer` | `true`, `false` | Start the local stats HTTP server automatically once immersion tracking is active. Default `true`. |
|
||||
|
||||
Usage notes:
|
||||
|
||||
- The browser UI is served at `http://127.0.0.1:<serverPort>`.
|
||||
- The overlay toggle is local to the focused visible overlay window; it is not registered as a global OS shortcut.
|
||||
- The dashboard reads from the same immersion-tracking database, so keep `immersionTracking.enabled` on if you want data to appear.
|
||||
- The UI includes Overview, Trends, Sessions, and Vocabulary tabs.
|
||||
|
||||
### YouTube Subtitle Generation
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Building & Testing
|
||||
|
||||
For internal architecture/workflow guidance, use `docs/README.md` at the repo root. This page stays focused on contributor-facing build and test commands.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [Bun](https://bun.sh)
|
||||
@@ -13,6 +15,7 @@ cd SubMiner
|
||||
git submodule update --init --recursive
|
||||
|
||||
bun install
|
||||
(cd stats && bun install --frozen-lockfile)
|
||||
(cd vendor/texthooker-ui && bun install --frozen-lockfile)
|
||||
```
|
||||
|
||||
@@ -200,7 +203,7 @@ Run `make help` for a full list of targets. Key ones:
|
||||
| `make build-launcher` | Generate Bun launcher wrapper at `dist/launcher/subminer` |
|
||||
| `make install` | Install platform artifacts (wrapper, theme, AppImage/app bundle) |
|
||||
| `make install-plugin` | Install mpv Lua plugin and config |
|
||||
| `make deps` | Install JS dependencies (root + texthooker-ui) |
|
||||
| `make deps` | Install JS dependencies (root + stats + texthooker-ui) |
|
||||
| `make pretty` | Run scoped Prettier formatting for maintained source/config files |
|
||||
| `make generate-config` | Generate default config from centralized registry |
|
||||
| `make build-linux` | Convenience wrapper for Linux packaging |
|
||||
@@ -214,7 +217,7 @@ Run `make help` for a full list of targets. Key ones:
|
||||
- To add/change generated config template blocks/comments, update `src/config/definitions/template-sections.ts`.
|
||||
- Keep `src/config/definitions.ts` as the composed public API (`DEFAULT_CONFIG`, registries, template export) that wires domain modules together.
|
||||
- Overlay window/visibility state is owned by `src/core/services/overlay-manager.ts`.
|
||||
- Runtime architecture/module-boundary conventions are documented in [Architecture](/architecture); keep contributor changes aligned with that canonical guide.
|
||||
- Runtime architecture/module-boundary conventions are summarized in [Architecture](/architecture), with canonical internal guidance in `docs/architecture/README.md` at the repo root.
|
||||
- Linux packaged desktop launches pass `--background` using electron-builder `build.linux.executableArgs` in `package.json`.
|
||||
- Prefer direct inline deps objects in `src/main/` modules for simple pass-through wiring.
|
||||
- Add a helper/adapter service only when it performs meaningful adaptation, validation, or reuse (not identity mapping).
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Immersion Tracking
|
||||
|
||||
SubMiner can log your watching and mining activity to a local SQLite database. This is optional and disabled by default.
|
||||
SubMiner can log your watching and mining activity to a local SQLite database, then surface it in the built-in stats dashboard. Tracking is enabled by default and can be turned off if you do not want local analytics.
|
||||
|
||||
When enabled, SubMiner records per-session statistics (watch time, subtitle lines seen, words encountered, cards mined) and maintains daily and monthly rollups. You can query the database directly with any SQLite tool to track your progress over time.
|
||||
When enabled, SubMiner records per-session statistics (watch time, subtitle lines seen, words encountered, cards mined) and maintains daily and monthly rollups. You can view that data in SubMiner's stats UI or query the database directly with any SQLite tool.
|
||||
|
||||
## Enabling
|
||||
|
||||
@@ -18,6 +18,41 @@ When enabled, SubMiner records per-session statistics (watch time, subtitle line
|
||||
- Leave `dbPath` empty to use the default location (`immersion.sqlite` in SubMiner's app-data directory).
|
||||
- Set an explicit path to move the database (useful for backups, cloud syncing, or external tools).
|
||||
|
||||
## Stats Dashboard
|
||||
|
||||
The same immersion data powers the stats dashboard.
|
||||
|
||||
- In-app overlay: focus the visible overlay, then press the key from `stats.toggleKey` (default: `` ` `` / `Backquote`).
|
||||
- Launcher command: run `subminer stats` to start the local stats server on demand and open the dashboard in your browser.
|
||||
- Maintenance command: run `subminer stats cleanup` or `subminer stats cleanup -v` to backfill/repair vocabulary metadata (`headword`, `reading`, POS) and purge stale or excluded rows from `imm_words` on demand.
|
||||
- Browser page: open `http://127.0.0.1:5175` directly if the local stats server is already running.
|
||||
|
||||
Dashboard tabs:
|
||||
|
||||
- Overview: recent sessions, quick totals, short-term watch-time chart
|
||||
- Trends: watch time, cards mined, cards/hour, lookup hit rate over time
|
||||
- Sessions: per-session breakdown with timeline and event drill-down
|
||||
- Vocabulary: top words, kanji frequency, recent vocabulary growth, and click-through occurrence drilldown in a right-side drawer
|
||||
|
||||
Stats server config lives under `stats`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"stats": {
|
||||
"toggleKey": "Backquote",
|
||||
"serverPort": 5175,
|
||||
"autoStartServer": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `toggleKey` is overlay-local, not a system-wide shortcut.
|
||||
- `serverPort` controls the localhost dashboard URL.
|
||||
- `autoStartServer` starts the local stats HTTP server on launch once immersion tracking is active.
|
||||
- `subminer stats` forces the dashboard server to start even when `autoStartServer` is `false`.
|
||||
- `subminer stats` fails with an error when `immersionTracking.enabled` is `false`.
|
||||
- `subminer stats cleanup` defaults to vocabulary cleanup, repairs stale `headword`, `reading`, and `part_of_speech` values, attempts best-effort MeCab backfill for legacy rows, and removes rows that still fail vocab filtering.
|
||||
|
||||
## Retention Defaults
|
||||
|
||||
Data is kept for the following durations before automatic cleanup:
|
||||
|
||||
@@ -73,9 +73,9 @@ features:
|
||||
src: /assets/tokenization.svg
|
||||
alt: Tracking chart icon
|
||||
title: Immersion Tracking
|
||||
details: Logs watch time, words encountered, and cards mined to SQLite with daily and monthly rollups for long-term progress tracking.
|
||||
details: Logs watch time, words encountered, and cards mined to SQLite, then surfaces the same data in a local stats dashboard with rollups and session drill-down.
|
||||
link: /immersion-tracking
|
||||
linkText: Tracking details
|
||||
linkText: Stats details
|
||||
- icon:
|
||||
src: /assets/cross-platform.svg
|
||||
alt: Cross-platform icon
|
||||
@@ -102,7 +102,7 @@ const demoAssetVersion = '20260223-2';
|
||||
<div class="workflow-step" style="animation-delay: 60ms">
|
||||
<div class="step-number">02</div>
|
||||
<div class="step-title">Lookup</div>
|
||||
<div class="step-desc">Hover or click a token in the interactive overlay to open Yomitan context.</div>
|
||||
<div class="step-desc">Hover a token in the interactive overlay, then trigger Yomitan lookup to open context.</div>
|
||||
</div>
|
||||
<div class="workflow-connector" aria-hidden="true"></div>
|
||||
<div class="workflow-step" style="animation-delay: 120ms">
|
||||
|
||||
@@ -4,10 +4,10 @@ This guide walks through the sentence mining loop — from watching a video to c
|
||||
|
||||
## Overview
|
||||
|
||||
SubMiner runs as a transparent overlay on top of mpv. As subtitles play, the overlay displays them as interactive text. You click a word to look it up with Yomitan, then create an Anki card with a single action. SubMiner automatically attaches the sentence, audio clip, and screenshot.
|
||||
SubMiner runs as a transparent overlay on top of mpv. As subtitles play, the overlay displays them as interactive text. You hover a word, trigger Yomitan lookup with your configured lookup key/modifier, then create an Anki card with a single action. SubMiner automatically attaches the sentence, audio clip, and screenshot.
|
||||
|
||||
```text
|
||||
Watch video → See subtitle → Click word → Yomitan lookup → Add to Anki
|
||||
Watch video → See subtitle → Hover word + trigger lookup → Yomitan popup → Add to Anki
|
||||
↓
|
||||
SubMiner auto-fills:
|
||||
sentence, audio, image, translation
|
||||
@@ -30,9 +30,9 @@ SubMiner uses one overlay window with modal surfaces.
|
||||
|
||||
### Primary Subtitle Layer
|
||||
|
||||
The visible overlay renders subtitles as tokenized, clickable word spans. Each word is a separate element with reading and headword data attached. This plane is styled independently from mpv subtitles and supports:
|
||||
The visible overlay renders subtitles as tokenized hoverable word spans. Each word is a separate element with reading and headword data attached. This plane is styled independently from mpv subtitles and supports:
|
||||
|
||||
- Word-level click targets for Yomitan lookup
|
||||
- Word-level hover targets for Yomitan lookup
|
||||
- Auto pause/resume on subtitle hover (enabled by default via `subtitleStyle.autoPauseVideoOnHover`)
|
||||
- Optional pause while the Yomitan popup is open (`subtitleStyle.autoPauseVideoOnYomitanPopup`)
|
||||
- Right-click to pause/resume
|
||||
@@ -55,9 +55,10 @@ Jimaku search, field-grouping, runtime options, and manual subsync open as modal
|
||||
## Looking Up Words
|
||||
|
||||
1. Hover over the subtitle area — the overlay activates pointer events.
|
||||
2. Click any word. SubMiner uses Unicode-aware boundary detection (`Intl.Segmenter`) to select it. On macOS, hovering is enough.
|
||||
3. Yomitan detects the selection and opens its lookup popup.
|
||||
4. From the popup, add the word to Anki.
|
||||
2. Hover the word you want. SubMiner keeps per-token boundaries so Yomitan can target that token cleanly.
|
||||
3. Trigger Yomitan lookup with your configured lookup key/modifier (for example `Shift` if that is how your Yomitan profile is set up).
|
||||
4. Yomitan opens its lookup popup for the hovered token.
|
||||
5. From the popup, add the word to Anki.
|
||||
|
||||
### Controller Workflow
|
||||
|
||||
@@ -83,7 +84,7 @@ There are three ways to create cards, depending on your workflow.
|
||||
|
||||
This is the most common flow. Yomitan creates a card in Anki, and SubMiner enriches it automatically.
|
||||
|
||||
1. Click a word → Yomitan popup appears.
|
||||
1. Hover a word, then trigger Yomitan lookup → Yomitan popup appears.
|
||||
2. Click the Anki icon in Yomitan to add the word.
|
||||
3. SubMiner receives or detects the new card:
|
||||
- **Proxy mode** (`ankiConnect.proxy.enabled: true`): immediate enrich after successful `addNote` / `addNotes`.
|
||||
@@ -194,7 +195,7 @@ See [Subtitle Annotations — N+1](/subtitle-annotations#n1-word-highlighting) f
|
||||
|
||||
## Immersion Tracking
|
||||
|
||||
SubMiner can log your watching and mining activity to a local SQLite database — session times, words seen, cards mined, and daily/monthly rollups.
|
||||
SubMiner can log your watching and mining activity to a local SQLite database and expose it in the built-in stats dashboard — session times, words seen, cards mined, and daily/monthly rollups.
|
||||
|
||||
Enable it in your config:
|
||||
|
||||
@@ -205,6 +206,8 @@ Enable it in your config:
|
||||
}
|
||||
```
|
||||
|
||||
See [Immersion Tracking](/immersion-tracking) for the full schema and retention settings.
|
||||
Open the dashboard in the overlay with `stats.toggleKey` (default: `` ` ``), launch it in a browser with `subminer stats`, or visit `http://127.0.0.1:5175` directly if the local stats server is already running.
|
||||
|
||||
See [Immersion Tracking](/immersion-tracking) for dashboard details, schema, and retention settings.
|
||||
|
||||
Next: [Anki Integration](/anki-integration) — field mapping, media generation, and card enrichment configuration.
|
||||
|
||||
@@ -461,5 +461,16 @@
|
||||
"monthlyRollupsDays": 1825, // Monthly rollup retention window in days.
|
||||
"vacuumIntervalDays": 7 // Minimum days between VACUUM runs.
|
||||
} // Retention setting.
|
||||
} // Enable/disable immersion tracking.
|
||||
}, // Enable/disable immersion tracking.
|
||||
|
||||
// ==========================================
|
||||
// Stats Dashboard
|
||||
// Local immersion stats dashboard served on localhost and available as an in-app overlay.
|
||||
// Uses the immersion tracking database for overview, trends, sessions, and vocabulary views.
|
||||
// ==========================================
|
||||
"stats": {
|
||||
"toggleKey": "Backquote", // Key code to toggle the stats overlay.
|
||||
"serverPort": 5175, // Port for the stats HTTP server.
|
||||
"autoStartServer": true // Automatically start the stats server on launch. Values: true | false
|
||||
} // Local immersion stats dashboard served on localhost and available as an in-app overlay.
|
||||
}
|
||||
|
||||
@@ -68,6 +68,9 @@ Mouse-hover playback behavior is configured separately from shortcuts: `subtitle
|
||||
| `Ctrl/Cmd+Shift+O` | Open runtime options palette | `shortcuts.openRuntimeOptions` |
|
||||
| `Ctrl+Shift+J` | Open Jimaku subtitle search modal | `shortcuts.openJimaku` |
|
||||
| `Ctrl+Alt+S` | Open subtitle sync (subsync) modal | `shortcuts.triggerSubsync` |
|
||||
| `` ` `` | Toggle stats overlay | `stats.toggleKey` |
|
||||
|
||||
The stats toggle is handled inside the focused visible overlay window. It is configurable through the top-level `stats.toggleKey` setting and defaults to `Backquote`.
|
||||
|
||||
## Controller Shortcuts
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ Set `refreshMinutes` to `1440` (24 hours) for daily sync if your Anki collection
|
||||
|
||||
## Character-Name Highlighting
|
||||
|
||||
Character-name matches are built from the active merged SubMiner character dictionary, which auto-syncs character data from AniList for your recently-watched titles. Matching names are highlighted in subtitles and become clickable for full character profiles — portraits, roles, voice actors, and biographical detail.
|
||||
Character-name matches are built from the active merged SubMiner character dictionary, which auto-syncs character data from AniList for your recently-watched titles. Matching names are highlighted in subtitles and become available for hover-driven Yomitan character profiles — portraits, roles, voice actors, and biographical detail.
|
||||
|
||||
**How it works:**
|
||||
|
||||
|
||||
@@ -178,12 +178,12 @@ SubMiner does not load the source tree directly from `vendor/subminer-yomitan`;
|
||||
|
||||
If you installed from the AppImage and see this error, the package may be incomplete. Re-download the AppImage or place the unpacked Yomitan extension manually in `~/.config/SubMiner/yomitan`.
|
||||
|
||||
**Yomitan popup does not appear when clicking words**
|
||||
**Yomitan popup does not appear when hovering words and triggering lookup**
|
||||
|
||||
- Verify Yomitan loaded successfully — check the terminal output for "Loaded Yomitan extension".
|
||||
- Yomitan requires dictionaries to be installed. Open Yomitan settings (`Alt+Shift+Y` or `SubMiner.AppImage --settings`) and confirm at least one dictionary is imported.
|
||||
- If `yomitan.externalProfilePath` is set, import/check dictionaries in the external app/profile instead. SubMiner treats that profile as read-only and does not open its own Yomitan settings window.
|
||||
- If the overlay shows subtitles but words are not clickable, the tokenizer may have failed. See the MeCab section below.
|
||||
- If the overlay shows subtitles but hover lookup never resolves on tokens, the tokenizer may have failed. See the MeCab section below.
|
||||
|
||||
## MeCab / Tokenization
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
3. The overlay connects and subscribes to subtitle changes
|
||||
4. Subtitles are tokenized with Yomitan's internal parser
|
||||
5. Words are displayed as interactive spans in the overlay
|
||||
6. Hovering or clicking a word triggers Yomitan popup for dictionary lookup
|
||||
6. Hover a word, then trigger Yomitan lookup with your configured lookup key/modifier to open the Yomitan popup
|
||||
7. Optional [subtitle annotations](/subtitle-annotations) (N+1, character-name, frequency, JLPT) highlight useful cues in real time
|
||||
|
||||
There are two ways to use SubMiner:
|
||||
|
||||
Reference in New Issue
Block a user