sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-07-27 04:49:49 -07:00
Code Issues Packages Projects Releases Wiki Activity

Sync Stats & History window, headless --sync-cli, and Windows remote support (#160)

Browse Source
This commit is contained in:
Kyle Yasuda
2026-07-13 18:56:51 -07:00
committed by GitHub
parent 66f8ca4f80
commit 49b926e08c
111 changed files with 6983 additions and 1130 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs-site/changelog.md
+6
View File
@@ -1,5 +1,11 @@
# Changelog
## Unreleased
**Added**
- Stats Sync Without the Launcher: The stats sync engine now runs only inside the app. The sync window and the `subminer sync` command both delegate to `SubMiner --sync-cli` (headless, works over SSH with no display), so neither machine needs bun or the command-line launcher. A remote machine only needs SubMiner itself, found automatically as the app binary or via the launcher proxy.
- Stats Sync With Windows Remotes: Sync now detects the remote shell (POSIX, cmd, or PowerShell) and manages remote temp files through SubMiner itself (`sync --make-temp`/`--remove-temp`) instead of `mktemp`/`rm`, so a Windows machine with the built-in OpenSSH Server works as a sync remote; SubMiner is found in its default Windows install location automatically.
## v0.18.0 (2026-07-10)
**Added**
docs-site/launcher-script.md
+66 -43
View File
@@ -81,21 +81,27 @@ Series whose directories are not currently accessible (e.g. an unmounted network
## Sync Between Machines
`subminer sync <host>` merges immersion stats and watch history between two machines over SSH, so both end up with the union of sessions, lifetime totals, vocabulary counts, daily/monthly charts, and `--history` entries. `<host>` is anything `ssh` accepts (`user@hostname` or an ssh config alias); SubMiner must be installed on both machines at the same version.
`subminer sync <host>` merges immersion stats and watch history between two machines over SSH, so both end up with the union of sessions, lifetime totals, vocabulary counts, daily/monthly charts, and `--history` entries. `<host>` is anything `ssh` accepts (`user@hostname` or an ssh config alias); SubMiner must be installed on both machines at the same version. The sync engine runs only inside the app (`SubMiner --sync-cli sync ...`): the sync window spawns it that way, `subminer sync` is a thin proxy that forwards to the installed app, and the remote side is found automatically whether it has the launcher or just the app. The command-line launcher is optional everywhere.
```bash
subminer sync macbook # two-way sync with the host "macbook"
subminer sync macbook --push # merge local data into macbook only
subminer sync macbook --pull # merge macbook data into local only
subminer sync user@192.168.1.20 # explicit user@host
subminer sync macbook --remote-cmd ~/bin/subminer # custom remote launcher path
subminer sync macbook --remote-cmd ~/bin/subminer # custom remote SubMiner/launcher path
subminer sync macbook --check # test SSH + remote SubMiner without syncing
subminer sync --ui # open the sync window (also in the tray menu)
```
How it works: each side takes a consistent snapshot of its database (`VACUUM INTO`), the snapshots are exchanged over `scp`, and each machine merges the other's snapshot into its own database. The merge is an insert-only union keyed on stable identifiers (session UUIDs, video keys, series title keys, word/kanji identity), so it is safe to re-run at any time — syncing twice changes nothing, and nothing is ever overwritten or summed twice. Lifetime totals and rollup charts are updated incrementally, so history older than the session retention window is preserved on both sides.
How it works: each side takes a consistent snapshot of its database (`VACUUM INTO`), the snapshots are exchanged over `scp`, and each machine merges the other's snapshot into its own database. The merge is an insert-only union keyed on stable identifiers (session UUIDs, video keys, series title keys, word/kanji identity), so it is safe to re-run at any time. Syncing twice changes nothing, and nothing is ever overwritten or summed twice. Lifetime totals and rollup charts are updated incrementally, so history older than the session retention window is preserved on both sides.
For a one-way transfer, `--push` snapshots the local database and merges it into the host without changing the local database. `--pull` snapshots the host and merges it into the local database without changing the host. These modes add missing data; they do not delete destination-only data or make the destination an exact mirror.
Close SubMiner (and stop the background stats daemon, `subminer stats -s`) on both machines before syncing; the command refuses to run while a SubMiner process may be writing the database (`--force` overrides). The mpv safety check requires a live socket connection, so a stale socket file left after mpv exits does not block sync. Both machines must be on the same SubMiner version — the sync aborts on a stats schema mismatch. Remote sync checks standard SubMiner and Bun locations (`~/.local/bin`, `~/.bun/bin`, Homebrew, `/usr/local/bin`, `/usr/bin`, and `/bin`) even when the non-interactive SSH shell omits them from `PATH`.
Command-line sync defaults to a cold-start safety check: close SubMiner (and stop the background stats daemon with `subminer stats -s`) on both machines before running it, or pass `--force`. Syncs started from the Sync window use live mode automatically, including scheduled auto-syncs while SubMiner or playback is active. SQLite WAL provides a consistent snapshot, the transactional merge serializes with live writes, and each machine's unfinished session is excluded from the transfer; that session syncs normally after it finishes. The mpv safety check requires a live socket connection, so a stale socket file left after mpv exits does not block command-line sync. Both machines must be on the same SubMiner version; otherwise, the sync aborts on a stats schema mismatch.
On the remote, sync looks for the `subminer` launcher first (PATH and `~/.local/bin`), then the app binary in `--sync-cli` mode (`SubMiner` on PATH, then the standard macOS `/Applications` and `~/Applications` installs), checking standard SubMiner and Bun locations (`~/.local/bin`, `~/.bun/bin`, Homebrew, `/usr/local/bin`, `/usr/bin`, and `/bin`) even when the non-interactive SSH shell omits them from `PATH`. An AppImage in a custom location can be addressed with `--remote-cmd /path/to/SubMiner.AppImage` (or symlink it as `SubMiner` somewhere on the remote PATH).
Windows remotes are supported: enable Windows' built-in **OpenSSH Server** and sync detects the remote shell (cmd or PowerShell) automatically, finding SubMiner in its default install location (`%LOCALAPPDATA%\Programs\SubMiner`), the launcher shim (`%LOCALAPPDATA%\SubMiner\bin`), or on PATH. Temp files on the remote are created and removed by SubMiner itself (`sync --make-temp` / `--remove-temp`), so no POSIX tools are required on the remote side.
Two lower-level modes are used internally over SSH and also work standalone for manual transfers (e.g. via a USB drive):
@@ -104,7 +110,22 @@ subminer sync --snapshot /tmp/stats.sqlite # write a consistent snapshot of th
subminer sync --merge /tmp/stats.sqlite # merge a snapshot file into the local database
```
Unfinished sessions (a crash mid-playback) are skipped until the app finalizes them; they sync on the next run. Word/kanji "known" state from Anki is not part of the database and does not sync — each machine derives it from its own Anki collection.
Unfinished sessions (a crash mid-playback) are skipped until the app finalizes them; they sync on the next run. Word/kanji "known" state from Anki is not part of the database and does not sync. Each machine derives it from its own Anki collection.
`subminer sync <host> --check` verifies a host without touching any data: it probes the SSH connection, locates SubMiner on the remote (launcher or app binary), and reports its version. `--json` switches any sync mode to machine-readable NDJSON progress output (this is what the sync window consumes).
`sync --make-temp` creates a restricted temporary directory and prints its path; `sync --remove-temp <dir>` removes one created by that command. They are internal SSH transfer helpers, exposed for compatibility but normally invoked only by sync itself. `SubMiner --sync-cli sync ...` is the packaged app's headless compatibility entrypoint; use `SubMiner --sync-cli --help` for its sync-specific help. The `subminer sync` launcher command selects this entrypoint automatically and runs AppImages in Node-only mode, so remote sync does not require a graphical session.
### Sync window
`subminer sync --ui` opens a dedicated window for the same engine in a detached app process, returning the shell immediately. Closing that standalone-launched window exits its app instance. Opening **Sync Stats & History** from the tray keeps the resident app running when the window closes:
- **Devices:** saved hosts with a per-host direction (two-way / push / pull), an auto-sync toggle, last-sync status, and one-click **Sync now** / **Test** / **Remove**. Hosts synced from the command line appear here automatically.
- **Add a device:** test SSH + remote SubMiner availability before saving, with a setup checklist for first-time SSH configuration.
- **Activity:** live stage-by-stage progress, remote output, and separate merge summaries (sessions, words, kanji, rollups) for each machine updated by the run. Runs can be cancelled and can proceed while the app, stats server, or playback is active.
- **Snapshots:** create manual database snapshots (stored in `/tmp/subminer-db-snapshots/` by default), merge a snapshot file into the local database, or reveal/delete existing snapshots.
Hosts with **Auto-sync** enabled are synced in the background on a configurable interval (default every 60 minutes), including during active playback; results surface as overlay notifications. The unfinished playback session is skipped until a later sync sees it finalized. Host bookkeeping lives in `<config dir>/sync-hosts.json`.
## Common Commands
@@ -120,51 +141,53 @@ subminer stats -b # start background stats daemon
## Subcommands
| Subcommand | Purpose |
| ------------------------------------------ | ------------------------------------------------------------------ |
| `subminer jellyfin` / `jf` | Jellyfin workflows (`-d` discovery, `-p` play, `-l` login, `--logout`, `--setup`) |
| `subminer stats` | Start the stats server (opens the dashboard when `stats.autoOpenBrowser` is on) |
| `subminer stats -b` / `-s` | Start/reuse or stop the background stats daemon |
| `subminer stats cleanup` | Backfill vocabulary metadata and prune stale rows (`-v` vocab, `-l` lifetime summaries) |
| `subminer stats rebuild` / `backfill` | Rebuild or backfill rollup data |
| Subcommand | Purpose |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------- |
| `subminer jellyfin` / `jf` | Jellyfin workflows (`-d` discovery, `-p` play, `-l` login, `--logout`, `--setup`) |
| `subminer stats` | Start the stats server (opens the dashboard when `stats.autoOpenBrowser` is on) |
| `subminer stats -b` / `-s` | Start/reuse or stop the background stats daemon |
| `subminer stats cleanup` | Backfill vocabulary metadata and prune stale rows (`-v` vocab, `-l` lifetime summaries) |
| `subminer stats rebuild` / `backfill` | Rebuild or backfill rollup data |
| `subminer doctor` | Dependency + config + socket diagnostics (`--refresh-known-words` refreshes the known-word cache) |
| `subminer settings` | Open the SubMiner settings window |
| `subminer logs -e` | Export a sanitized local-date log ZIP and print its path |
| `subminer config path` | Print active config file path |
| `subminer config show` | Print active config contents |
| `subminer mpv status` | Check mpv socket readiness |
| `subminer mpv socket` | Print active socket path |
| `subminer mpv idle` | Launch detached idle mpv instance |
| `subminer sync <host>` | Two-way stats/history sync with another machine over SSH |
| `subminer sync <host> --push` | Merge local stats/history into another machine only |
| `subminer sync <host> --pull` | Merge another machine's stats/history into the local database only |
| `subminer dictionary <path>` / `dict` | Generate character dictionary ZIP from file/dir target |
| `subminer dictionary --candidates <path>` | List AniList candidate matches for character dictionary correction |
| `subminer dictionary --select <id> <path>` | Pin an AniList media ID for that target series |
| `subminer texthooker` | Launch texthooker-only mode |
| `subminer texthooker -o` | Launch texthooker and open it in the default browser |
| `subminer app` / `bin` | Pass arguments directly to SubMiner binary (e.g. `subminer app --setup`) |
| `subminer settings` | Open the SubMiner settings window |
| `subminer logs -e` | Export a sanitized local-date log ZIP and print its path |
| `subminer config path` | Print active config file path |
| `subminer config show` | Print active config contents |
| `subminer mpv status` | Check mpv socket readiness |
| `subminer mpv socket` | Print active socket path |
| `subminer mpv idle` | Launch detached idle mpv instance |
| `subminer sync <host>` | Two-way stats/history sync with another machine over SSH |
| `subminer sync <host> --push` | Merge local stats/history into another machine only |
| `subminer sync <host> --pull` | Merge another machine's stats/history into the local database only |
| `subminer sync <host> --check` | Test SSH connection and remote launcher availability |
| `subminer sync --ui` | Open the sync window (saved devices, auto-sync, snapshots) |
| `subminer dictionary <path>` / `dict` | Generate character dictionary ZIP from file/dir target |
| `subminer dictionary --candidates <path>` | List AniList candidate matches for character dictionary correction |
| `subminer dictionary --select <id> <path>` | Pin an AniList media ID for that target series |
| `subminer texthooker` | Launch texthooker-only mode |
| `subminer texthooker -o` | Launch texthooker and open it in the default browser |
| `subminer app` / `bin` | Pass arguments directly to SubMiner binary (e.g. `subminer app --setup`) |
Use `subminer <subcommand> -h` for command-specific help.
## Options
| Flag | Description |
| --------------------- | -------------------------------------------------------------------- |
| `-d, --directory` | Video search directory (default: cwd) |
| `-r, --recursive` | Search directories recursively |
| `-R, --rofi` | Use rofi instead of fzf |
| `-H, --history` | Browse local watch history (see [Watch History](#watch-history)) |
| Flag | Description |
| --------------------- | --------------------------------------------------------------------------- |
| `-d, --directory` | Video search directory (default: cwd) |
| `-r, --recursive` | Search directories recursively |
| `-R, --rofi` | Use rofi instead of fzf |
| `-H, --history` | Browse local watch history (see [Watch History](#watch-history)) |
| `-v, --version` | Print the launcher's own version (can differ from the installed app binary) |
| `-u, --update` | Check for SubMiner updates and update the app/launcher when possible |
| `--start` | Explicitly start overlay after mpv launches |
| `-S, --start-overlay` | Force the visible overlay on start |
| `-T, --no-texthooker` | Disable texthooker server |
| `-p, --profile` | mpv profile name (no default; omitted unless set) |
| `-a, --args` | Pass additional mpv arguments as a quoted string |
| `-b, --backend` | Force window backend (`hyprland`, `sway`, `x11`, `macos`, `windows`) |
| `--settings` | Open the SubMiner settings window |
| `--log-level` | Logger verbosity (`debug`, `info`, `warn`, `error`) |
| `-u, --update` | Check for SubMiner updates and update the app/launcher when possible |
| `--start` | Explicitly start overlay after mpv launches |
| `-S, --start-overlay` | Force the visible overlay on start |
| `-T, --no-texthooker` | Disable texthooker server |
| `-p, --profile` | mpv profile name (no default; omitted unless set) |
| `-a, --args` | Pass additional mpv arguments as a quoted string |
| `-b, --backend` | Force window backend (`hyprland`, `sway`, `x11`, `macos`, `windows`) |
| `--settings` | Open the SubMiner settings window |
| `--log-level` | Logger verbosity (`debug`, `info`, `warn`, `error`) |
App-binary flags such as `--setup`, `--dev`, and `--debug` are not launcher flags - pass them through with `subminer app`, for example `subminer app --setup`.
docs-site/usage.md
+10 -1
View File
@@ -122,8 +122,13 @@ subminer mpv idle # Launch detached idle mpv with SubMiner defau
subminer sync media-box # Sync stats/watch history with an SSH host
subminer sync media-box --push # Merge this machine's stats into the host only
subminer sync media-box --pull # Merge the host's stats into this machine only
subminer sync media-box --check # Verify SSH and remote SubMiner without syncing
subminer sync media-box --json # Emit machine-readable NDJSON progress
subminer sync --ui # Open the Sync Stats & History window
subminer sync --snapshot ~/subminer-snapshot.sqlite # Write a local DB snapshot
subminer sync --merge ~/subminer-snapshot.sqlite # Merge a snapshot into the local DB
subminer sync --make-temp # Create an internal sync temp directory
subminer sync --remove-temp /tmp/subminer-sync-123 # Remove an internal sync temp directory
subminer dictionary /path/to/file-or-directory # Generate character dictionary ZIP from target (manual Yomitan import)
subminer dictionary --candidates /path/to/file.mkv
subminer dictionary --select 21355 /path/to/file.mkv
@@ -159,12 +164,16 @@ SubMiner.AppImage --jellyfin-libraries
SubMiner.AppImage --jellyfin-items --jellyfin-library-id LIBRARY_ID --jellyfin-search anime --jellyfin-limit 20
SubMiner.AppImage --jellyfin-play --jellyfin-item-id ITEM_ID --jellyfin-audio-stream-index 1 --jellyfin-subtitle-stream-index 2 # Requires connected mpv IPC (--start)
SubMiner.AppImage --jellyfin-remote-announce # Force cast-target capability announce + visibility check
SubMiner.AppImage --sync-cli --help # Show the packaged app's headless sync help
SubMiner.AppImage --sync-cli sync media-box # Run the sync engine directly in headless mode
SubMiner.AppImage --dictionary # Generate character dictionary ZIP for current anime
SubMiner.AppImage --dictionary-candidates # List AniList candidates for current character dictionary series
SubMiner.AppImage --dictionary-select --dictionary-anilist-id 21355 # Pin correct AniList media for series
SubMiner.AppImage --help # Show all options
```
`--check` performs connection and version checks without changing data. `--json` emits the NDJSON event protocol used by the sync window. `--ui` opens that window in a detached app process and returns the shell immediately; closing a standalone-launched Sync window exits that app instance. `--make-temp` and `--remove-temp` are internal remote-transfer helpers and should normally be left to SubMiner. The packaged app's `--sync-cli` flag selects its headless sync-compatible entrypoint; the `subminer sync` launcher command proxies to it automatically.
The tray menu includes `Export Logs`, which creates the same sanitized local-date log ZIP as `subminer logs -e` and shows the archive path when complete. Export sanitization masks common PII and secrets, including home-directory usernames, IP addresses, emails, auth/cookie headers, yt-dlp cookie arguments, URL credentials, token/key/password fields, and signed YouTube media URL query strings. The exported copy is sanitized; source log files remain unredacted on disk.
Once Jellyfin is configured, the tray menu includes `Jellyfin Discovery` for starting or stopping cast discovery in the current app session without changing config.
@@ -367,7 +376,7 @@ See [Keyboard Shortcuts](/shortcuts) for the full reference, including mining sh
| Keybind | Action | Scope |
| ------------- | ---------------------- | -------------------------------------------------------------------------------------------------- |
| `Alt+Shift+O` | Toggle visible overlay | Works while the overlay or mpv has focus (configurable via `shortcuts.toggleVisibleOverlayGlobal`) |
| `Alt+Shift+Y` | Open Yomitan settings | OS-global - registered with the system, works from any window |
| `Alt+Shift+Y` | Open Yomitan settings | OS-global - registered with the system, works from any window |
`Alt+Shift+Y` is fixed and not configurable. All other shortcuts can be changed under `shortcuts` in your config.