30 Commits

Author SHA1 Message Date
0ba3911a2d update 2026-03-15 15:40:30 -07:00
e940205479 fix: satisfy subtitle cue parser typecheck 2026-03-15 15:33:40 -07:00
d069df2124 fix: guard subtitle prefetch init races 2026-03-15 14:42:36 -07:00
87bf3cef0c fix: address coderabbit review findings 2026-03-15 14:26:30 -07:00
5c31be99b5 feat: wire up subtitle prefetch service to MPV events
Initializes prefetch on external subtitle track activation, detects
seeks via time-pos delta threshold, pauses prefetch during live
subtitle processing, and restarts on cache invalidation.

- Extract loadSubtitleSourceText into reusable function
- Add prefetch service state and initSubtitlePrefetch helper
- Thread onTimePosUpdate through event actions/bindings/main-deps
- Pause prefetch on subtitle change, resume on emit
- Restart prefetch after tokenization cache invalidation
- Query track-list on media path change to find external subs
2026-03-15 13:16:19 -07:00
05fe9c8fdf test: add seek and pause/resume tests for prefetch service 2026-03-15 13:04:50 -07:00
f89aec31e8 feat: add subtitle prefetch service with priority window
Implements background tokenization of upcoming subtitle cues with a
configurable priority window. Supports stop, pause/resume, seek
re-prioritization, and cache-full stopping condition.
2026-03-15 13:04:26 -07:00
6cf0272e7e feat: add unified parseSubtitleCues with format auto-detection 2026-03-15 13:01:50 -07:00
7ea303361d feat: add ASS subtitle cue parser 2026-03-15 13:01:34 -07:00
95ec3946d2 feat: add SRT/VTT subtitle cue parser 2026-03-15 13:01:08 -07:00
d71e9e841e perf: use cloneNode template and replaceChildren for DOM rendering
Replace createElement('span') with cloneNode(false) from a lazily
initialized template span. Replace innerHTML='' with replaceChildren()
to avoid HTML parser invocation on clear. Add cloneNode/replaceChildren
to FakeElement in tests to support the new APIs.
2026-03-15 12:52:56 -07:00
047b349d05 perf: batch annotation passes into single loop
Collapse applyKnownWordMarking, applyFrequencyMarking, and
applyJlptMarking into a single .map() call. markNPlusOneTargets
remains a separate pass (needs full array with isKnown set).

Eliminates 3 intermediate array allocations and 3 redundant
iterations over the token array.
2026-03-15 12:49:08 -07:00
35946624c2 feat: add preCacheTokenization and isCacheFull to SubtitleProcessingController 2026-03-15 12:46:43 -07:00
bb13e3c895 docs: add renderer performance implementation plan
Detailed TDD implementation plan for 3 optimizations:
- Subtitle prefetching with priority window
- Batched annotation passes
- DOM template pooling via cloneNode
2026-03-15 12:42:35 -07:00
36181d8dfc docs: address second round of spec review feedback
Add isCacheFull() method to controller interface for prefetcher
stopping condition, specify pause flag mechanism for live priority,
and clarify replaceChildren() applies to all subtitle root clears.
2026-03-15 12:22:00 -07:00
40117d3b73 docs: address spec review feedback
Clarify frequency rank assignment vs filtering pipeline, add cache
capacity strategy for prefetch, specify seek detection threshold,
document cache invalidation re-prefetch behavior, detail ASS parsing
edge cases, add error handling section, and use replaceChildren().
2026-03-15 12:19:35 -07:00
7fcd3e8e94 docs: add renderer performance optimization design spec
Covers three optimizations to minimize subtitle-to-annotation latency:
subtitle prefetching with prioritized sliding window, batched annotation
passes, and DOM template pooling via cloneNode.
2026-03-15 12:15:46 -07:00
467ed02c80 fix(launcher): default stats cleanup to vocab mode 2026-03-15 00:19:08 -07:00
e68defbedf docs: update stats dashboard docs, config, and releasing checklist
- Update dashboard tab descriptions to include Anime tab and richer session timelines
- Add autoOpenBrowser config option to stats section
- Add subminer stats cleanup command to changelog fragment
- Expand releasing checklist with doc verification, changelog lint, and build gates
2026-03-14 23:11:27 -07:00
c4bea1f9ca feat(stats): redesign session timeline and clean up vocabulary tab
- Replace cumulative line chart with activity-focused area chart showing per-interval new words
- Add total words as a blue line on a secondary right Y-axis
- Add pause shaded regions, seek markers, and card mined markers with numeric x-axis for reliable rendering
- Add clickable header logo with proper aspect ratio
- Remove unused "Hide particles & single kana" checkbox from vocabulary tab
2026-03-14 23:11:27 -07:00
7d76c44f7f feat(stats): add episodes completed and anime completed to tracking snapshot
- Query watched videos count and anime with all episodes watched
- Display in overview tracking snapshot
- Remove efficiency section from trends
2026-03-14 23:11:27 -07:00
5e944e8a17 chore: remove implementation plan documents 2026-03-14 23:11:27 -07:00
06745ff63a feat: add AniList rate limiter and remaining backlog tasks 2026-03-14 23:11:27 -07:00
fcfa323e0f docs: update documentation site for stats dashboard and immersion tracking 2026-03-14 23:11:27 -07:00
5506a75ef8 feat(stats): build anime-centric stats dashboard frontend
5-tab React dashboard with Catppuccin Mocha theme:
- Overview: hero stats, streak calendar, watch time chart, recent sessions
- Anime: grid with cover art, episode list with completion %, detail view
- Trends: 15 charts across Activity, Efficiency, Anime, and Patterns
- Vocabulary: POS-filtered word/kanji lists with detail panels
- Sessions: expandable session history with event timeline

Features:
- Cross-tab navigation (anime <-> vocabulary)
- Global word detail panel overlay
- Expandable episode detail with Anki card links (Expression field)
- Per-anime multi-line trend charts
- Watch time by day-of-week and hour-of-day
- Collapsible sections with accessibility (aria-expanded)
- Card size selector for anime grid
- Cover art caching via AniList
- HTTP API client with file:// protocol fallback for Electron overlay
2026-03-14 23:11:27 -07:00
e374e53d97 feat(stats): add launcher stats command and build integration
- Launcher stats subcommand with cleanup mode
- Stats frontend build integrated into Makefile
- CI workflow updated for stats package
- Config example updated with stats section
- mpv plugin menu entry for stats toggle
2026-03-14 23:11:27 -07:00
6d8650994f feat(stats): wire stats server, overlay, and CLI into main process
- Stats server auto-start on immersion tracker init
- Stats overlay toggle via keybinding and IPC
- Stats CLI command (subminer stats) with cleanup mode
- mpv plugin menu integration for stats toggle
- CLI args for --stats, --stats-cleanup, --stats-response-path
2026-03-14 23:11:27 -07:00
a7c294a90c feat(stats): add stats server, API endpoints, config, and Anki integration
- Hono HTTP server with 20+ REST endpoints for stats data
- Stats overlay BrowserWindow with toggle keybinding
- IPC channel definitions and preload bridge
- Stats config section (toggleKey, serverPort, autoStartServer, autoOpenBrowser)
- Config resolver for stats section
- AnkiConnect proxy endpoints (guiBrowse, notesInfo)
- Note ID passthrough in card mining callback chain
- Stats CLI command with autoOpenBrowser respect
2026-03-14 23:11:27 -07:00
f005f542a3 feat(immersion): add anime metadata, occurrence tracking, and schema upgrades
- Add imm_anime table with AniList integration
- Add imm_subtitle_lines, imm_word_line_occurrences, imm_kanji_line_occurrences
- Add POS fields (part_of_speech, pos1, pos2, pos3) to imm_words
- Add anime metadata parsing with guessit fallback
- Add video duration tracking and watched status
- Add episode, streak, trend, and word/kanji detail queries
- Deduplicate subtitle line recording within sessions
- Pass Anki note IDs through card mining callback chain
2026-03-14 23:11:27 -07:00
ee95e86ad5 docs: add stats dashboard design docs, plans, and knowledge base
- Stats dashboard redesign design and implementation plans
- Episode detail and Anki card link design
- Internal knowledge base restructure
- Backlog tasks for testing, verification, and occurrence tracking
2026-03-14 23:11:27 -07:00
233 changed files with 16874 additions and 341 deletions

View File

@@ -27,13 +27,16 @@ jobs:
path: |
~/.bun/install/cache
node_modules
stats/node_modules
vendor/subminer-yomitan/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('bun.lock', 'vendor/subminer-yomitan/package-lock.json') }}
key: ${{ runner.os }}-bun-${{ hashFiles('bun.lock', 'stats/bun.lock', 'vendor/subminer-yomitan/package-lock.json') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile
run: |
bun install --frozen-lockfile
cd stats && bun install --frozen-lockfile
- name: Lint changelog fragments
run: bun run changelog:lint
@@ -49,6 +52,9 @@ jobs:
- name: Verify generated config examples
run: bun run verify:config-example
- name: Internal docs knowledge-base checks
run: bun run test:docs:kb
- name: Test suite (source)
run: bun run test:fast

1
.gitignore vendored
View File

@@ -51,3 +51,4 @@ tests/*
.agents/skills/subminer-scrum-master/*
!.agents/skills/subminer-scrum-master/SKILL.md
favicon.png
!stats/public/favicon.png

View File

@@ -1,17 +1,29 @@
# AGENTS.MD
## Internal Docs
Start here, then leave this file.
- Internal system of record: [`docs/README.md`](./docs/README.md)
- Architecture map: [`docs/architecture/README.md`](./docs/architecture/README.md)
- Workflow map: [`docs/workflow/README.md`](./docs/workflow/README.md)
- Verification lanes: [`docs/workflow/verification.md`](./docs/workflow/verification.md)
- Knowledge-base rules: [`docs/knowledge-base/README.md`](./docs/knowledge-base/README.md)
- Release guide: [`docs/RELEASING.md`](./docs/RELEASING.md)
`docs-site/` is user-facing. Do not treat it as the canonical internal source of truth.
## Quick Start
- Read [`docs-site/development.md`](./docs-site/development.md) and [`docs-site/architecture.md`](./docs-site/architecture.md) before substantial changes; follow them unless task requires deviation.
- Init workspace: `git submodule update --init --recursive`.
- Install deps: `make deps` or `bun install` plus `(cd vendor/texthooker-ui && bun install --frozen-lockfile)`.
- Fast dev loop: `make dev-watch`.
- Full local run: `bun run dev`.
- Verbose Electron debug: `electron . --start --dev --log-level debug`.
- Init workspace: `git submodule update --init --recursive`
- Install deps: `make deps` or `bun install` plus `(cd vendor/texthooker-ui && bun install --frozen-lockfile)`
- Fast dev loop: `make dev-watch`
- Full local run: `bun run dev`
- Verbose Electron debug: `electron . --start --dev --log-level debug`
## Build / Test
- Use repo package manager/runtime only: Bun (`packageManager: bun@1.3.5`).
- Runtime/package manager: Bun (`packageManager: bun@1.3.5`)
- Default handoff gate:
`bun run typecheck`
`bun run test:fast`
@@ -21,59 +33,37 @@
- If `docs-site/` changed, also run:
`bun run docs:test`
`bun run docs:build`
- Formatting: prefer `make pretty` and `bun run format:check:src`; use `bun run format` only intentionally.
- Keep verification observable; capture failing command + exact error in notes/handoff.
- Prefer `make pretty` and `bun run format:check:src`
## Change-Specific Checks
- Config/schema/defaults changes: run `bun run test:config`; if config template/defaults changed, run `bun run generate:config-example`.
- Launcher/plugin changes: run `bun run test:launcher` or `bun run test:env`; use `bun run test:launcher:smoke:src` for focused launcher e2e checks.
- Runtime-compat or compiled/dist-sensitive changes: run `bun run test:runtime:compat`.
- Docs-only changes: at least `bun run docs:test` if docs behavior/assertions changed; `bun run docs:build` before handoff.
- Config/schema/defaults: `bun run test:config`; if template/defaults changed, `bun run generate:config-example`
- Launcher/plugin: `bun run test:launcher` or `bun run test:env`
- Runtime-compat / dist-sensitive: `bun run test:runtime:compat`
- Docs-only: `bun run docs:test`, then `bun run docs:build`
## Generated / Sensitive Files
## Sensitive Files
- Launcher source of truth: `launcher/*.ts`.
- Generated launcher artifact: `dist/launcher/subminer`; never hand-edit it.
- Repo-root `./subminer` is stale artifact path; do not revive/use it.
- `bun run build` rebuilds bundled Yomitan from `vendor/subminer-yomitan`; check submodules before debugging build failures.
- Avoid changing packaging/signing identifiers (`build.appId`, mac entitlements, signing-related settings) unless task explicitly requires it.
- Launcher source of truth: `launcher/*.ts`
- Generated launcher artifact: `dist/launcher/subminer`; never hand-edit it
- Repo-root `./subminer` is stale; do not revive it
- `bun run build` rebuilds bundled Yomitan from `vendor/subminer-yomitan`
- Do not change signing/packaging identifiers unless the task explicitly requires it
## Docs
## Release / PR Notes
- Docs site lives in-repo under [`docs-site/`](./docs-site/).
- Update docs for new/breaking behavior; no ship with stale docs.
- Make sure [`docs-site/changelog.md`](./docs-site/changelog.md) is updated on each release.
- User-visible PRs need one fragment in `changes/*.md`
- CI enforces `bun run changelog:lint` and `bun run changelog:pr-check`
- PR review helpers:
- `gh pr view --json number,title,url --jq '"PR #\\(.number): \\(.title)\\n\\(.url)"'`
- `gh api repos/:owner/:repo/pulls/<num>/comments --paginate`
## PR Feedback
## Runtime Notes
- Active PR: `gh pr view --json number,title,url --jq '"PR #\\(.number): \\(.title)\\n\\(.url)"'`.
- PR comments: `gh pr view …` + `gh api …/comments --paginate`.
- Replies: cite fix + file/line; resolve threads only after fix lands.
## Changelog
- User-visible PRs: add one fragment in `changes/*.md`.
- Fragment format:
`type: added|changed|fixed|docs|internal`
`area: <short-area>`
blank line
`- bullet`
- `changes/README.md`: instructions only; generator ignores it.
- No release-note entry wanted: use PR label `skip-changelog`.
- CI runs `bun run changelog:lint` + `bun run changelog:pr-check` on PRs.
- Release prep: `bun run changelog:build`, review `CHANGELOG.md` + `release/release-notes.md`, commit generated changelog + fragment deletions, then tag.
- Release CI expects committed changelog entry already present; do not rely on tag job to invent notes.
## Flow & Runtime
- Use Codex background for long jobs; tmux only for interactive/persistent (debugger/server).
- CI red: `gh run list/view`, rerun, fix, push, repeat til green.
## Language/Stack Notes
- Swift: use workspace helper/daemon; validate `swift build` + tests; keep concurrency attrs right.
- TypeScript: use repo PM; keep files small; follow existing patterns.
- Use Codex background for long jobs; tmux only when persistence/interaction is required
- CI red: `gh run list/view`, rerun, fix, repeat until green
- TypeScript: keep files small; follow existing patterns
- Swift: use workspace helper/daemon; validate `swift build` + tests
<!-- BACKLOG.MD MCP GUIDELINES START -->

View File

@@ -69,7 +69,7 @@ help:
" generate-config Generate ~/.config/SubMiner/config.jsonc from centralized defaults" \
"" \
"Other targets:" \
" deps Install JS dependencies (root + texthooker-ui)" \
" deps Install JS dependencies (root + stats + texthooker-ui)" \
" uninstall-linux Remove Linux install artifacts" \
" uninstall-macos Remove macOS install artifacts" \
" uninstall-windows Remove Windows mpv plugin artifacts" \
@@ -104,6 +104,7 @@ print-dirs:
deps:
@$(MAKE) --no-print-directory ensure-bun
@bun install
@cd stats && bun install --frozen-lockfile
@cd vendor/texthooker-ui && bun install --frozen-lockfile
ensure-bun:

View File

@@ -27,6 +27,7 @@ SubMiner is an Electron overlay that sits on top of mpv. It turns your video pla
- **Look up words as you watch** — Yomitan dictionary popups on hover or keyboard-driven token-by-token navigation
- **One-key Anki mining** — Creates cards with sentence, audio, screenshot, and translation; optional local AnkiConnect proxy auto-enriches Yomitan cards instantly
- **Reading annotations** — N+1 targeting, frequency-dictionary highlighting, JLPT underlining, and character name dictionary for anime/manga proper nouns
- **Immersion stats** — Optional local dashboard and overlay for watch time, anime progress, session drill-down, vocabulary growth, and mining throughput
- **Subtitle tools** — Download from Jimaku, sync with alass/ffsubsync
- **Jellyfin & AniList integration** — Remote playback, cast device mode, and automatic episode progress tracking
- **Texthooker & API** — Built-in texthooker page and annotated websocket feed for external clients
@@ -101,6 +102,7 @@ The mpv plugin step is optional. Yomitan must report at least one installed dict
```bash
subminer video.mkv # default plugin config auto-starts visible overlay + resumes playback when ready
subminer --start video.mkv # optional explicit overlay start when plugin auto_start=no
subminer stats # open the local stats dashboard in your browser
```
## Requirements
@@ -118,7 +120,7 @@ Windows builds use native window tracking and do not require the Linux composito
## Documentation
For full guides on configuration, Anki, Jellyfin, and more, see [docs.subminer.moe](https://docs.subminer.moe). The VitePress source for that site lives in [`docs-site/`](./docs-site/).
For full guides on configuration, Anki, Jellyfin, immersion tracking/stats, and more, see [docs.subminer.moe](https://docs.subminer.moe). The VitePress source for that site lives in [`docs-site/`](./docs-site/).
## Acknowledgments

View File

@@ -0,0 +1,64 @@
---
id: TASK-165
title: Rewrite SubMiner agentic testing automation plan
status: Done
assignee: []
created_date: '2026-03-13 04:45'
updated_date: '2026-03-13 04:47'
labels:
- planning
- testing
- agents
dependencies: []
references:
- /home/sudacode/projects/japanese/SubMiner/testing-plan.md
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-change-verification/SKILL.md
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-scrum-master/SKILL.md
documentation:
- /home/sudacode/projects/japanese/SubMiner/docs-site/development.md
- /home/sudacode/projects/japanese/SubMiner/docs-site/architecture.md
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Replace the current generic Electron/mpv testing plan with a SubMiner-specific plan that uses the existing skills as the source of truth, treats real launcher/plugin/mpv runtime verification as primary, and defines a non-interference contract for parallel agent work.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 `testing-plan.md` is rewritten for SubMiner rather than a generic Electron+mpv app
- [x] #2 The plan keeps `subminer-scrum-master` and `subminer-change-verification` as the primary orchestration and verification entrypoints
- [x] #3 The plan defines real launcher/plugin/mpv runtime verification as the authoritative lane for runtime bug claims
- [x] #4 The plan defines explicit session isolation and non-interference rules for parallel agent work
- [x] #5 The plan defines artifact/reporting expectations and phased rollout, with synthetic/headless verification clearly secondary to real-runtime verification
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Review the existing testing plan and compare it against current SubMiner architecture, verification lanes, and skills.
2. Replace the generic Electron/mpv harness framing with a SubMiner-specific control plane centered on existing skills.
3. Define the authoritative real-runtime lane, session isolation rules, concurrency classes, and reporting contract.
4. Sanity-check the rewritten document against current repo docs and skill contracts before handoff.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
Rewrote `testing-plan.md` around existing `subminer-scrum-master` and `subminer-change-verification` responsibilities instead of proposing a competing new top-level testing skill.
Set real launcher/plugin/mpv/runtime verification as the authoritative lane for runtime bug claims and made synthetic/headless verification explicitly secondary.
Defined session-scoped paths, unique mutable resources, concurrency classes, and an exclusive lease for conflicting real-runtime verification to prevent parallel interference.
Sanity-checked the final document by inspecting the rewritten file content and diff.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Rewrote `testing-plan.md` into a SubMiner-specific agentic verification plan. The new document keeps `subminer-scrum-master` and `subminer-change-verification` as the primary orchestration and verification entrypoints, treats the real launcher/plugin/mpv/runtime path as authoritative for runtime bug claims, and defines a hard non-interference contract for parallel work through session isolation and an exclusive real-runtime lease. The plan now also includes an explicit reporting schema, capture policy, phased rollout, and a clear statement that true parallel full-app instances are not a phase-1 requirement. Verification for this task was a document sanity pass against the current repo docs, skills, and the resulting file diff.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,64 @@
---
id: TASK-166
title: Harden SubMiner change verification for authoritative agentic runtime checks
status: Done
assignee: []
created_date: '2026-03-13 05:19'
updated_date: '2026-03-13 05:36'
labels:
- testing
- agents
- verification
dependencies: []
references:
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-change-verification/SKILL.md
documentation:
- /home/sudacode/projects/japanese/SubMiner/testing-plan.md
- /home/sudacode/projects/japanese/SubMiner/docs-site/development.md
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Tighten the SubMiner change-verification classifier and verifier so the implementation matches the approved agentic verification plan: authoritative runtime verification must fail closed when unavailable, lane naming should use real-runtime semantics, session and artifact identities must be unique, and the verifier must be safer for parallel agent use.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 The verifier uses `real-runtime` terminology instead of `real-gui` for authoritative runtime verification
- [x] #2 Requested authoritative runtime verification fails closed with a non-green outcome when it cannot run, and unknown lanes do not pass open
- [x] #3 The verifier allocates a unique session identifier and artifact root that does not rely on second-level timestamp uniqueness alone
- [x] #4 The verifier summary/report output includes explicit top-level status and session metadata needed for agent aggregation
- [x] #5 The classifier and verifier better reflect runtime-escalation cases for launcher/plugin/socket/runtime-sensitive changes
- [x] #6 Regression tests cover the new verifier/classifier behavior
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Add regression tests for classifier/verifier behavior before changing the scripts.
2. Harden `verify_subminer_change.sh` to use `real-runtime` terminology, fail closed for blocked/unknown authoritative verification, and emit unique session metadata in summaries.
3. Update `classify_subminer_diff.sh` and the skill doc to use `real-runtime` escalation language and better flag launcher/plugin/runtime-sensitive paths.
4. Run targeted regression tests plus a focused dry-run verifier check, then record outcomes and blockers in the task.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
Added `scripts/subminer-change-verification.test.ts` to regression-test classifier/verifier behavior around `real-runtime` naming, fail-closed authoritative verification, unknown lanes, and unique session metadata.
Reworked `verify_subminer_change.sh` to normalize `real-gui` to `real-runtime`, emit unique session IDs and richer summary metadata, block authoritative runtime verification when unavailable, and fail closed for unknown lanes.
Updated `classify_subminer_diff.sh` to emit `real-runtime-candidate` for launcher/plugin/runtime-sensitive paths, and updated the active skill doc wording to match the new lane terminology.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Hardened the SubMiner change-verification tooling to match the approved agentic verification plan. The verifier now uses `real-runtime` terminology for authoritative runtime verification, preserves compatibility with the deprecated `real-gui` alias, fails closed for unknown lanes, and returns a blocked non-green outcome when requested authoritative runtime verification cannot run. It now allocates a unique session ID and artifact root by default, writes richer session metadata and top-level status into `summary.json`/`summary.txt` plus `reports/summary.*`, and records path selection mode, blockers, and session-local env roots for agent aggregation. The classifier now emits `real-runtime-candidate` for launcher/plugin/runtime-sensitive paths, and the active skill doc uses the same terminology. Verification ran via `bun test scripts/subminer-change-verification.test.ts`, direct dry-run smoke checks for blocked `real-runtime` and failed unknown-lane execution, and a targeted classifier invocation for launcher/plugin paths.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,51 @@
---
id: TASK-167
title: Track shared SubMiner agent skills in git and clean up ignore rules
status: Done
assignee: []
created_date: '2026-03-13 05:46'
updated_date: '2026-03-13 05:47'
labels:
- git
- agents
- repo-hygiene
dependencies: []
references:
- /home/sudacode/projects/japanese/SubMiner/.gitignore
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-change-verification/SKILL.md
- >-
/home/sudacode/projects/japanese/SubMiner/.agents/skills/subminer-scrum-master/SKILL.md
documentation:
- /home/sudacode/projects/japanese/SubMiner/testing-plan.md
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Adjust the repository ignore rules so the shared SubMiner agent skill files can be committed while keeping unrelated local agent state ignored. Also ensure generated local verification artifacts like `.tmp/` do not pollute git status.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 Root ignore rules allow the shared SubMiner skill files under `.agents/skills/` to be tracked without broadly unignoring local agent state
- [x] #2 The changed shared skill files appear in git status as trackable files after the ignore update
- [x] #3 Local generated verification artifact directories remain ignored so git status stays clean
- [x] #4 The updated ignore rules are minimal and scoped to the repo-shared skill files
<!-- AC:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
Updated `.gitignore` to keep `.agents` ignored by default while narrowly unignoring the repo-shared SubMiner skill files and verifier scripts.
Added `.tmp/` to the root ignore rules so local verification artifacts stop polluting `git status`.
Verified the result with `git status --untracked-files=all` and `git check-ignore -v`, confirming the shared skill files are now trackable and `.tmp/` remains ignored.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Adjusted the root `.gitignore` so the shared SubMiner agent skill files can be committed cleanly without broadly unignoring local agent state. The repo now tracks the shared `subminer-change-verification` skill files and the `subminer-scrum-master` skill doc, while `.tmp/` is ignored so generated verification artifacts do not pollute git status. Verified with `git status --untracked-files=all` and `git check-ignore -v` that the intended skill files are commit-ready and `.tmp/` remains ignored.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,45 @@
---
id: TASK-168
title: Document immersion stats dashboard and config
status: Done
assignee:
- codex
created_date: '2026-03-12 22:53'
updated_date: '2026-03-12 22:53'
labels:
- docs
- immersion
dependencies: []
priority: medium
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Refresh user-facing docs for the new immersion stats dashboard so README, docs-site pages, changelog notes, and generated config examples describe how to access the dashboard and which `stats.*` settings control it.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 README mentions the new stats surface in product-facing feature/docs copy.
- [x] #2 Docs explain how to access the stats dashboard in-app and via localhost, and document the `stats` config block.
- [x] #3 Changelog/release-note input includes the new stats dashboard.
- [x] #4 Generated config examples include the new `stats` section.
<!-- AC:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Updated README and the docs-site immersion/config/mining/shortcut/homepage copy to describe the new stats dashboard, including the overlay toggle (`stats.toggleKey`, default `Backquote`) and the localhost browser UI (`http://127.0.0.1:5175` by default).
Added a changelog fragment for the stats dashboard release notes and extended the config template sections so regenerated `config.example.jsonc` artifacts now include the `stats` block.
Verified with `bun run test:config`, `bun run generate:config-example`, `bun run docs:test`, `bun run docs:build`, and `bun run changelog:lint`.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,79 @@
---
id: TASK-169
title: Add anime-level immersion metadata and link videos
status: Done
assignee:
- codex
created_date: '2026-03-13 19:34'
updated_date: '2026-03-13 21:46'
labels:
- immersion
- stats
- database
- anilist
dependencies: []
references:
- /home/sudacode/projects/japanese/SubMiner/docs/plans/2026-03-13-immersion-anime-metadata-design.md
- /home/sudacode/projects/japanese/SubMiner/docs/plans/2026-03-13-immersion-anime-metadata.md
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Add first-class anime metadata to the immersion tracker so stats can group sessions and videos by anime, season, and episode instead of relying only on per-video canonical titles. The new model should deduplicate anime-level metadata across rewatches and multiple files, use guessit-first filename parsing with built-in parser fallback, and create provisional anime rows even when AniList lookup fails.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 The immersion schema includes a new anime-level table plus additive video linkage/parsed metadata fields needed for anime, season, and episode stats.
- [x] #2 Media ingest creates or reuses anime rows, stores parsed season/episode metadata on videos, and upgrades provisional anime rows when AniList data becomes available.
- [x] #3 Query surfaces expose anime-level aggregation suitable for library/detail/episode stats without breaking current video/session queries.
- [x] #4 Focused regression coverage exists for schema/storage/query/service behavior, including provisional anime rows and guessit-first parser fallback behavior.
- [x] #5 Verification covers the SQLite immersion lane and any broader lanes required by the touched runtime/query files.
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Add red tests for the new schema shape in the SQLite immersion lane before changing storage code.
2. Implement `imm_anime` plus additive `imm_videos` metadata fields and focused storage helpers for provisional anime creation and AniList upgrade.
3. Add a guessit-first parser helper with built-in fallback and wire media ingest to persist anime/video metadata during `handleMediaChange(...)`.
4. Add anime-level query surfaces for library/detail/episode aggregation and expose them only where needed.
5. Run focused SQLite verification first, then broader verification lanes only if touched runtime/API files require them.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
2026-03-13: Design approved in-thread. Initial scope excluded migration/backfill work, but implementation was corrected in-thread to add a legacy DB migration/backfill path based on filename parsing.
2026-03-13: Detailed implementation plan written at `docs/plans/2026-03-13-immersion-anime-metadata.md`.
2026-03-13: Task 6 export/API work was intentionally skipped because no current stats API/UI consumer needs the anime query surface yet, and widening the contract would have touched unrelated dirty stats files.
2026-03-13: Verification commands run:
- `bun test src/core/services/immersion-tracker/storage-session.test.ts`
- `bun test src/core/services/immersion-tracker/metadata.test.ts`
- `bun test src/core/services/immersion-tracker-service.test.ts`
- `bun test src/core/services/immersion-tracker/__tests__/query.test.ts`
- `bun run test:immersion:sqlite:src`
- `bash .agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh src/core/services/immersion-tracker/storage.ts src/core/services/immersion-tracker/storage-session.test.ts src/core/services/immersion-tracker/metadata.ts src/core/services/immersion-tracker/metadata.test.ts src/core/services/immersion-tracker/query.ts src/core/services/immersion-tracker/types.ts src/core/services/immersion-tracker/__tests__/query.test.ts src/core/services/immersion-tracker-service.ts src/core/services/immersion-tracker-service.test.ts`
- `bash .agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh --lane core src/core/services/immersion-tracker/storage.ts src/core/services/immersion-tracker/storage-session.test.ts src/core/services/immersion-tracker/metadata.ts src/core/services/immersion-tracker/metadata.test.ts src/core/services/immersion-tracker/query.ts src/core/services/immersion-tracker/types.ts src/core/services/immersion-tracker/__tests__/query.test.ts src/core/services/immersion-tracker-service.ts src/core/services/immersion-tracker-service.test.ts`
2026-03-13: Verification results:
- `bun run test:immersion:sqlite:src`: passed
- verifier lane selection: `core`
- verifier result: passed (`bun run typecheck`, `bun run test:fast`)
- verifier artifacts: `.tmp/skill-verification/subminer-verify-20260313-214533-Ciw3L0/`
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Added `imm_anime`, additive `imm_videos` anime/parser metadata fields, and a legacy migration/backfill path that links existing videos to provisional anime rows from parsed filenames.
Added focused storage helpers for normalized anime identity reuse, later AniList upgrades, and per-video season/episode/parser metadata linking. Media ingest now parses and links anime metadata during `handleMediaChange(...)`.
Added anime-level query surfaces for library/detail/episode aggregation and regression coverage for schema, migration, storage, parser fallback, service ingest wiring, and anime stats queries.
Verified with the focused SQLite lane plus verifier-selected `core` coverage (`typecheck`, `test:fast`). No stats API/UI export was added yet because there is no current consumer for the new anime query surface.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,45 @@
---
id: TASK-170
title: 'Fix imm_words POS filtering and add stats cleanup maintenance command'
status: Done
assignee: []
created_date: '2026-03-13 00:00'
updated_date: '2026-03-14 18:31'
labels: []
dependencies: []
priority: high
ordinal: 9010
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
`imm_words` is currently populated from raw subtitle text instead of tokenized subtitle metadata, so ignored functional/noise tokens leak into stats and no POS metadata is stored. Fix live persistence to follow the existing token annotation exclusion rules and add an on-demand stats cleanup command to remove stale bad vocabulary rows from the stats DB.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 New `imm_words` inserts use tokenized subtitle data, persist POS metadata, and skip tokens excluded by existing POS-based vocabulary ignore rules.
- [x] #2 `subminer stats cleanup` supports `-v` / `--vocab`, defaults to vocab cleanup, and removes stale bad `imm_words` rows on demand.
- [x] #3 Regression coverage exists for persistence filtering, cleanup behavior, and stats cleanup CLI wiring.
<!-- AC:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Fixed `imm_words` persistence so the tracker now consumes tokenized subtitle data, stores POS metadata (`part_of_speech`, `pos1`, `pos2`, `pos3`), preserves distinct surface/lemma fields (`word` vs `headword`) when tokenization provides them, and skips vocabulary rows excluded by the existing POS/noise rules instead of mining raw subtitle fragments. Added `subminer stats cleanup` with default vocab cleanup plus `-v/--vocab`; the cleanup pass now repairs stale `headword`, `reading`, and `part_of_speech` values, attempts best-effort MeCab backfill for legacy rows, and removes rows that still have no usable POS metadata or fail the vocab filters.
Verification:
- `bun run typecheck`
- `bun test src/core/services/immersion-tracker-service.test.ts src/core/services/immersion-tracker/__tests__/query.test.ts src/core/services/immersion-tracker/storage-session.test.ts launcher/parse-args.test.ts launcher/commands/command-modules.test.ts src/main/runtime/stats-cli-command.test.ts src/main/runtime/mpv-main-event-main-deps.test.ts src/core/services/cli-command.test.ts`
- `bun run docs:test`
- `bun run docs:build`
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,77 @@
---
id: TASK-171
title: Add normalized immersion word and kanji occurrence tracking
status: Done
assignee:
- codex
created_date: '2026-03-14 11:30'
updated_date: '2026-03-14 11:48'
labels:
- immersion
- stats
- database
dependencies: []
references:
- /home/sudacode/projects/japanese/SubMiner/docs/plans/2026-03-14-immersion-occurrence-tracking-design.md
- /home/sudacode/projects/japanese/SubMiner/docs/plans/2026-03-14-immersion-occurrence-tracking.md
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Add normalized occurrence tables for immersion-tracked words and kanji so stats can map vocabulary back to the exact anime, episode, timestamp, and subtitle line where each item appeared. Preserve repeated tokens within the same line via counted occurrences instead of deduping, while avoiding duplicated token text storage.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 The immersion schema adds normalized subtitle-line and counted occurrence tables for words and kanji, with additive migration support for existing databases.
- [x] #2 Subtitle-line tracking writes one subtitle-line row per seen line plus counted word/kanji occurrences linked back to the line, session, video, and anime context.
- [x] #3 Query surfaces can map a word or kanji back to anime/episode/line/timestamp rows without breaking current top-level vocabulary and kanji stats.
- [x] #4 Focused regression coverage exists for schema, counted occurrence persistence, and reverse-mapping queries.
- [x] #5 Verification covers the SQLite immersion lane and any broader lanes required by touched service/API files.
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Add red tests for new line/occurrence schema and migration shape in the SQLite immersion lane.
2. Add red tests for service-level subtitle persistence that writes one line row plus counted word/kanji occurrences.
3. Implement additive schema, write-path plumbing, and counted occurrence upserts with minimal disruption to existing aggregate tables.
4. Add reverse-mapping query surfaces for word and kanji occurrences, plus focused API/service exposure only where needed.
5. Run focused SQLite verification first, then broader verification only if touched runtime/API files require it.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
2026-03-14: Design approved in-thread. Chosen shape: `imm_subtitle_lines` plus counted bridge tables `imm_word_line_occurrences` and `imm_kanji_line_occurrences`, retaining repeated tokens within a line via `occurrence_count`.
2026-03-14: Implemented additive schema version bump to 7. `recordSubtitleLine(...)` now queues one normalized subtitle-line write that owns aggregate word/kanji upserts plus counted bridge-row inserts.
2026-03-14: Added reverse-mapping query surfaces for exact word triples and single kanji lookups. No stats API/UI consumer was widened in this change.
2026-03-14: Verification commands run:
- `bun test src/core/services/immersion-tracker-service.test.ts`
- `bun test src/core/services/immersion-tracker/storage-session.test.ts`
- `bun test src/core/services/immersion-tracker/__tests__/query.test.ts`
- `bun run typecheck`
- `bash .agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh src/core/services/immersion-tracker/types.ts src/core/services/immersion-tracker/storage.ts src/core/services/immersion-tracker/query.ts src/core/services/immersion-tracker-service.ts src/core/services/immersion-tracker/storage-session.test.ts src/core/services/immersion-tracker-service.test.ts src/core/services/immersion-tracker/__tests__/query.test.ts`
- `bash .agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh --lane core src/core/services/immersion-tracker/types.ts src/core/services/immersion-tracker/storage.ts src/core/services/immersion-tracker/query.ts src/core/services/immersion-tracker-service.ts src/core/services/immersion-tracker/storage-session.test.ts src/core/services/immersion-tracker-service.test.ts src/core/services/immersion-tracker/__tests__/query.test.ts`
- `bun run test:immersion:sqlite:src`
2026-03-14: Verification results:
- targeted tracker/query tests: passed
- verifier lane selection: `core`
- verifier result: passed (`typecheck`, `test:fast`)
- verifier artifacts: `.tmp/skill-verification/subminer-verify-20260314-114630-abO7mb/`
- maintained immersion SQLite lane: passed
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Added normalized subtitle-line occurrence tracking to immersion stats with three additive tables: `imm_subtitle_lines`, `imm_word_line_occurrences`, and `imm_kanji_line_occurrences`.
`recordSubtitleLine(...)` now preserves repeated allowed tokens and repeated kanji within the same subtitle line via `occurrence_count`, while still updating canonical `imm_words` and `imm_kanji` aggregates.
Added reverse-mapping queries for exact word triples and kanji so callers can fetch anime/video/session/line/timestamp context for each occurrence without duplicating token text storage.
Verified with targeted tracker/query tests, `bun run typecheck`, verifier-selected `core` coverage, and the maintained `bun run test:immersion:sqlite:src` lane.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,65 @@
---
id: TASK-172
title: Wire immersion occurrence drilldown into stats API and vocabulary drawer
status: Done
assignee:
- codex
created_date: '2026-03-14 12:05'
updated_date: '2026-03-14 12:11'
labels:
- immersion
- stats
- ui
dependencies:
- TASK-171
references: []
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Expose the new immersion word/kanji occurrence queries through the stats server and add a right-side Vocabulary drawer that shows recent occurrence rows with paging when a word or kanji is clicked.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 Stats server exposes word and kanji occurrence endpoints with bounded recent-first paging.
- [x] #2 Stats client/types support loading occurrence pages for a selected word or kanji.
- [x] #3 Vocabulary tab opens a right drawer for the selected word/kanji, shows recent occurrences, and supports loading more.
- [x] #4 Focused regression coverage exists for the server endpoint contract, and the stats UI still typechecks/builds.
- [x] #5 Verification covers the cheapest sufficient backend and stats-UI lanes.
<!-- AC:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
2026-03-14: Design approved in-thread. Chosen UX: click a word chip or kanji glyph to open a right-side drawer with recent-first occurrences, initial cap 50, plus “Load more”.
2026-03-14: Implemented `/api/stats/vocabulary/occurrences` and `/api/stats/kanji/occurrences` with `limit` + `offset` paging. The drawer uses direct stats HTTP client calls and keeps existing aggregate vocabulary data flow intact.
2026-03-14: Verification commands run:
- `bun test src/core/services/__tests__/stats-server.test.ts`
- `bun run typecheck`
- `cd stats && bun run build`
- `bun run docs:test`
- `bun run docs:build`
- `cd stats && bunx tsc --noEmit`
- `bash .agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh --lane core src/core/services/stats-server.ts src/core/services/__tests__/stats-server.test.ts`
2026-03-14: Verification results:
- stats server endpoint tests: passed
- root typecheck: passed
- stats UI production build: passed
- docs-site test/build: passed
- `cd stats && bunx tsc --noEmit`: passed after removing stale `hasCoverArt` prop usage in the library stats UI
- verifier result: passed (`typecheck`, `test:fast`)
- verifier artifacts: `.tmp/skill-verification/subminer-verify-20260314-120900-J0VvB0/`
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Wired occurrence drilldown into the stats server and Vocabulary tab. Words and kanji now open a right-side drawer that loads recent occurrence rows 50 at a time and supports “Load more”.
Added bounded recent-first occurrence endpoints to the stats HTTP API, extended the stats client/type surface, and made word chips plus kanji glyphs selectable with active-state styling.
Updated the immersion-tracking docs to mention vocabulary occurrence drilldown. Verified with focused stats-server tests, root typecheck, stats UI production build, docs-site test/build, the repo verifier core lane, and a direct `stats` package typecheck after removing the stale `MediaHeader` prop mismatch.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,40 @@
---
id: TASK-173
title: Remove Avg Frequency metric from Vocabulary tab summary cards
status: Done
assignee: []
created_date: '2026-03-15 00:13'
updated_date: '2026-03-15 00:15'
labels:
- stats
- ui
dependencies: []
priority: low
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
User requested removing the Avg Frequency card/metric because it is not useful. Remove the UI card and stop computing/storing the summary field in dashboard summary shaping code.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 Vocabulary tab no longer renders an "Avg Frequency" stat card.
- [x] #2 Vocabulary summary model no longer exposes or computes averageFrequency.
- [x] #3 Typecheck/tests covering dashboard summary and vocabulary tab pass.
<!-- AC:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Removed the Vocabulary tab "Avg Frequency" card and deleted the corresponding `averageFrequency` field from `VocabularySummary` and `buildVocabularySummary`.
Verification run:
- `bun test stats/src/lib/dashboard-data.test.ts`
- `bun run typecheck`
- `bun run test:fast`
- `bun run build`
- `bun run test:env`
- `bun run test:smoke:dist`
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -0,0 +1,50 @@
---
id: TASK-176
title: Fix failing CI on PR 24
status: Done
assignee:
- '@Codex'
created_date: '2026-03-15 22:32'
updated_date: '2026-03-15 22:36'
labels:
- ci
- github-actions
- pr-24
dependencies: []
references:
- 'PR #24'
---
## Description
<!-- SECTION:DESCRIPTION:BEGIN -->
Inspect the failing GitHub Actions check on PR 24, reproduce the actionable failure locally when possible, implement the minimum fix, and push until the required PR checks are green.
<!-- SECTION:DESCRIPTION:END -->
## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 The failing GitHub Actions job on PR 24 is inspected and the concrete failure cause is identified.
- [x] #2 A minimal code or workflow fix is implemented for the actionable failure.
- [x] #3 Relevant local verification is run and recorded.
- [x] #4 The fix is pushed and CI is rechecked until required GitHub Actions checks for the PR are green or a specific external blocker is identified.
<!-- AC:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
Investigated failing GitHub Actions run `23120841305` for PR 24. Concrete failure was `bun run typecheck` in step `Build (TypeScript check)` with `src/core/services/subtitle-cue-parser.ts(154,15): error TS18048: 'normalizedSource' is possibly 'undefined'.`
Fixed by making the URL/path normalization split result total in `detectSubtitleFormat()`, committed as `e940205` (`fix: satisfy subtitle cue parser typecheck`), and pushed to `origin/feature/renderer-performance`.
Verification: `bun test src/core/services/subtitle-cue-parser.test.ts` passed; a narrow `bunx tsc --noEmit --strict --target es2022 --module esnext --moduleResolution bundler src/core/services/subtitle-cue-parser.ts` compile also passed.
Post-push PR state is `mergeStateStatus=CLEAN` / `mergeable=MERGEABLE`; GitHub no longer reports the failed `build-test-audit` check. No new GitHub Actions CI run attached to the new head SHA within the follow-up polling window, but the PR is green from GitHub's current mergeability/check view.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Cleared the failing PR CI by fixing the strict-nullability issue in `detectSubtitleFormat()` that broke the TypeScript check on run `23120841305`.
Pushed `e940205` to the PR branch and confirmed the PR now reports `mergeStateStatus=CLEAN` with no failing checks.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -5,9 +5,11 @@
"": {
"name": "subminer",
"dependencies": {
"@hono/node-server": "^1.19.11",
"axios": "^1.13.5",
"commander": "^14.0.3",
"discord-rpc": "^4.0.1",
"hono": "^4.12.7",
"jsonc-parser": "^3.3.1",
"libsql": "^0.5.22",
"ws": "^8.19.0",
@@ -96,6 +98,8 @@
"@esbuild/win32-x64": ["@esbuild/win32-x64@0.25.12", "", { "os": "win32", "cpu": "x64" }, "sha512-alJC0uCZpTFrSL0CCDjcgleBXPnCrEAhTBILpeAp7M/OFgoqtAetfBzX0xM00MUsVVPpVjlPuMbREqnZCXaTnA=="],
"@hono/node-server": ["@hono/node-server@1.19.11", "", { "peerDependencies": { "hono": "^4" } }, "sha512-dr8/3zEaB+p0D2n/IUrlPF1HZm586qgJNXK1a9fhg/PzdtkK7Ksd5l312tJX2yBuALqDYBlG20QEbayqPyxn+g=="],
"@isaacs/cliui": ["@isaacs/cliui@9.0.0", "", {}, "sha512-AokJm4tuBHillT+FpMtxQ60n8ObyXBatq7jD2/JA9dxbDDokKQm8KMht5ibGzLVU9IJDIKK4TPKgMHEYMn3lMg=="],
"@isaacs/fs-minipass": ["@isaacs/fs-minipass@4.0.1", "", { "dependencies": { "minipass": "7.1.2" } }, "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w=="],
@@ -396,6 +400,8 @@
"hasown": ["hasown@2.0.2", "", { "dependencies": { "function-bind": "1.1.2" } }, "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ=="],
"hono": ["hono@4.12.7", "", {}, "sha512-jq9l1DM0zVIvsm3lv9Nw9nlJnMNPOcAtsbsgiUhWcFzPE99Gvo6yRTlszSLLYacMeQ6quHD6hMfId8crVHvexw=="],
"hosted-git-info": ["hosted-git-info@4.1.0", "", { "dependencies": { "lru-cache": "6.0.0" } }, "sha512-kyCuEOWjJqZuDbRHzL8V93NzQhwIB71oFWSyzVo+KPZI+pnQPPxucdkrOZvkLRnrf5URsQM+IJ09Dw29cRALIA=="],
"http-cache-semantics": ["http-cache-semantics@4.2.0", "", {}, "sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ=="],

6
changes/stats-command.md Normal file
View File

@@ -0,0 +1,6 @@
type: added
area: launcher
- Added `subminer stats` to launch the local stats dashboard, force-start the stats server on demand, and open the dashboard in your browser.
- Added `subminer stats cleanup` to backfill vocabulary metadata and prune stale or excluded immersion rows on demand.
- Added `stats.autoOpenBrowser` so browser launch after `subminer stats` can be enabled or disabled explicitly.

View File

@@ -0,0 +1,7 @@
type: added
area: immersion
- Added a local stats dashboard for immersion tracking with Overview, Anime, Trends, Vocabulary, and Sessions views.
- Added anime progress, episode completion, Anki card links, and occurrence drill-down across the stats dashboard.
- Added richer session timelines with new-word activity, cumulative totals, and pause/seek/card event markers.
- Added completed-episodes and completed-anime totals to the Overview tracking snapshot.

View File

@@ -461,5 +461,17 @@
"monthlyRollupsDays": 1825, // Monthly rollup retention window in days.
"vacuumIntervalDays": 7 // Minimum days between VACUUM runs.
} // Retention setting.
} // Enable/disable immersion tracking.
}, // Enable/disable immersion tracking.
// ==========================================
// Stats Dashboard
// Local immersion stats dashboard served on localhost and available as an in-app overlay.
// Uses the immersion tracking database for overview, trends, sessions, and vocabulary views.
// ==========================================
"stats": {
"toggleKey": "Backquote", // Key code to toggle the stats overlay.
"serverPort": 5175, // Port for the stats HTTP server.
"autoStartServer": true, // Automatically start the stats server on launch. Values: true | false
"autoOpenBrowser": true // Automatically open the stats dashboard in a browser when the server starts. Values: true | false
} // Local immersion stats dashboard served on localhost and available as an in-app overlay.
}

View File

@@ -2,6 +2,8 @@
In-repo VitePress documentation source for SubMiner.
Internal architecture/workflow source of truth lives in `docs/README.md` at the repo root. Keep `docs-site/` user-facing.
## Local development
```bash

View File

@@ -1,5 +1,7 @@
# Architecture
This page is a contributor-facing architecture summary. Canonical internal architecture guidance lives in `docs/architecture/README.md` at the repo root.
SubMiner is split into three cooperating runtimes:
- Electron desktop app (`src/`) for overlay/UI/runtime orchestration.

View File

@@ -2,7 +2,7 @@
SubMiner can build a Yomitan-compatible character dictionary from AniList metadata so that character names in subtitles are recognized, highlighted, and enrichable with context — portraits, roles, voice actors, and biographical detail — without leaving the overlay.
The dictionary is generated per-media, merged across your recently-watched titles, and auto-imported into Yomitan. When a character name appears in a subtitle line, it gets highlighted and becomes clickable for a full profile lookup.
The dictionary is generated per-media, merged across your recently-watched titles, and auto-imported into Yomitan. When a character name appears in a subtitle line, it gets highlighted and becomes available for hover-driven Yomitan profile lookup.
## How It Works

View File

@@ -117,6 +117,7 @@ The configuration file includes several main sections:
- [**Jellyfin**](#jellyfin) - Optional Jellyfin auth, library listing, and playback launch
- [**Discord Rich Presence**](#discord-rich-presence) - Optional Discord activity card updates
- [**Immersion Tracking**](#immersion-tracking) - Track subtitle sessions and mining activity in SQLite
- [**Stats Dashboard**](#stats-dashboard) - Local dashboard and overlay for immersion progress
- [**YouTube Subtitle Generation**](#youtube-subtitle-generation) - Launcher defaults for yt-dlp + local whisper fallback
## Core Settings
@@ -1144,7 +1145,7 @@ Troubleshooting:
### Immersion Tracking
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions:
Enable or disable local immersion analytics stored in SQLite for mined subtitles and media sessions. This data also powers the stats dashboard:
```json
{
@@ -1190,7 +1191,36 @@ When `dbPath` is blank or omitted, SubMiner writes telemetry and session summari
Set `dbPath` only if you want to relocate the database (for backup, syncing, or inspection workflows). The database is created when tracking starts for the first time.
See [Immersion Tracking Storage](/immersion-tracking) for schema details, query templates, retention/rollup behavior, backend portability notes, and the dedicated SQLite verification command.
See [Immersion Tracking Storage](/immersion-tracking) for schema details, query templates, dashboard access, retention/rollup behavior, backend portability notes, and the dedicated SQLite verification command.
### Stats Dashboard
Configure the local stats UI served from SubMiner and the in-app stats overlay toggle:
```json
{
"stats": {
"toggleKey": "Backquote",
"serverPort": 5175,
"autoStartServer": true,
"autoOpenBrowser": true
}
}
```
| Option | Values | Description |
| ----------------- | ----------------- | --------------------------------------------------------------------------- |
| `toggleKey` | Electron key code | Overlay-local key code used to toggle the stats overlay. Default `Backquote`. |
| `serverPort` | integer | Localhost port for the browser stats UI. Default `5175`. |
| `autoStartServer` | `true`, `false` | Start the local stats HTTP server automatically once immersion tracking is active. Default `true`. |
| `autoOpenBrowser` | `true`, `false` | When `subminer stats` starts the server on demand, also open the dashboard in your default browser. Default `true`. |
Usage notes:
- The browser UI is served at `http://127.0.0.1:<serverPort>`.
- The overlay toggle is local to the focused visible overlay window; it is not registered as a global OS shortcut.
- The dashboard reads from the same immersion-tracking database, so keep `immersionTracking.enabled` on if you want data to appear.
- The UI includes Overview, Anime, Trends, Vocabulary, and Sessions tabs.
### YouTube Subtitle Generation

View File

@@ -1,5 +1,7 @@
# Building & Testing
For internal architecture/workflow guidance, use `docs/README.md` at the repo root. This page stays focused on contributor-facing build and test commands.
## Prerequisites
- [Bun](https://bun.sh)
@@ -13,6 +15,7 @@ cd SubMiner
git submodule update --init --recursive
bun install
(cd stats && bun install --frozen-lockfile)
(cd vendor/texthooker-ui && bun install --frozen-lockfile)
```
@@ -200,7 +203,7 @@ Run `make help` for a full list of targets. Key ones:
| `make build-launcher` | Generate Bun launcher wrapper at `dist/launcher/subminer` |
| `make install` | Install platform artifacts (wrapper, theme, AppImage/app bundle) |
| `make install-plugin` | Install mpv Lua plugin and config |
| `make deps` | Install JS dependencies (root + texthooker-ui) |
| `make deps` | Install JS dependencies (root + stats + texthooker-ui) |
| `make pretty` | Run scoped Prettier formatting for maintained source/config files |
| `make generate-config` | Generate default config from centralized registry |
| `make build-linux` | Convenience wrapper for Linux packaging |
@@ -214,7 +217,7 @@ Run `make help` for a full list of targets. Key ones:
- To add/change generated config template blocks/comments, update `src/config/definitions/template-sections.ts`.
- Keep `src/config/definitions.ts` as the composed public API (`DEFAULT_CONFIG`, registries, template export) that wires domain modules together.
- Overlay window/visibility state is owned by `src/core/services/overlay-manager.ts`.
- Runtime architecture/module-boundary conventions are documented in [Architecture](/architecture); keep contributor changes aligned with that canonical guide.
- Runtime architecture/module-boundary conventions are summarized in [Architecture](/architecture), with canonical internal guidance in `docs/architecture/README.md` at the repo root.
- Linux packaged desktop launches pass `--background` using electron-builder `build.linux.executableArgs` in `package.json`.
- Prefer direct inline deps objects in `src/main/` modules for simple pass-through wiring.
- Add a helper/adapter service only when it performs meaningful adaptation, validation, or reuse (not identity mapping).

View File

@@ -1,8 +1,8 @@
# Immersion Tracking
SubMiner can log your watching and mining activity to a local SQLite database. This is optional and disabled by default.
SubMiner can log your watching and mining activity to a local SQLite database, then surface it in the built-in stats dashboard. Tracking is enabled by default and can be turned off if you do not want local analytics.
When enabled, SubMiner records per-session statistics (watch time, subtitle lines seen, words encountered, cards mined) and maintains daily and monthly rollups. You can query the database directly with any SQLite tool to track your progress over time.
When enabled, SubMiner records per-session statistics (watch time, subtitle lines seen, words encountered, cards mined) and maintains daily and monthly rollups. You can view that data in SubMiner's stats UI or query the database directly with any SQLite tool.
## Enabling
@@ -18,6 +18,44 @@ When enabled, SubMiner records per-session statistics (watch time, subtitle line
- Leave `dbPath` empty to use the default location (`immersion.sqlite` in SubMiner's app-data directory).
- Set an explicit path to move the database (useful for backups, cloud syncing, or external tools).
## Stats Dashboard
The same immersion data powers the stats dashboard.
- In-app overlay: focus the visible overlay, then press the key from `stats.toggleKey` (default: `` ` `` / `Backquote`).
- Launcher command: run `subminer stats` to start the local stats server on demand and open the dashboard in your browser.
- Maintenance command: run `subminer stats cleanup` or `subminer stats cleanup -v` to backfill/repair vocabulary metadata (`headword`, `reading`, POS) and purge stale or excluded rows from `imm_words` on demand.
- Browser page: open `http://127.0.0.1:5175` directly if the local stats server is already running.
Dashboard tabs:
- Overview: recent sessions, streak calendar, watch-time history, and a tracking snapshot with completed episodes/anime totals
- Anime: cover-art library, per-series progress, episode drill-down, and direct links into mined cards
- Trends: watch time, sessions, words seen, and per-anime progress/pattern charts
- Sessions: expandable session history with new-word activity, cumulative totals, and pause/seek/card markers
- Vocabulary: top repeated words, new-word timeline, kanji breakdown, and click-through occurrence drilldown in a right-side drawer
Stats server config lives under `stats`:
```jsonc
{
"stats": {
"toggleKey": "Backquote",
"serverPort": 5175,
"autoStartServer": true,
"autoOpenBrowser": true
}
}
```
- `toggleKey` is overlay-local, not a system-wide shortcut.
- `serverPort` controls the localhost dashboard URL.
- `autoStartServer` starts the local stats HTTP server on launch once immersion tracking is active.
- `autoOpenBrowser` controls whether `subminer stats` launches the dashboard URL in your browser after ensuring the server is running.
- `subminer stats` forces the dashboard server to start even when `autoStartServer` is `false`.
- `subminer stats` fails with an error when `immersionTracking.enabled` is `false`.
- `subminer stats cleanup` defaults to vocabulary cleanup, repairs stale `headword`, `reading`, and `part_of_speech` values, attempts best-effort MeCab backfill for legacy rows, and removes rows that still fail vocab filtering.
## Retention Defaults
Data is kept for the following durations before automatic cleanup:

View File

@@ -73,9 +73,9 @@ features:
src: /assets/tokenization.svg
alt: Tracking chart icon
title: Immersion Tracking
details: Logs watch time, words encountered, and cards mined to SQLite with daily and monthly rollups for long-term progress tracking.
details: Logs watch time, words encountered, and cards mined to SQLite, then surfaces the same data in a local stats dashboard with rollups and session drill-down.
link: /immersion-tracking
linkText: Tracking details
linkText: Stats details
- icon:
src: /assets/cross-platform.svg
alt: Cross-platform icon
@@ -102,7 +102,7 @@ const demoAssetVersion = '20260223-2';
<div class="workflow-step" style="animation-delay: 60ms">
<div class="step-number">02</div>
<div class="step-title">Lookup</div>
<div class="step-desc">Hover or click a token in the interactive overlay to open Yomitan context.</div>
<div class="step-desc">Hover a token in the interactive overlay, then trigger Yomitan lookup to open context.</div>
</div>
<div class="workflow-connector" aria-hidden="true"></div>
<div class="workflow-step" style="animation-delay: 120ms">

View File

@@ -4,10 +4,10 @@ This guide walks through the sentence mining loop — from watching a video to c
## Overview
SubMiner runs as a transparent overlay on top of mpv. As subtitles play, the overlay displays them as interactive text. You click a word to look it up with Yomitan, then create an Anki card with a single action. SubMiner automatically attaches the sentence, audio clip, and screenshot.
SubMiner runs as a transparent overlay on top of mpv. As subtitles play, the overlay displays them as interactive text. You hover a word, trigger Yomitan lookup with your configured lookup key/modifier, then create an Anki card with a single action. SubMiner automatically attaches the sentence, audio clip, and screenshot.
```text
Watch video → See subtitle → Click word → Yomitan lookup → Add to Anki
Watch video → See subtitle → Hover word + trigger lookup → Yomitan popup → Add to Anki
SubMiner auto-fills:
sentence, audio, image, translation
@@ -30,9 +30,9 @@ SubMiner uses one overlay window with modal surfaces.
### Primary Subtitle Layer
The visible overlay renders subtitles as tokenized, clickable word spans. Each word is a separate element with reading and headword data attached. This plane is styled independently from mpv subtitles and supports:
The visible overlay renders subtitles as tokenized hoverable word spans. Each word is a separate element with reading and headword data attached. This plane is styled independently from mpv subtitles and supports:
- Word-level click targets for Yomitan lookup
- Word-level hover targets for Yomitan lookup
- Auto pause/resume on subtitle hover (enabled by default via `subtitleStyle.autoPauseVideoOnHover`)
- Optional pause while the Yomitan popup is open (`subtitleStyle.autoPauseVideoOnYomitanPopup`)
- Right-click to pause/resume
@@ -55,9 +55,10 @@ Jimaku search, field-grouping, runtime options, and manual subsync open as modal
## Looking Up Words
1. Hover over the subtitle area — the overlay activates pointer events.
2. Click any word. SubMiner uses Unicode-aware boundary detection (`Intl.Segmenter`) to select it. On macOS, hovering is enough.
3. Yomitan detects the selection and opens its lookup popup.
4. From the popup, add the word to Anki.
2. Hover the word you want. SubMiner keeps per-token boundaries so Yomitan can target that token cleanly.
3. Trigger Yomitan lookup with your configured lookup key/modifier (for example `Shift` if that is how your Yomitan profile is set up).
4. Yomitan opens its lookup popup for the hovered token.
5. From the popup, add the word to Anki.
### Controller Workflow
@@ -83,7 +84,7 @@ There are three ways to create cards, depending on your workflow.
This is the most common flow. Yomitan creates a card in Anki, and SubMiner enriches it automatically.
1. Click a word → Yomitan popup appears.
1. Hover a word, then trigger Yomitan lookup → Yomitan popup appears.
2. Click the Anki icon in Yomitan to add the word.
3. SubMiner receives or detects the new card:
- **Proxy mode** (`ankiConnect.proxy.enabled: true`): immediate enrich after successful `addNote` / `addNotes`.
@@ -194,7 +195,7 @@ See [Subtitle Annotations — N+1](/subtitle-annotations#n1-word-highlighting) f
## Immersion Tracking
SubMiner can log your watching and mining activity to a local SQLite database — session times, words seen, cards mined, and daily/monthly rollups.
SubMiner can log your watching and mining activity to a local SQLite database and expose it in the built-in stats dashboard — session times, words seen, cards mined, and daily/monthly rollups.
Enable it in your config:
@@ -205,6 +206,8 @@ Enable it in your config:
}
```
See [Immersion Tracking](/immersion-tracking) for the full schema and retention settings.
Open the dashboard in the overlay with `stats.toggleKey` (default: `` ` ``), launch it in a browser with `subminer stats`, or visit `http://127.0.0.1:5175` directly if the local stats server is already running. The dashboard covers overview totals, anime progress, session detail, and vocabulary drill-down from the same local immersion database.
See [Immersion Tracking](/immersion-tracking) for dashboard details, schema, and retention settings.
Next: [Anki Integration](/anki-integration) — field mapping, media generation, and card enrichment configuration.

View File

@@ -461,5 +461,17 @@
"monthlyRollupsDays": 1825, // Monthly rollup retention window in days.
"vacuumIntervalDays": 7 // Minimum days between VACUUM runs.
} // Retention setting.
} // Enable/disable immersion tracking.
}, // Enable/disable immersion tracking.
// ==========================================
// Stats Dashboard
// Local immersion stats dashboard served on localhost and available as an in-app overlay.
// Uses the immersion tracking database for overview, trends, sessions, and vocabulary views.
// ==========================================
"stats": {
"toggleKey": "Backquote", // Key code to toggle the stats overlay.
"serverPort": 5175, // Port for the stats HTTP server.
"autoStartServer": true, // Automatically start the stats server on launch. Values: true | false
"autoOpenBrowser": true // Automatically open the stats dashboard in a browser when the server starts. Values: true | false
} // Local immersion stats dashboard served on localhost and available as an in-app overlay.
}

View File

@@ -68,6 +68,9 @@ Mouse-hover playback behavior is configured separately from shortcuts: `subtitle
| `Ctrl/Cmd+Shift+O` | Open runtime options palette | `shortcuts.openRuntimeOptions` |
| `Ctrl+Shift+J` | Open Jimaku subtitle search modal | `shortcuts.openJimaku` |
| `Ctrl+Alt+S` | Open subtitle sync (subsync) modal | `shortcuts.triggerSubsync` |
| `` ` `` | Toggle stats overlay | `stats.toggleKey` |
The stats toggle is handled inside the focused visible overlay window. It is configurable through the top-level `stats.toggleKey` setting and defaults to `Backquote`.
## Controller Shortcuts

View File

@@ -34,7 +34,7 @@ Set `refreshMinutes` to `1440` (24 hours) for daily sync if your Anki collection
## Character-Name Highlighting
Character-name matches are built from the active merged SubMiner character dictionary, which auto-syncs character data from AniList for your recently-watched titles. Matching names are highlighted in subtitles and become clickable for full character profiles — portraits, roles, voice actors, and biographical detail.
Character-name matches are built from the active merged SubMiner character dictionary, which auto-syncs character data from AniList for your recently-watched titles. Matching names are highlighted in subtitles and become available for hover-driven Yomitan character profiles — portraits, roles, voice actors, and biographical detail.
**How it works:**

View File

@@ -178,12 +178,12 @@ SubMiner does not load the source tree directly from `vendor/subminer-yomitan`;
If you installed from the AppImage and see this error, the package may be incomplete. Re-download the AppImage or place the unpacked Yomitan extension manually in `~/.config/SubMiner/yomitan`.
**Yomitan popup does not appear when clicking words**
**Yomitan popup does not appear when hovering words and triggering lookup**
- Verify Yomitan loaded successfully — check the terminal output for "Loaded Yomitan extension".
- Yomitan requires dictionaries to be installed. Open Yomitan settings (`Alt+Shift+Y` or `SubMiner.AppImage --settings`) and confirm at least one dictionary is imported.
- If `yomitan.externalProfilePath` is set, import/check dictionaries in the external app/profile instead. SubMiner treats that profile as read-only and does not open its own Yomitan settings window.
- If the overlay shows subtitles but words are not clickable, the tokenizer may have failed. See the MeCab section below.
- If the overlay shows subtitles but hover lookup never resolves on tokens, the tokenizer may have failed. See the MeCab section below.
## MeCab / Tokenization

View File

@@ -11,7 +11,7 @@
3. The overlay connects and subscribes to subtitle changes
4. Subtitles are tokenized with Yomitan's internal parser
5. Words are displayed as interactive spans in the overlay
6. Hovering or clicking a word triggers Yomitan popup for dictionary lookup
6. Hover a word, then trigger Yomitan lookup with your configured lookup key/modifier to open the Yomitan popup
7. Optional [subtitle annotations](/subtitle-annotations) (N+1, character-name, frequency, JLPT) highlight useful cues in real time
There are two ways to use SubMiner:

33
docs/README.md Normal file
View File

@@ -0,0 +1,33 @@
<!-- read_when: starting substantial work in this repo or looking for the internal source of truth -->
# SubMiner Internal Docs
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: you need internal architecture, workflow, verification, or release guidance
`docs/` is the internal system of record for agent and contributor knowledge. Start here, then drill into the smallest doc that fits the task.
## Start Here
- [Architecture](./architecture/README.md) - runtime map, domains, layering rules
- [Workflow](./workflow/README.md) - planning, execution, verification expectations
- [Knowledge Base](./knowledge-base/README.md) - how docs are structured, maintained, and audited
- [Release Guide](./RELEASING.md) - tagged release checklist
- [Plans](./plans/) - active design and implementation artifacts
## Fast Paths
- New feature or refactor: [Workflow](./workflow/README.md), then [Architecture](./architecture/README.md)
- Test/build/release work: [Verification](./workflow/verification.md), then [Release Guide](./RELEASING.md)
- “What owns this behavior?”: [Domains](./architecture/domains.md)
- “Can these modules depend on each other?”: [Layering](./architecture/layering.md)
- “What doc should exist for this?”: [Catalog](./knowledge-base/catalog.md)
## Rules
- Treat `docs/` as canonical for internal guidance.
- Treat `docs-site/` as user-facing/public docs.
- Keep `AGENTS.md` short; deep detail belongs here.
- Update docs when behavior, architecture, or workflow meaningfully changes.

View File

@@ -3,22 +3,30 @@
# Releasing
1. Confirm `main` is green: `gh run list --workflow CI --limit 5`.
2. Bump `package.json` to the release version.
3. Build release metadata before tagging:
2. Confirm release-facing docs are current: `README.md`, `changes/*.md`, and any touched `docs-site/` pages/config examples.
3. Run `bun run changelog:lint`.
4. Bump `package.json` to the release version.
5. Build release metadata before tagging:
`bun run changelog:build --version <version>`
4. Review `CHANGELOG.md`.
5. Run release gate locally:
6. Review `CHANGELOG.md` and `release/release-notes.md`.
7. Run release gate locally:
`bun run changelog:check --version <version>`
`bun run verify:config-example`
`bun run test:fast`
`bun run typecheck`
6. Commit release prep.
7. Tag the commit: `git tag v<version>`.
8. Push commit + tag.
`bun run test:fast`
`bun run test:env`
`bun run build`
8. If `docs-site/` changed, also run:
`bun run docs:test`
`bun run docs:build`
9. Commit release prep.
10. Tag the commit: `git tag v<version>`.
11. Push commit + tag.
Notes:
- `changelog:check` now rejects tag/package version mismatches.
- `changelog:build` generates `CHANGELOG.md` + `release/release-notes.md` and removes the released `changes/*.md` fragments.
- Do not tag while `changes/*.md` fragments still exist.
- Tagged release workflow now also attempts to update `subminer-bin` on the AUR after GitHub Release publication.
- Required GitHub Actions secret: `AUR_SSH_PRIVATE_KEY`. Add the matching public key to your AUR account before relying on the automation.

View File

@@ -0,0 +1,283 @@
# Renderer Performance Optimizations
**Date:** 2026-03-15
**Status:** Draft
## Goal
Minimize the time between a subtitle line appearing and annotations being displayed. Three optimizations target different pipeline stages to achieve this.
## Current Pipeline (Warm State)
```text
MPV subtitle change (0ms)
-> IPC to main (5ms)
-> Cache check (2ms)
-> [CACHE MISS] Yomitan parser (35-180ms)
-> Parallel: MeCab enrichment (20-80ms) + Frequency lookup (15-50ms)
-> Annotation stage: 4 sequential passes (25-70ms)
-> IPC to renderer (10ms)
-> DOM render: createElement per token (15-50ms)
─────────────────────────────────
Total: ~200-320ms (cache miss)
Total: ~72ms (cache hit)
```
## Target Pipeline
```text
MPV subtitle change (0ms)
-> IPC to main (5ms)
-> Cache check (2ms)
-> [CACHE HIT via prefetch] (0ms)
-> IPC to renderer (10ms)
-> DOM render: cloneNode from template (10-30ms)
─────────────────────────────────
Total: ~30-50ms (prefetch-warmed, normal playback)
[CACHE MISS, e.g. immediate seek]
-> Yomitan parser (35-180ms)
-> Parallel: MeCab enrichment + Frequency lookup
-> Annotation stage: 1 batched pass (10-25ms)
-> IPC to renderer (10ms)
-> DOM render: cloneNode from template (10-30ms)
─────────────────────────────────
Total: ~150-260ms (cache miss, still improved)
```
---
## Optimization 1: Subtitle Prefetching
### Summary
A new `SubtitlePrefetchService` parses external subtitle files and tokenizes upcoming lines in the background before they appear on screen. This converts most cache misses into cache hits during normal playback.
### Scope
External subtitle files only (SRT, VTT, ASS). Embedded subtitle tracks are out of scope since Japanese subtitles are virtually always external files.
### Architecture
#### Subtitle File Parsing
A new cue parser that extracts both timing and text content from subtitle files. The existing `parseSrtOrVttStartTimes` in `subtitle-delay-shift.ts` only extracts timing; this needs a companion that also extracts the dialogue text.
**Parsed cue structure:**
```typescript
interface SubtitleCue {
startTime: number; // seconds
endTime: number; // seconds
text: string; // raw subtitle text
}
```
**Supported formats:**
- SRT/VTT: Regex-based parsing of timing lines + text content between timing blocks.
- ASS: Parse `[Events]` section, extract `Dialogue:` lines, split on the first 9 commas only (ASS v4+ has 10 fields; the last field is Text which can itself contain commas). Strip ASS override tags (`{\...}`) from the text before storing.
ASS text fields contain inline override tags like `{\b1}`, `{\an8}`, `{\fad(200,300)}`. The cue parser strips these during extraction so the tokenizer receives clean text.
#### Prefetch Service Lifecycle
1. **Activation trigger:** When a subtitle track is activated (or changes), check if it's external via MPV's `track-list` property. If `external === true`, read the file via `external-filename` using the existing `loadSubtitleSourceText` infrastructure.
2. **Parse phase:** Parse all cues from the file content. Sort by start time. Store as an ordered array.
3. **Priority window:** Determine the current playback position. Identify the next 10 cues as the priority window.
4. **Priority tokenization:** Tokenize the priority window cues sequentially, storing results into the `SubtitleProcessingController`'s tokenization cache.
5. **Background tokenization:** After the priority window is done, tokenize remaining cues working forward from the current position, then wrapping around to cover earlier cues. The prefetcher stops once it has tokenized all cues or the cache is full (whichever comes first) to avoid wasteful eviction churn. For files with more cues than the cache limit, background tokenization focuses on cues ahead of the current position.
6. **Seek handling:** On seek, re-compute the priority window from the new position. A seek is detected by observing MPV's `time-pos` property and checking if the delta from the last observed position exceeds a threshold (e.g., > 3 seconds forward or any backward jump). The current in-flight tokenization finishes naturally, then the new priority window takes over.
7. **Teardown:** When the subtitle track changes or playback ends, stop all prefetch work and discard state.
#### Live Priority
The prefetcher and live subtitle handler share the Yomitan parser (single-threaded IPC). Live subtitle requests must always take priority. The prefetcher:
- Checks a `paused` flag before each cue tokenization. The live handler sets `paused = true` on subtitle change and clears it after emission.
- Yields between each background cue tokenization (via `setTimeout(0)` or equivalent) so the live handler can set the pause flag between cues.
- When paused, the prefetcher waits (polling the flag on a short interval or awaiting a resume signal) before continuing with the next cue.
#### Cache Integration
The prefetcher calls the same `tokenizeSubtitle` function used by live processing to produce `SubtitleData` results, then stores them into the existing `SubtitleProcessingController` tokenization cache via a new method:
```typescript
// New methods on SubtitleProcessingController
preCacheTokenization: (text: string, data: SubtitleData) => void;
isCacheFull: () => boolean;
```
`preCacheTokenization` uses the same `setCachedTokenization` logic internally (LRU eviction, Map-based storage). `isCacheFull` returns `true` when the cache has reached its limit, allowing the prefetcher to stop background tokenization and avoid wasteful eviction churn.
#### Cache Invalidation
When the user marks a word as known (or any event triggers `invalidateTokenizationCache()`), all cached results are cleared -- including prefetched ones, since they share the same cache. After invalidation, the prefetcher re-computes the priority window from the current playback position and re-tokenizes those cues to restore warm cache state.
#### Error Handling
If the subtitle file is malformed or partially parseable, the cue parser uses what it can extract. A file that yields zero cues disables prefetching silently (falls back to live-only processing). Encoding errors from `loadSubtitleSourceText` are caught and logged; prefetching is skipped for that track.
#### Integration Points
- **MPV property subscriptions:** Needs `track-list` (to detect external subtitle file path) and `time-pos` (to track playback position for window calculation and seek detection).
- **File loading:** Uses existing `loadSubtitleSourceText` dependency.
- **Tokenization:** Calls the same `tokenizeSubtitle` function used by live processing.
- **Cache:** Writes into `SubtitleProcessingController`'s cache.
- **Cache invalidation:** Listens for cache invalidation events to re-prefetch the priority window.
### Files Affected
- **New:** `src/core/services/subtitle-prefetch.ts` -- the prefetch service
- **New:** `src/core/services/subtitle-cue-parser.ts` -- SRT/VTT/ASS cue parser (text + timing)
- **Modified:** `src/core/services/subtitle-processing-controller.ts` -- expose `preCacheTokenization` method
- **Modified:** `src/main.ts` -- wire up the prefetch service, listen to track changes
---
## Optimization 2: Batched Annotation Pass
### Summary
Collapse the 4 sequential annotation passes (`applyKnownWordMarking` -> `applyFrequencyMarking` -> `applyJlptMarking` -> `markNPlusOneTargets`) into a single iteration over the token array, followed by N+1 marking.
**Important context:** Frequency rank _values_ (`token.frequencyRank`) are already assigned at the parser level by `applyFrequencyRanks()` in `tokenizer.ts`, before the annotation stage is called. The annotation stage's `applyFrequencyMarking` only performs POS-based _filtering_ -- clearing `frequencyRank` to `undefined` for tokens that should be excluded (particles, noise tokens, etc.) and normalizing valid ranks. This optimization does not change the parser-level frequency rank assignment; it only batches the annotation-level filtering.
### Current Flow (4 passes, 4 array copies)
```text
tokens (already have frequencyRank values from parser-level applyFrequencyRanks)
-> applyKnownWordMarking() // .map() -> new array
-> applyFrequencyMarking() // .map() -> new array (POS-based filtering only)
-> applyJlptMarking() // .map() -> new array
-> markNPlusOneTargets() // .map() -> new array
```
### Dependency Analysis
All annotations either depend on MeCab POS data or benefit from running after it:
- **Known word marking:** Needs base tokens (surface/headword). No POS dependency, but no reason to run separately.
- **Frequency filtering:** Uses `pos1Exclusions` and `pos2Exclusions` to clear frequency ranks on excluded tokens (particles, noise). Depends on MeCab POS data.
- **JLPT marking:** Uses `shouldIgnoreJlptForMecabPos1` to filter. Depends on MeCab POS data.
- **N+1 marking:** Uses POS exclusion sets to filter candidates. Depends on known word status + MeCab POS.
Since frequency filtering and JLPT marking both depend on POS data from MeCab enrichment, and MeCab enrichment already happens before the annotation stage, all four can run in a single pass after MeCab completes.
### New Flow (1 pass + N+1)
```typescript
function annotateTokens(tokens, deps, options): MergedToken[] {
const pos1Exclusions = resolvePos1Exclusions(options);
const pos2Exclusions = resolvePos2Exclusions(options);
// Single pass: known word + frequency filtering + JLPT computed together
const annotated = tokens.map((token) => {
const isKnown = nPlusOneEnabled
? token.isKnown || computeIsKnown(token, deps)
: false;
// Filter frequency rank using POS exclusions (rank values already set at parser level)
const frequencyRank = frequencyEnabled
? filterFrequencyRank(token, pos1Exclusions, pos2Exclusions)
: undefined;
const jlptLevel = jlptEnabled
? computeJlptLevel(token, deps.getJlptLevel)
: undefined;
return { ...token, isKnown, frequencyRank, jlptLevel };
});
// N+1 must run after known word status is set for all tokens
if (nPlusOneEnabled) {
return markNPlusOneTargets(annotated, minSentenceWords, pos1Exclusions, pos2Exclusions);
}
return annotated;
}
```
### What Changes
- The individual `applyKnownWordMarking`, `applyFrequencyMarking`, `applyJlptMarking` functions are refactored into per-token computation helpers (pure functions that compute a single field). The frequency helper is named `filterFrequencyRank` to clarify it performs POS-based exclusion, not rank computation.
- The `annotateTokens` orchestrator runs one `.map()` call that invokes all three helpers per token.
- `markNPlusOneTargets` remains a separate pass because it needs the full array with `isKnown` set (it examines sentence-level context).
- The parser-level `applyFrequencyRanks()` call in `tokenizer.ts` is unchanged -- it remains a separate step outside the annotation stage.
- Net: 4 array copies + 4 iterations become 1 array copy + 1 iteration + N+1 pass.
### Expected Savings
~15-45ms saved (3 fewer array allocations + 3 fewer full iterations). Annotation drops from ~25-70ms to ~10-25ms.
### Files Affected
- **Modified:** `src/core/services/tokenizer/annotation-stage.ts` -- refactor into batched single-pass
---
## Optimization 3: DOM Template Pooling
### Summary
Replace `document.createElement('span')` calls in the renderer with `templateSpan.cloneNode(false)` from a pre-created template element.
### Current Behavior
In `renderWithTokens` (`subtitle-render.ts`), each render cycle:
1. Clears DOM with `innerHTML = ''`
2. Creates a `DocumentFragment`
3. Calls `document.createElement('span')` for each token (~10-15 per subtitle)
4. Sets `className`, `textContent`, `dataset.*` individually
5. Appends fragment to root
### New Behavior
1. At renderer initialization (`createSubtitleRenderer`), create a single template:
```typescript
const templateSpan = document.createElement('span');
```
2. In `renderWithTokens`, replace every `document.createElement('span')` with:
```typescript
const span = templateSpan.cloneNode(false) as HTMLSpanElement;
```
3. Replace all `innerHTML = ''` calls with `root.replaceChildren()` to avoid the HTML parser invocation on clear. This applies to `renderSubtitle` (primary subtitle root), `renderSecondarySub` (secondary subtitle root), and `renderCharacterLevel` if applicable.
4. Everything else stays the same (setting className, textContent, dataset, appending to fragment).
### Why cloneNode Over Full Node Recycling
Full recycling (collecting old nodes, clearing attributes, reusing them) requires carefully resetting every `dataset.*` property that might have been set on a previous render. This is error-prone -- a stale `data-frequency-rank` from a previous subtitle appearing on a new token would cause incorrect styling. `cloneNode(false)` on a bare template is nearly as fast and produces a clean node every time.
### Expected Savings
`cloneNode(false)` is ~2-3x faster than `createElement` in most browser engines. For 10-15 tokens per subtitle: ~3-8ms saved per render cycle.
### Files Affected
- **Modified:** `src/renderer/subtitle-render.ts` -- template creation + cloneNode usage
---
## Combined Impact Summary
| Scenario | Before | After | Improvement |
|----------|--------|-------|-------------|
| Normal playback (prefetch-warmed) | ~200-320ms | ~30-50ms | ~80-85% |
| Cache hit (repeated subtitle) | ~72ms | ~55-65ms | ~10-20% |
| Cache miss (immediate seek) | ~200-320ms | ~150-260ms | ~20-25% |
---
## Files Summary
### New Files
- `src/core/services/subtitle-prefetch.ts`
- `src/core/services/subtitle-cue-parser.ts`
### Modified Files
- `src/core/services/subtitle-processing-controller.ts` (expose `preCacheTokenization`)
- `src/core/services/tokenizer/annotation-stage.ts` (batched single-pass)
- `src/renderer/subtitle-render.ts` (template cloneNode)
- `src/main.ts` (wire up prefetch service)
### Test Files
- New tests for subtitle cue parser (SRT, VTT, ASS formats)
- New tests for subtitle prefetch service (priority window, seek, pause/resume)
- Updated tests for annotation stage (same behavior, new implementation)
- Updated tests for subtitle render (template cloning)

View File

@@ -0,0 +1,37 @@
<!-- read_when: changing runtime wiring, moving code across layers, or trying to find ownership -->
# Architecture Map
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: runtime ownership, composition boundaries, or layering questions
SubMiner runs as three cooperating runtimes:
- Electron desktop app in `src/`
- Launcher CLI in `launcher/`
- mpv Lua plugin in `plugin/subminer/`
The desktop app keeps `src/main.ts` as composition root and pushes behavior into small runtime/domain modules.
## Read Next
- [Domains](./domains.md) - who owns what
- [Layering](./layering.md) - how modules should depend on each other
- Public contributor summary: [`docs-site/architecture.md`](../../docs-site/architecture.md)
## Current Shape
- `src/main/` owns composition, runtime setup, IPC wiring, and app lifecycle adapters.
- `src/core/services/` owns focused runtime services plus pure or side-effect-bounded logic.
- `src/renderer/` owns overlay rendering and input behavior.
- `src/config/` owns config definitions, defaults, loading, and resolution.
- `src/main/runtime/composers/` owns larger domain compositions.
## Architecture Intent
- Small units, explicit boundaries
- Composition over monoliths
- Pure helpers where possible
- Stable user behavior while internals evolve

View File

@@ -0,0 +1,38 @@
<!-- read_when: locating ownership for a runtime, feature, or integration -->
# Domain Ownership
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: you need to find the owner module for a behavior or test surface
## Runtime Domains
- Desktop app runtime: `src/main.ts`, `src/main/`, `src/core/services/`
- Overlay renderer: `src/renderer/`
- Launcher CLI: `launcher/`
- mpv plugin: `plugin/subminer/`
## Product / Integration Domains
- Config system: `src/config/`
- Overlay/window state: `src/core/services/overlay-*`, `src/main/overlay-*.ts`
- MPV runtime and protocol: `src/core/services/mpv*.ts`
- Subtitle/token pipeline: `src/core/services/tokenizer*`, `src/subtitle/`, `src/tokenizers/`
- Anki workflow: `src/anki-integration/`, `src/core/services/anki-jimaku*.ts`
- Immersion tracking: `src/core/services/immersion-tracker/`
- AniList tracking: `src/core/services/anilist/`, `src/main/runtime/composers/anilist-*`
- Jellyfin integration: `src/core/services/jellyfin*.ts`, `src/main/runtime/composers/jellyfin-*`
- Window trackers: `src/window-trackers/`
- Stats app: `stats/`
- Public docs site: `docs-site/`
## Ownership Heuristics
- Runtime wiring or dependency setup: start in `src/main/`
- Business logic or service behavior: start in `src/core/services/`
- UI interaction or overlay DOM behavior: start in `src/renderer/`
- Command parsing or mpv launch flow: start in `launcher/`
- User-facing docs: `docs-site/`
- Internal process/docs: `docs/`

View File

@@ -0,0 +1,33 @@
<!-- read_when: adding dependencies, moving files, or reviewing architecture drift -->
# Layering Rules
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: deciding whether a dependency direction is acceptable
## Preferred Dependency Flow
1. `src/main.ts`
2. `src/main/` composition and runtime adapters
3. `src/core/services/` focused services
4. `src/core/utils/` and other pure helpers
Renderer, launcher, plugin, and stats each keep their own local layering and should not become a grab bag for unrelated cross-runtime behavior.
## Rules
- Keep `src/main.ts` thin; wire, do not implement.
- Prefer injecting dependencies from `src/main/` instead of reaching outward from core services.
- Keep side effects explicit and close to composition boundaries.
- Put reusable business logic in focused services, not in top-level lifecycle files.
- Keep renderer concerns in `src/renderer/`; avoid leaking DOM behavior into main-process code.
- Treat `launcher/*.ts` as source of truth for the launcher. Never hand-edit `dist/launcher/subminer`.
## Smells
- `main.ts` grows because logic was not extracted
- service reaches directly into unrelated runtime state
- renderer code depends on main-process internals
- docs-site page becomes the only place internal architecture is explained

View File

@@ -0,0 +1,35 @@
<!-- read_when: changing internal docs structure, adding guidance, or debugging doc drift -->
# Knowledge Base Rules
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: maintaining the internal doc system itself
This section defines how the internal knowledge base is organized and maintained.
## Read Next
- [Core Beliefs](./core-beliefs.md) - agent-first operating principles
- [Catalog](./catalog.md) - indexed docs and verification status
- [Quality](./quality.md) - current doc and architecture quality grades
## Policy
- `AGENTS.md` is an entrypoint only.
- `docs/` is the internal system of record.
- `docs-site/` is user-facing; do not treat it as canonical internal design or workflow storage.
- Internal docs should be short, cross-linked, and specific.
- Every core internal doc should include:
- `Status`
- `Last verified`
- `Owner`
- `Read when`
## Maintenance
- Update the relevant internal doc when behavior or workflow changes.
- Add new docs to the [Catalog](./catalog.md).
- Record architectural quality drift in [Quality](./quality.md).
- Keep stale docs obvious; do not leave ambiguity about whether a page is trustworthy.

View File

@@ -0,0 +1,29 @@
<!-- read_when: you need to know what internal docs exist, whether they are current, or what should be updated -->
# Documentation Catalog
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: finding internal docs or checking verification status
| Area | Path | Status | Last verified | Notes |
| --- | --- | --- | --- | --- |
| KB home | `docs/README.md` | active | 2026-03-13 | internal entrypoint |
| Architecture index | `docs/architecture/README.md` | active | 2026-03-13 | top-level runtime map |
| Domain ownership | `docs/architecture/domains.md` | active | 2026-03-13 | runtime and feature ownership |
| Layering rules | `docs/architecture/layering.md` | active | 2026-03-13 | dependency direction and smells |
| KB rules | `docs/knowledge-base/README.md` | active | 2026-03-13 | maintenance policy |
| Core beliefs | `docs/knowledge-base/core-beliefs.md` | active | 2026-03-13 | agent-first principles |
| Quality scorecard | `docs/knowledge-base/quality.md` | active | 2026-03-13 | quality grades and gaps |
| Workflow index | `docs/workflow/README.md` | active | 2026-03-13 | execution map |
| Planning guide | `docs/workflow/planning.md` | active | 2026-03-13 | lightweight vs execution plans |
| Verification guide | `docs/workflow/verification.md` | active | 2026-03-13 | maintained verification lanes |
| Release guide | `docs/RELEASING.md` | active | 2026-03-13 | release checklist |
| Active plans | `docs/plans/` | active | 2026-03-13 | task-scoped design and implementation artifacts |
## Update Rules
- Add a row when introducing a new core internal doc.
- Update `Status` and `Last verified` when a page is materially revised.
- If a page is known inaccurate, mark it stale immediately instead of leaving silent drift.

View File

@@ -0,0 +1,25 @@
<!-- read_when: deciding how much context to inject, where docs should live, or how agents should navigate the repo -->
# Core Beliefs
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: making decisions about agent ergonomics, doc structure, or repository guidance
## Agent-First Principles
- Progressive disclosure beats giant injected context.
- `AGENTS.md` should map the territory, not duplicate it.
- Canonical internal guidance belongs in versioned docs near the code.
- Plans are first-class while active work is happening.
- Mechanical checks beat social convention when the boundary matters.
- Small focused docs are easier to trust, update, and verify.
- User-facing docs and internal operating docs should not blur together.
## What This Means Here
- Start from `AGENTS.md`, then move into `docs/`.
- Prefer links to canonical docs over repeating long instructions.
- Keep architecture and workflow docs in separate pages so updates stay targeted.
- When a page becomes long or multi-purpose, split it.

View File

@@ -0,0 +1,40 @@
<!-- read_when: assessing architecture health, doc gaps, or where cleanup effort should go next -->
# Quality Scorecard
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: triaging internal quality gaps or deciding where follow-up work is needed
Grades are directional, not ceremonial. The point is to keep gaps visible.
## Product / Runtime Domains
| Area | Grade | Notes |
| --- | --- | --- |
| Desktop runtime composition | B | strong modularization; still easy for `main` wiring drift to reappear |
| Launcher CLI | B | focused surface; generated/stale artifact hazards need constant guarding |
| mpv plugin | B | modular, but Lua/runtime coupling still specialized |
| Overlay renderer | B | improved modularity; interaction complexity remains |
| Config system | A- | clear defaults/definitions split and good validation surface |
| Immersion / AniList / Jellyfin surfaces | B- | growing product scope; ownership spans multiple services |
| Internal docs system | B | new structure in place; needs habitual maintenance |
| Public docs site | B | strong user docs; must stay separate from internal KB |
## Architectural Layers
| Layer | Grade | Notes |
| --- | --- | --- |
| `src/main.ts` composition root | B | direction good; still needs vigilance against logic creep |
| `src/main/` runtime adapters | B | mostly clear; can accumulate wiring debt |
| `src/core/services/` | B+ | good extraction pattern; some domains remain broad |
| `src/renderer/` | B | cleaner than before; UI/runtime behavior still dense |
| `launcher/` | B | clear command boundaries |
| `docs/` internal KB | B | structure exists; enforcement now guards core rules |
## Current Gaps
- Some deep architecture detail still lives in `docs-site/architecture.md` and may merit later migration.
- Quality grading is manual and should be refreshed when major refactors land.
- Active plans can accumulate without lifecycle cleanup if humans do not prune them.

30
docs/workflow/README.md Normal file
View File

@@ -0,0 +1,30 @@
<!-- read_when: starting implementation, deciding whether to plan, or checking handoff expectations -->
# Workflow
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: planning or executing nontrivial work in this repo
This section is the internal workflow map for contributors and agents.
## Read Next
- [Planning](./planning.md) - when to write a lightweight plan vs a full execution plan
- [Verification](./verification.md) - maintained test/build lanes and handoff gate
- [Release Guide](../RELEASING.md) - tagged release workflow
## Default Flow
1. Read the smallest relevant docs from `docs/`.
2. Decide whether the work needs a written plan.
3. Implement in small, reviewable edits.
4. Run the cheapest sufficient verification lane.
5. Escalate to the full maintained gate before handoff when the change is substantial.
## Boundaries
- Internal process lives in `docs/`.
- Public/product docs live in `docs-site/`.
- Generated artifacts are never edited by hand.

41
docs/workflow/planning.md Normal file
View File

@@ -0,0 +1,41 @@
<!-- read_when: deciding whether work needs a plan or writing one -->
# Planning
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: the task spans multiple files, subsystems, or verification lanes
## Plan Types
- Lightweight plan: small change, a few reversible steps, minimal coordination
- Execution plan: nontrivial feature/refactor/debugging effort with multiple phases or important decisions
## Use a Lightweight Plan When
- one subsystem
- obvious change shape
- low risk
- easy to verify
## Use an Execution Plan When
- multiple subsystems or runtimes
- architectural tradeoffs matter
- staged verification is needed
- the work should be resumable by another agent or human
## Plan Location
- active design and implementation docs live in `docs/plans/`
- keep names date-prefixed and task-specific
- remove or archive old plans deliberately; do not leave mystery artifacts
## Plan Contents
- problem / goal
- non-goals
- file ownership or edit scope
- verification plan
- decisions made during execution

View File

@@ -0,0 +1,41 @@
<!-- read_when: choosing what tests/build steps to run before handoff -->
# Verification
Status: active
Last verified: 2026-03-13
Owner: Kyle Yasuda
Read when: selecting the right verification lane for a change
## Default Handoff Gate
```bash
bun run typecheck
bun run test:fast
bun run test:env
bun run build
bun run test:smoke:dist
```
If `docs-site/` changed, also run:
```bash
bun run docs:test
bun run docs:build
```
## Cheap-First Lane Selection
- Docs-only boundary/content changes: `bun run docs:test`, `bun run docs:build`
- Internal KB / `AGENTS.md` changes: `bun run test:docs:kb`
- Config/schema/defaults: `bun run test:config`, then `bun run generate:config-example` if template/defaults changed
- Launcher/plugin: `bun run test:launcher` or `bun run test:env`
- Runtime-compat / compiled behavior: `bun run test:runtime:compat`
- Deep/local full gate: default handoff gate above
## Rules
- Capture exact failing command and error when verification breaks.
- Prefer the cheapest sufficient lane first.
- Escalate when the change crosses boundaries or touches release-sensitive behavior.
- Never hand-edit `dist/launcher/subminer`; validate it through build/test flow instead.

View File

@@ -7,6 +7,7 @@ import { runConfigCommand } from './config-command.js';
import { runDictionaryCommand } from './dictionary-command.js';
import { runDoctorCommand } from './doctor-command.js';
import { runMpvPreAppCommand } from './mpv-command.js';
import { runStatsCommand } from './stats-command.js';
class ExitSignal extends Error {
code: number;
@@ -128,3 +129,98 @@ test('dictionary command throws if app handoff unexpectedly returns', () => {
/unexpectedly returned/,
);
});
test('stats command launches attached app command with response path', async () => {
const context = createContext();
context.args.stats = true;
context.args.logLevel = 'debug';
const forwarded: string[][] = [];
const handled = await runStatsCommand(context, {
createTempDir: () => '/tmp/subminer-stats-test',
joinPath: (...parts) => parts.join('/'),
runAppCommandAttached: async (_appPath, appArgs) => {
forwarded.push(appArgs);
return 0;
},
waitForStatsResponse: async () => ({ ok: true, url: 'http://127.0.0.1:5175' }),
removeDir: () => {},
});
assert.equal(handled, true);
assert.deepEqual(forwarded, [
['--stats', '--stats-response-path', '/tmp/subminer-stats-test/response.json', '--log-level', 'debug'],
]);
});
test('stats cleanup command forwards cleanup vocab flags to the app', async () => {
const context = createContext();
context.args.stats = true;
context.args.statsCleanup = true;
context.args.statsCleanupVocab = true;
const forwarded: string[][] = [];
const handled = await runStatsCommand(context, {
createTempDir: () => '/tmp/subminer-stats-test',
joinPath: (...parts) => parts.join('/'),
runAppCommandAttached: async (_appPath, appArgs) => {
forwarded.push(appArgs);
return 0;
},
waitForStatsResponse: async () => ({ ok: true }),
removeDir: () => {},
});
assert.equal(handled, true);
assert.deepEqual(forwarded, [
[
'--stats',
'--stats-response-path',
'/tmp/subminer-stats-test/response.json',
'--stats-cleanup',
'--stats-cleanup-vocab',
],
]);
});
test('stats command throws when stats response reports an error', async () => {
const context = createContext();
context.args.stats = true;
await assert.rejects(
async () => {
await runStatsCommand(context, {
createTempDir: () => '/tmp/subminer-stats-test',
joinPath: (...parts) => parts.join('/'),
runAppCommandAttached: async () => 0,
waitForStatsResponse: async () => ({
ok: false,
error: 'Immersion tracking is disabled in config.',
}),
removeDir: () => {},
});
},
/Immersion tracking is disabled in config\./,
);
});
test('stats command fails if attached app exits before startup response', async () => {
const context = createContext();
context.args.stats = true;
await assert.rejects(
async () => {
await runStatsCommand(context, {
createTempDir: () => '/tmp/subminer-stats-test',
joinPath: (...parts) => parts.join('/'),
runAppCommandAttached: async () => 2,
waitForStatsResponse: async () => {
await new Promise((resolve) => setTimeout(resolve, 25));
return { ok: true, url: 'http://127.0.0.1:5175' };
},
removeDir: () => {},
});
},
/Stats app exited before startup response \(status 2\)\./,
);
});

View File

@@ -0,0 +1,108 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { runAppCommandAttached } from '../mpv.js';
import { sleep } from '../util.js';
import type { LauncherCommandContext } from './context.js';
type StatsCommandResponse = {
ok: boolean;
url?: string;
error?: string;
};
type StatsCommandDeps = {
createTempDir: (prefix: string) => string;
joinPath: (...parts: string[]) => string;
runAppCommandAttached: (
appPath: string,
appArgs: string[],
logLevel: LauncherCommandContext['args']['logLevel'],
label: string,
) => Promise<number>;
waitForStatsResponse: (responsePath: string) => Promise<StatsCommandResponse>;
removeDir: (targetPath: string) => void;
};
const defaultDeps: StatsCommandDeps = {
createTempDir: (prefix) => fs.mkdtempSync(path.join(os.tmpdir(), prefix)),
joinPath: (...parts) => path.join(...parts),
runAppCommandAttached: (appPath, appArgs, logLevel, label) =>
runAppCommandAttached(appPath, appArgs, logLevel, label),
waitForStatsResponse: async (responsePath) => {
const deadline = Date.now() + 8000;
while (Date.now() < deadline) {
try {
if (fs.existsSync(responsePath)) {
return JSON.parse(fs.readFileSync(responsePath, 'utf8')) as StatsCommandResponse;
}
} catch {
// retry until timeout
}
await sleep(100);
}
return {
ok: false,
error: 'Timed out waiting for stats dashboard startup response.',
};
},
removeDir: (targetPath) => {
fs.rmSync(targetPath, { recursive: true, force: true });
},
};
export async function runStatsCommand(
context: LauncherCommandContext,
deps: StatsCommandDeps = defaultDeps,
): Promise<boolean> {
const { args, appPath } = context;
if (!args.stats || !appPath) {
return false;
}
const tempDir = deps.createTempDir('subminer-stats-');
const responsePath = deps.joinPath(tempDir, 'response.json');
try {
const forwarded = ['--stats', '--stats-response-path', responsePath];
if (args.statsCleanup) {
forwarded.push('--stats-cleanup');
}
if (args.statsCleanupVocab) {
forwarded.push('--stats-cleanup-vocab');
}
if (args.logLevel !== 'info') {
forwarded.push('--log-level', args.logLevel);
}
const attachedExitPromise = deps.runAppCommandAttached(
appPath,
forwarded,
args.logLevel,
'stats',
);
const startupResult = await Promise.race([
deps.waitForStatsResponse(responsePath).then((response) => ({ kind: 'response' as const, response })),
attachedExitPromise.then((status) => ({ kind: 'exit' as const, status })),
]);
if (startupResult.kind === 'exit') {
if (startupResult.status !== 0) {
throw new Error(`Stats app exited before startup response (status ${startupResult.status}).`);
}
const response = await deps.waitForStatsResponse(responsePath);
if (!response.ok) {
throw new Error(response.error || 'Stats dashboard failed to start.');
}
return true;
}
if (!startupResult.response.ok) {
throw new Error(startupResult.response.error || 'Stats dashboard failed to start.');
}
const exitStatus = await attachedExitPromise;
if (exitStatus !== 0) {
throw new Error(`Stats app exited with status ${exitStatus}.`);
}
return true;
} finally {
deps.removeDir(tempDir);
}
}

View File

@@ -122,6 +122,9 @@ export function createDefaultArgs(launcherConfig: LauncherYoutubeSubgenConfig):
jellyfinPlay: false,
jellyfinDiscovery: false,
dictionary: false,
stats: false,
statsCleanup: false,
statsCleanupVocab: false,
doctor: false,
configPath: false,
configShow: false,
@@ -188,6 +191,9 @@ export function applyRootOptionsToArgs(
export function applyInvocationsToArgs(parsed: Args, invocations: CliInvocations): void {
if (invocations.dictionaryTriggered) parsed.dictionary = true;
if (invocations.statsTriggered) parsed.stats = true;
if (invocations.statsCleanup) parsed.statsCleanup = true;
if (invocations.statsCleanupVocab) parsed.statsCleanupVocab = true;
if (invocations.dictionaryTarget) {
parsed.dictionaryTarget = parseDictionaryTarget(invocations.dictionaryTarget);
}
@@ -256,6 +262,9 @@ export function applyInvocationsToArgs(parsed: Args, invocations: CliInvocations
if (invocations.dictionaryLogLevel) {
parsed.logLevel = parseLogLevel(invocations.dictionaryLogLevel);
}
if (invocations.statsLogLevel) {
parsed.logLevel = parseLogLevel(invocations.statsLogLevel);
}
if (invocations.doctorLogLevel) parsed.logLevel = parseLogLevel(invocations.doctorLogLevel);
if (invocations.texthookerLogLevel)

View File

@@ -40,6 +40,10 @@ export interface CliInvocations {
dictionaryTriggered: boolean;
dictionaryTarget: string | null;
dictionaryLogLevel: string | null;
statsTriggered: boolean;
statsCleanup: boolean;
statsCleanupVocab: boolean;
statsLogLevel: string | null;
doctorTriggered: boolean;
doctorLogLevel: string | null;
texthookerTriggered: boolean;
@@ -87,6 +91,7 @@ function getTopLevelCommand(argv: string[]): { name: string; index: number } | n
'mpv',
'dictionary',
'dict',
'stats',
'texthooker',
'app',
'bin',
@@ -137,6 +142,10 @@ export function parseCliPrograms(
let dictionaryTriggered = false;
let dictionaryTarget: string | null = null;
let dictionaryLogLevel: string | null = null;
let statsTriggered = false;
let statsCleanup = false;
let statsCleanupVocab = false;
let statsLogLevel: string | null = null;
let doctorLogLevel: string | null = null;
let texthookerLogLevel: string | null = null;
let doctorTriggered = false;
@@ -241,6 +250,21 @@ export function parseCliPrograms(
dictionaryLogLevel = typeof options.logLevel === 'string' ? options.logLevel : null;
});
commandProgram
.command('stats')
.description('Launch the local immersion stats dashboard')
.argument('[action]', 'cleanup')
.option('-v, --vocab', 'Clean vocabulary rows in the stats database')
.option('--log-level <level>', 'Log level')
.action((action: string | undefined, options: Record<string, unknown>) => {
statsTriggered = true;
if ((action || '').toLowerCase() === 'cleanup') {
statsCleanup = true;
statsCleanupVocab = options.vocab !== false;
}
statsLogLevel = typeof options.logLevel === 'string' ? options.logLevel : null;
});
commandProgram
.command('doctor')
.description('Run dependency and environment checks')
@@ -319,6 +343,10 @@ export function parseCliPrograms(
dictionaryTriggered,
dictionaryTarget,
dictionaryLogLevel,
statsTriggered,
statsCleanup,
statsCleanupVocab,
statsLogLevel,
doctorTriggered,
doctorLogLevel,
texthookerTriggered,

View File

@@ -335,6 +335,55 @@ test('dictionary command forwards --dictionary and --dictionary-target to app co
});
});
test('stats command launches attached app flow and waits for response file', { timeout: 15000 }, () => {
withTempDir((root) => {
const homeDir = path.join(root, 'home');
const xdgConfigHome = path.join(root, 'xdg');
const appPath = path.join(root, 'fake-subminer.sh');
const capturePath = path.join(root, 'captured-args.txt');
fs.writeFileSync(
appPath,
`#!/bin/sh
set -eu
response_path=""
prev=""
for arg in "$@"; do
if [ "$prev" = "--stats-response-path" ]; then
response_path="$arg"
prev=""
continue
fi
case "$arg" in
--stats-response-path=*)
response_path="\${arg#--stats-response-path=}"
;;
--stats-response-path)
prev="--stats-response-path"
;;
esac
done
if [ -n "$SUBMINER_TEST_STATS_CAPTURE" ]; then
printf '%s\\n' "$@" > "$SUBMINER_TEST_STATS_CAPTURE"
fi
mkdir -p "$(dirname "$response_path")"
printf '%s' '{"ok":true,"url":"http://127.0.0.1:5175"}' > "$response_path"
exit 0
`,
);
fs.chmodSync(appPath, 0o755);
const env = {
...makeTestEnv(homeDir, xdgConfigHome),
SUBMINER_APPIMAGE_PATH: appPath,
SUBMINER_TEST_STATS_CAPTURE: capturePath,
};
const result = runLauncher(['stats', '--log-level', 'debug'], env);
assert.equal(result.status, 0, `stdout:\n${result.stdout}\nstderr:\n${result.stderr}`);
assert.match(fs.readFileSync(capturePath, 'utf8'), /^--stats\n--stats-response-path\n.+\n--log-level\ndebug\n$/);
});
});
test('jellyfin discovery routes to app --background and remote announce with log-level forwarding', () => {
withTempDir((root) => {
const homeDir = path.join(root, 'home');

View File

@@ -14,6 +14,7 @@ import { runConfigCommand } from './commands/config-command.js';
import { runMpvPostAppCommand, runMpvPreAppCommand } from './commands/mpv-command.js';
import { runAppPassthroughCommand, runTexthookerCommand } from './commands/app-command.js';
import { runDictionaryCommand } from './commands/dictionary-command.js';
import { runStatsCommand } from './commands/stats-command.js';
import { runJellyfinCommand } from './commands/jellyfin-command.js';
import { runPlaybackCommand } from './commands/playback-command.js';
@@ -95,6 +96,10 @@ async function main(): Promise<void> {
return;
}
if (await runStatsCommand(appContext)) {
return;
}
if (await runJellyfinCommand(appContext)) {
return;
}

View File

@@ -133,6 +133,7 @@ function makeArgs(overrides: Partial<Args> = {}): Args {
jellyfinPlay: false,
jellyfinDiscovery: false,
dictionary: false,
stats: false,
doctor: false,
configPath: false,
configShow: false,

View File

@@ -756,6 +756,37 @@ export function runAppCommandCaptureOutput(
};
}
export function runAppCommandAttached(
appPath: string,
appArgs: string[],
logLevel: LogLevel,
label: string,
): Promise<number> {
if (maybeCaptureAppArgs(appArgs)) {
return Promise.resolve(0);
}
const target = resolveAppSpawnTarget(appPath, appArgs);
log(
'debug',
logLevel,
`${label}: launching attached app with args: ${[target.command, ...target.args].join(' ')}`,
);
return new Promise((resolve, reject) => {
const proc = spawn(target.command, target.args, {
stdio: 'inherit',
env: buildAppEnv(),
});
proc.once('error', (error) => {
reject(error);
});
proc.once('exit', (code) => {
resolve(code ?? 0);
});
});
}
export function runAppCommandWithInheritLogged(
appPath: string,
appArgs: string[],
@@ -786,10 +817,24 @@ export function runAppCommandWithInheritLogged(
export function launchAppStartDetached(appPath: string, logLevel: LogLevel): void {
const startArgs = ['--start'];
if (logLevel !== 'info') startArgs.push('--log-level', logLevel);
if (maybeCaptureAppArgs(startArgs)) {
launchAppCommandDetached(appPath, startArgs, logLevel, 'start');
}
export function launchAppCommandDetached(
appPath: string,
appArgs: string[],
logLevel: LogLevel,
label: string,
): void {
if (maybeCaptureAppArgs(appArgs)) {
return;
}
const target = resolveAppSpawnTarget(appPath, startArgs);
const target = resolveAppSpawnTarget(appPath, appArgs);
log(
'debug',
logLevel,
`${label}: launching detached app with args: ${[target.command, ...target.args].join(' ')}`,
);
const proc = spawn(target.command, target.args, {
stdio: 'ignore',
detached: true,

View File

@@ -58,3 +58,26 @@ test('parseArgs maps dictionary command and log-level override', () => {
assert.equal(parsed.dictionaryTarget, process.cwd());
assert.equal(parsed.logLevel, 'debug');
});
test('parseArgs maps stats command and log-level override', () => {
const parsed = parseArgs(['stats', '--log-level', 'debug'], 'subminer', {});
assert.equal(parsed.stats, true);
assert.equal(parsed.logLevel, 'debug');
});
test('parseArgs maps stats cleanup to vocab mode by default', () => {
const parsed = parseArgs(['stats', 'cleanup'], 'subminer', {});
assert.equal(parsed.stats, true);
assert.equal(parsed.statsCleanup, true);
assert.equal(parsed.statsCleanupVocab, true);
});
test('parseArgs maps explicit stats cleanup vocab flag', () => {
const parsed = parseArgs(['stats', 'cleanup', '-v'], 'subminer', {});
assert.equal(parsed.stats, true);
assert.equal(parsed.statsCleanup, true);
assert.equal(parsed.statsCleanupVocab, true);
});

View File

@@ -111,6 +111,9 @@ export interface Args {
jellyfinPlay: boolean;
jellyfinDiscovery: boolean;
dictionary: boolean;
stats: boolean;
statsCleanup?: boolean;
statsCleanupVocab?: boolean;
dictionaryTarget?: string;
doctor: boolean;
configPath: boolean;

View File

@@ -13,7 +13,9 @@
"test-yomitan-parser:electron": "bun run build:yomitan && bun build scripts/test-yomitan-parser.ts --format=cjs --target=node --outfile dist/scripts/test-yomitan-parser.js --external electron && electron dist/scripts/test-yomitan-parser.js",
"build:yomitan": "bun scripts/build-yomitan.mjs",
"build:assets": "bun scripts/prepare-build-assets.mjs",
"build": "bun run build:yomitan && tsc -p tsconfig.json && bun run build:renderer && bun run build:assets",
"build:stats": "cd stats && bun run build",
"dev:stats": "cd stats && bun run dev",
"build": "bun run build:yomitan && bun run build:stats && tsc -p tsconfig.json && bun run build:renderer && bun run build:assets",
"build:renderer": "esbuild src/renderer/renderer.ts --bundle --platform=browser --format=esm --target=es2022 --outfile=dist/renderer/renderer.js --sourcemap",
"changelog:build": "bun run scripts/build-changelog.ts build",
"changelog:check": "bun run scripts/build-changelog.ts check",
@@ -28,6 +30,7 @@
"docs:build": "bun run --cwd docs-site docs:build",
"docs:preview": "bun run --cwd docs-site docs:preview",
"docs:test": "bun run --cwd docs-site test",
"test:docs:kb": "bun test scripts/docs-knowledge-base.test.ts",
"test:config:src": "bun test src/config/config.test.ts src/config/path-resolution.test.ts src/config/resolve/anki-connect.test.ts src/config/resolve/subtitle-style.test.ts src/config/resolve/jellyfin.test.ts src/config/definitions/domain-registry.test.ts src/generate-config-example.test.ts src/verify-config-example.test.ts",
"test:config:dist": "bun test dist/config/config.test.js dist/config/path-resolution.test.js dist/config/resolve/anki-connect.test.js dist/config/resolve/subtitle-style.test.js dist/config/resolve/jellyfin.test.js dist/config/definitions/domain-registry.test.js dist/generate-config-example.test.js dist/verify-config-example.test.js",
"test:config:smoke:dist": "bun test dist/config/path-resolution.test.js",
@@ -54,7 +57,7 @@
"test:launcher": "bun run test:launcher:src",
"test:core": "bun run test:core:src",
"test:subtitle": "bun run test:subtitle:src",
"test:fast": "bun run test:config:src && bun run test:core:src && bun test src/main-entry-runtime.test.ts src/anki-integration/anki-connect-proxy.test.ts src/release-workflow.test.ts src/ci-workflow.test.ts scripts/build-changelog.test.ts scripts/mkv-to-readme-video.test.ts && bun run tsc && bun test dist/main/runtime/registry.test.js",
"test:fast": "bun run test:config:src && bun run test:core:src && bun run test:docs:kb && bun test src/main-entry-runtime.test.ts src/anki-integration/anki-connect-proxy.test.ts src/release-workflow.test.ts src/ci-workflow.test.ts scripts/build-changelog.test.ts scripts/mkv-to-readme-video.test.ts && bun run tsc && bun test dist/main/runtime/registry.test.js",
"generate:config-example": "bun run src/generate-config-example.ts",
"verify:config-example": "bun run src/verify-config-example.ts",
"start": "bun run build && electron . --start",
@@ -81,9 +84,11 @@
"author": "",
"license": "GPL-3.0-or-later",
"dependencies": {
"@hono/node-server": "^1.19.11",
"axios": "^1.13.5",
"commander": "^14.0.3",
"discord-rpc": "^4.0.1",
"hono": "^4.12.7",
"jsonc-parser": "^3.3.1",
"libsql": "^0.5.22",
"ws": "^8.19.0"
@@ -147,6 +152,7 @@
},
"files": [
"dist/**/*",
"stats/dist/**/*",
"vendor/texthooker-ui/docs/**/*",
"vendor/texthooker-ui/package.json",
"package.json",

View File

@@ -44,6 +44,9 @@ function M.create(ctx)
mp.register_script_message(hover.HOVER_MESSAGE_NAME_LEGACY, function(payload_json)
hover.handle_hover_message(payload_json)
end)
mp.register_script_message("subminer-stats-toggle", function()
mp.osd_message("Stats: press ` (backtick) in overlay", 3)
end)
end
return {

View File

@@ -32,6 +32,7 @@ function M.create(ctx)
"Open options",
"Restart overlay",
"Check status",
"Stats",
}
local actions = {
@@ -53,6 +54,9 @@ function M.create(ctx)
function()
process.check_status()
end,
function()
mp.commandv("script-message", "subminer-stats-toggle")
end,
}
input.select({

View File

@@ -0,0 +1,68 @@
import assert from 'node:assert/strict';
import { existsSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import test from 'node:test';
const repoRoot = process.cwd();
function read(relativePath: string): string {
return readFileSync(join(repoRoot, relativePath), 'utf8');
}
const requiredDocs = [
'docs/README.md',
'docs/architecture/README.md',
'docs/architecture/domains.md',
'docs/architecture/layering.md',
'docs/knowledge-base/README.md',
'docs/knowledge-base/core-beliefs.md',
'docs/knowledge-base/catalog.md',
'docs/knowledge-base/quality.md',
'docs/workflow/README.md',
'docs/workflow/planning.md',
'docs/workflow/verification.md',
] as const;
const metadataFields = ['Status:', 'Last verified:', 'Owner:', 'Read when:'] as const;
test('required internal knowledge-base docs exist', () => {
for (const relativePath of requiredDocs) {
assert.equal(existsSync(join(repoRoot, relativePath)), true, `${relativePath} should exist`);
}
});
test('core internal docs include metadata fields', () => {
for (const relativePath of requiredDocs) {
const contents = read(relativePath);
for (const field of metadataFields) {
assert.match(contents, new RegExp(field.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')));
}
}
});
test('AGENTS.md is a compact map to internal docs', () => {
const agentsContents = read('AGENTS.md');
const lineCount = agentsContents.trimEnd().split('\n').length;
assert.ok(lineCount <= 110, `AGENTS.md should stay compact; got ${lineCount} lines`);
assert.match(agentsContents, /\.\/docs\/README\.md/);
assert.match(agentsContents, /\.\/docs\/architecture\/README\.md/);
assert.match(agentsContents, /\.\/docs\/workflow\/README\.md/);
assert.match(agentsContents, /\.\/docs\/workflow\/verification\.md/);
assert.match(agentsContents, /\.\/docs\/knowledge-base\/README\.md/);
assert.match(agentsContents, /\.\/docs\/RELEASING\.md/);
assert.match(agentsContents, /`docs-site\/` is user-facing/);
assert.doesNotMatch(agentsContents, /\.\/docs-site\/development\.md/);
assert.doesNotMatch(agentsContents, /\.\/docs-site\/architecture\.md/);
});
test('docs-site contributor docs point internal readers to docs/', () => {
const developmentContents = read('docs-site/development.md');
const architectureContents = read('docs-site/architecture.md');
const docsReadmeContents = read('docs-site/README.md');
assert.match(developmentContents, /docs\/README\.md/);
assert.match(developmentContents, /docs\/architecture\/README\.md/);
assert.match(architectureContents, /docs\/architecture\/README\.md/);
assert.match(docsReadmeContents, /docs\/README\.md/);
});

View File

@@ -137,6 +137,7 @@ export class AnkiIntegration {
private fieldGroupingWorkflow: FieldGroupingWorkflow;
private runtime: AnkiIntegrationRuntime;
private aiConfig: AiConfig;
private recordCardsMinedCallback: ((count: number, noteIds?: number[]) => void) | null = null;
constructor(
config: AnkiConnectConfig,
@@ -150,6 +151,7 @@ export class AnkiIntegration {
}) => Promise<KikuFieldGroupingChoice>,
knownWordCacheStatePath?: string,
aiConfig: AiConfig = {},
recordCardsMined?: (count: number, noteIds?: number[]) => void,
) {
this.config = normalizeAnkiIntegrationConfig(config);
this.aiConfig = { ...aiConfig };
@@ -160,6 +162,7 @@ export class AnkiIntegration {
this.osdCallback = osdCallback || null;
this.notificationCallback = notificationCallback || null;
this.fieldGroupingCallback = fieldGroupingCallback || null;
this.recordCardsMinedCallback = recordCardsMined ?? null;
this.knownWordCache = this.createKnownWordCache(knownWordCacheStatePath);
this.pollingRunner = this.createPollingRunner();
this.cardCreationService = this.createCardCreationService();
@@ -208,6 +211,9 @@ export class AnkiIntegration {
(await this.client.findNotes(query, options)) as number[],
shouldAutoUpdateNewCards: () => this.config.behavior?.autoUpdateNewCards !== false,
processNewCard: (noteId) => this.processNewCard(noteId),
recordCardsAdded: (count, noteIds) => {
this.recordCardsMinedCallback?.(count, noteIds);
},
isUpdateInProgress: () => this.updateInProgress,
setUpdateInProgress: (value) => {
this.updateInProgress = value;
@@ -229,6 +235,9 @@ export class AnkiIntegration {
return new AnkiConnectProxyServer({
shouldAutoUpdateNewCards: () => this.config.behavior?.autoUpdateNewCards !== false,
processNewCard: (noteId: number) => this.processNewCard(noteId),
recordCardsAdded: (count, noteIds) => {
this.recordCardsMinedCallback?.(count, noteIds);
},
getDeck: () => this.config.deck,
findNotes: async (query, options) =>
(await this.client.findNotes(query, options)) as number[],
@@ -1112,4 +1121,8 @@ export class AnkiIntegration {
this.stop();
this.mediaGenerator.cleanup();
}
setRecordCardsMinedCallback(callback: ((count: number, noteIds?: number[]) => void) | null): void {
this.recordCardsMinedCallback = callback;
}
}

View File

@@ -17,11 +17,15 @@ async function waitForCondition(
test('proxy enqueues addNote result for enrichment', async () => {
const processed: number[] = [];
const recordedCards: number[] = [];
const proxy = new AnkiConnectProxyServer({
shouldAutoUpdateNewCards: () => true,
processNewCard: async (noteId) => {
processed.push(noteId);
},
recordCardsAdded: (count) => {
recordedCards.push(count);
},
logInfo: () => undefined,
logWarn: () => undefined,
logError: () => undefined,
@@ -38,6 +42,7 @@ test('proxy enqueues addNote result for enrichment', async () => {
await waitForCondition(() => processed.length === 1);
assert.deepEqual(processed, [42]);
assert.deepEqual(recordedCards, [1]);
});
test('proxy enqueues addNote bare numeric response for enrichment', async () => {
@@ -64,12 +69,16 @@ test('proxy enqueues addNote bare numeric response for enrichment', async () =>
test('proxy de-duplicates addNotes IDs within the same response', async () => {
const processed: number[] = [];
const recordedCards: number[] = [];
const proxy = new AnkiConnectProxyServer({
shouldAutoUpdateNewCards: () => true,
processNewCard: async (noteId) => {
processed.push(noteId);
await new Promise((resolve) => setTimeout(resolve, 5));
},
recordCardsAdded: (count) => {
recordedCards.push(count);
},
logInfo: () => undefined,
logWarn: () => undefined,
logError: () => undefined,
@@ -86,6 +95,7 @@ test('proxy de-duplicates addNotes IDs within the same response', async () => {
await waitForCondition(() => processed.length === 2);
assert.deepEqual(processed, [101, 102]);
assert.deepEqual(recordedCards, [2]);
});
test('proxy enqueues note IDs from multi action addNote/addNotes results', async () => {
@@ -277,12 +287,16 @@ test('proxy does not fallback-enqueue latest note for multi requests without add
test('proxy fallback-enqueues latest note for addNote responses without note IDs and escapes deck quotes', async () => {
const processed: number[] = [];
const recordedCards: number[] = [];
const findNotesQueries: string[] = [];
const proxy = new AnkiConnectProxyServer({
shouldAutoUpdateNewCards: () => true,
processNewCard: async (noteId) => {
processed.push(noteId);
},
recordCardsAdded: (count) => {
recordedCards.push(count);
},
getDeck: () => 'My "Japanese" Deck',
findNotes: async (query) => {
findNotesQueries.push(query);
@@ -305,6 +319,7 @@ test('proxy fallback-enqueues latest note for addNote responses without note IDs
await waitForCondition(() => processed.length === 1);
assert.deepEqual(findNotesQueries, ['"deck:My \\"Japanese\\" Deck" added:1']);
assert.deepEqual(processed, [501]);
assert.deepEqual(recordedCards, [1]);
});
test('proxy detects self-referential loop configuration', () => {

View File

@@ -15,6 +15,7 @@ interface AnkiConnectEnvelope {
export interface AnkiConnectProxyServerDeps {
shouldAutoUpdateNewCards: () => boolean;
processNewCard: (noteId: number) => Promise<void>;
recordCardsAdded?: (count: number, noteIds: number[]) => void;
getDeck?: () => string | undefined;
findNotes?: (
query: string,
@@ -332,12 +333,14 @@ export class AnkiConnectProxyServer {
private enqueueNotes(noteIds: number[]): void {
let enqueuedCount = 0;
const acceptedIds: number[] = [];
for (const noteId of noteIds) {
if (this.pendingNoteIdSet.has(noteId) || this.inFlightNoteIds.has(noteId)) {
continue;
}
this.pendingNoteIds.push(noteId);
this.pendingNoteIdSet.add(noteId);
acceptedIds.push(noteId);
enqueuedCount += 1;
}
@@ -345,6 +348,7 @@ export class AnkiConnectProxyServer {
return;
}
this.deps.recordCardsAdded?.(enqueuedCount, acceptedIds);
this.deps.logInfo(`[anki-proxy] Enqueued ${enqueuedCount} note(s) for enrichment`);
this.processQueue();
}

View File

@@ -0,0 +1,35 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { PollingRunner } from './polling';
test('polling runner records newly added cards after initialization', async () => {
const recordedCards: number[] = [];
let tracked = new Set<number>();
const responses = [[10, 11], [10, 11, 12, 13]];
const runner = new PollingRunner({
getDeck: () => 'Mining',
getPollingRate: () => 250,
findNotes: async () => responses.shift() ?? [],
shouldAutoUpdateNewCards: () => true,
processNewCard: async () => undefined,
recordCardsAdded: (count) => {
recordedCards.push(count);
},
isUpdateInProgress: () => false,
setUpdateInProgress: () => undefined,
getTrackedNoteIds: () => tracked,
setTrackedNoteIds: (noteIds) => {
tracked = noteIds;
},
showStatusNotification: () => undefined,
logDebug: () => undefined,
logInfo: () => undefined,
logWarn: () => undefined,
});
await runner.pollOnce();
await runner.pollOnce();
assert.deepEqual(recordedCards, [2]);
});

View File

@@ -9,6 +9,7 @@ export interface PollingRunnerDeps {
) => Promise<number[]>;
shouldAutoUpdateNewCards: () => boolean;
processNewCard: (noteId: number) => Promise<void>;
recordCardsAdded?: (count: number, noteIds: number[]) => void;
isUpdateInProgress: () => boolean;
setUpdateInProgress: (value: boolean) => void;
getTrackedNoteIds: () => Set<number>;
@@ -80,6 +81,7 @@ export class PollingRunner {
previousNoteIds.add(noteId);
}
this.deps.setTrackedNoteIds(previousNoteIds);
this.deps.recordCardsAdded?.(newNoteIds.length, newNoteIds);
if (this.deps.shouldAutoUpdateNewCards()) {
for (const noteId of newNoteIds) {

View File

@@ -143,6 +143,12 @@ test('hasExplicitCommand and shouldStartApp preserve command intent', () => {
assert.equal(dictionaryTarget.dictionary, true);
assert.equal(dictionaryTarget.dictionaryTarget, '/tmp/example.mkv');
const stats = parseArgs(['--stats', '--stats-response-path', '/tmp/subminer-stats-response.json']);
assert.equal(stats.stats, true);
assert.equal(stats.statsResponsePath, '/tmp/subminer-stats-response.json');
assert.equal(hasExplicitCommand(stats), true);
assert.equal(shouldStartApp(stats), true);
const jellyfinLibraries = parseArgs(['--jellyfin-libraries']);
assert.equal(jellyfinLibraries.jellyfinLibraries, true);
assert.equal(hasExplicitCommand(jellyfinLibraries), true);

View File

@@ -29,6 +29,10 @@ export interface CliArgs {
anilistRetryQueue: boolean;
dictionary: boolean;
dictionaryTarget?: string;
stats: boolean;
statsCleanup?: boolean;
statsCleanupVocab?: boolean;
statsResponsePath?: string;
jellyfin: boolean;
jellyfinLogin: boolean;
jellyfinLogout: boolean;
@@ -97,6 +101,9 @@ export function parseArgs(argv: string[]): CliArgs {
anilistSetup: false,
anilistRetryQueue: false,
dictionary: false,
stats: false,
statsCleanup: false,
statsCleanupVocab: false,
jellyfin: false,
jellyfinLogin: false,
jellyfinLogout: false,
@@ -162,6 +169,15 @@ export function parseArgs(argv: string[]): CliArgs {
} else if (arg === '--dictionary-target') {
const value = readValue(argv[i + 1]);
if (value) args.dictionaryTarget = value;
} else if (arg === '--stats') args.stats = true;
else if (arg === '--stats-cleanup') args.statsCleanup = true;
else if (arg === '--stats-cleanup-vocab') args.statsCleanupVocab = true;
else if (arg.startsWith('--stats-response-path=')) {
const value = arg.split('=', 2)[1];
if (value) args.statsResponsePath = value;
} else if (arg === '--stats-response-path') {
const value = readValue(argv[i + 1]);
if (value) args.statsResponsePath = value;
} else if (arg === '--jellyfin') args.jellyfin = true;
else if (arg === '--jellyfin-login') args.jellyfinLogin = true;
else if (arg === '--jellyfin-logout') args.jellyfinLogout = true;
@@ -331,6 +347,7 @@ export function hasExplicitCommand(args: CliArgs): boolean {
args.anilistSetup ||
args.anilistRetryQueue ||
args.dictionary ||
args.stats ||
args.jellyfin ||
args.jellyfinLogin ||
args.jellyfinLogout ||
@@ -367,6 +384,7 @@ export function shouldStartApp(args: CliArgs): boolean {
args.markAudioCard ||
args.openRuntimeOptions ||
args.dictionary ||
args.stats ||
args.jellyfin ||
args.jellyfinPlay ||
args.texthooker
@@ -408,6 +426,7 @@ export function shouldRunSettingsOnlyStartup(args: CliArgs): boolean {
!args.anilistSetup &&
!args.anilistRetryQueue &&
!args.dictionary &&
!args.stats &&
!args.jellyfin &&
!args.jellyfinLogin &&
!args.jellyfinLogout &&

View File

@@ -18,6 +18,7 @@ test('printHelp includes configured texthooker port', () => {
assert.match(output, /--help\s+Show this help/);
assert.match(output, /default: 7777/);
assert.match(output, /--launch-mpv/);
assert.match(output, /--stats\s+Open the stats dashboard in your browser/);
assert.match(output, /--refresh-known-words/);
assert.match(output, /--setup\s+Open first-run setup window/);
assert.match(output, /--anilist-status/);

View File

@@ -14,6 +14,7 @@ ${B}Session${R}
--start Connect to mpv and launch overlay
--launch-mpv ${D}[targets...]${R} Launch mpv with the SubMiner mpv profile and exit
--stop Stop the running instance
--stats Open the stats dashboard in your browser
--texthooker Start texthooker server only ${D}(no overlay)${R}
${B}Overlay${R}

View File

@@ -2,10 +2,12 @@ import { RawConfig, ResolvedConfig } from '../types';
import { CORE_DEFAULT_CONFIG } from './definitions/defaults-core';
import { IMMERSION_DEFAULT_CONFIG } from './definitions/defaults-immersion';
import { INTEGRATIONS_DEFAULT_CONFIG } from './definitions/defaults-integrations';
import { STATS_DEFAULT_CONFIG } from './definitions/defaults-stats';
import { SUBTITLE_DEFAULT_CONFIG } from './definitions/defaults-subtitle';
import { buildCoreConfigOptionRegistry } from './definitions/options-core';
import { buildImmersionConfigOptionRegistry } from './definitions/options-immersion';
import { buildIntegrationConfigOptionRegistry } from './definitions/options-integrations';
import { buildStatsConfigOptionRegistry } from './definitions/options-stats';
import { buildSubtitleConfigOptionRegistry } from './definitions/options-subtitle';
import { buildRuntimeOptionRegistry } from './definitions/runtime-options';
import { CONFIG_TEMPLATE_SECTIONS } from './definitions/template-sections';
@@ -36,6 +38,7 @@ const { ankiConnect, jimaku, anilist, yomitan, jellyfin, discordPresence, ai, yo
INTEGRATIONS_DEFAULT_CONFIG;
const { subtitleStyle } = SUBTITLE_DEFAULT_CONFIG;
const { immersionTracking } = IMMERSION_DEFAULT_CONFIG;
const { stats } = STATS_DEFAULT_CONFIG;
export const DEFAULT_CONFIG: ResolvedConfig = {
subtitlePosition,
@@ -60,6 +63,7 @@ export const DEFAULT_CONFIG: ResolvedConfig = {
ai,
youtubeSubgen,
immersionTracking,
stats,
};
export const DEFAULT_ANKI_CONNECT_CONFIG = DEFAULT_CONFIG.ankiConnect;
@@ -71,6 +75,7 @@ export const CONFIG_OPTION_REGISTRY = [
...buildSubtitleConfigOptionRegistry(DEFAULT_CONFIG),
...buildIntegrationConfigOptionRegistry(DEFAULT_CONFIG, RUNTIME_OPTION_REGISTRY),
...buildImmersionConfigOptionRegistry(DEFAULT_CONFIG),
...buildStatsConfigOptionRegistry(DEFAULT_CONFIG),
];
export { CONFIG_TEMPLATE_SECTIONS };

View File

@@ -0,0 +1,10 @@
import { ResolvedConfig } from '../../types.js';
export const STATS_DEFAULT_CONFIG: Pick<ResolvedConfig, 'stats'> = {
stats: {
toggleKey: 'Backquote',
serverPort: 5175,
autoStartServer: true,
autoOpenBrowser: true,
},
};

View File

@@ -0,0 +1,33 @@
import { ResolvedConfig } from '../../types.js';
import { ConfigOptionRegistryEntry } from './shared.js';
export function buildStatsConfigOptionRegistry(
defaultConfig: ResolvedConfig,
): ConfigOptionRegistryEntry[] {
return [
{
path: 'stats.toggleKey',
kind: 'string',
defaultValue: defaultConfig.stats.toggleKey,
description: 'Key code to toggle the stats overlay.',
},
{
path: 'stats.serverPort',
kind: 'number',
defaultValue: defaultConfig.stats.serverPort,
description: 'Port for the stats HTTP server.',
},
{
path: 'stats.autoStartServer',
kind: 'boolean',
defaultValue: defaultConfig.stats.autoStartServer,
description: 'Automatically start the stats server on launch.',
},
{
path: 'stats.autoOpenBrowser',
kind: 'boolean',
defaultValue: defaultConfig.stats.autoOpenBrowser,
description: 'Automatically open the stats dashboard in a browser when the server starts.',
},
];
}

View File

@@ -176,6 +176,14 @@ const IMMERSION_TEMPLATE_SECTIONS: ConfigTemplateSection[] = [
],
key: 'immersionTracking',
},
{
title: 'Stats Dashboard',
description: [
'Local immersion stats dashboard served on localhost and available as an in-app overlay.',
'Uses the immersion tracking database for overview, trends, sessions, and vocabulary views.',
],
key: 'stats',
},
];
export const CONFIG_TEMPLATE_SECTIONS: ConfigTemplateSection[] = [

View File

@@ -4,6 +4,7 @@ import { createResolveContext } from './resolve/context';
import { applyCoreDomainConfig } from './resolve/core-domains';
import { applyImmersionTrackingConfig } from './resolve/immersion-tracking';
import { applyIntegrationConfig } from './resolve/integrations';
import { applyStatsConfig } from './resolve/stats';
import { applySubtitleDomainConfig } from './resolve/subtitle-domains';
import { applyTopLevelConfig } from './resolve/top-level';
@@ -13,6 +14,7 @@ const APPLY_RESOLVE_STEPS = [
applySubtitleDomainConfig,
applyIntegrationConfig,
applyImmersionTrackingConfig,
applyStatsConfig,
applyAnkiConnectResolution,
] as const;

View File

@@ -0,0 +1,36 @@
import { ResolveContext } from './context';
import { asBoolean, asNumber, asString, isObject } from './shared';
export function applyStatsConfig(context: ResolveContext): void {
const { src, resolved, warn } = context;
if (!isObject(src.stats)) return;
const toggleKey = asString(src.stats.toggleKey);
if (toggleKey !== undefined) {
resolved.stats.toggleKey = toggleKey;
} else if (src.stats.toggleKey !== undefined) {
warn('stats.toggleKey', src.stats.toggleKey, resolved.stats.toggleKey, 'Expected string.');
}
const serverPort = asNumber(src.stats.serverPort);
if (serverPort !== undefined) {
resolved.stats.serverPort = serverPort;
} else if (src.stats.serverPort !== undefined) {
warn('stats.serverPort', src.stats.serverPort, resolved.stats.serverPort, 'Expected number.');
}
const autoStartServer = asBoolean(src.stats.autoStartServer);
if (autoStartServer !== undefined) {
resolved.stats.autoStartServer = autoStartServer;
} else if (src.stats.autoStartServer !== undefined) {
warn('stats.autoStartServer', src.stats.autoStartServer, resolved.stats.autoStartServer, 'Expected boolean.');
}
const autoOpenBrowser = asBoolean(src.stats.autoOpenBrowser);
if (autoOpenBrowser !== undefined) {
resolved.stats.autoOpenBrowser = autoOpenBrowser;
} else if (src.stats.autoOpenBrowser !== undefined) {
warn('stats.autoOpenBrowser', src.stats.autoOpenBrowser, resolved.stats.autoOpenBrowser, 'Expected boolean.');
}
}

View File

@@ -0,0 +1,773 @@
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { createStatsApp } from '../stats-server.js';
import type { ImmersionTrackerService } from '../immersion-tracker-service.js';
const SESSION_SUMMARIES = [
{
sessionId: 1,
canonicalTitle: 'Test',
videoId: 1,
animeId: null,
animeTitle: null,
startedAtMs: Date.now(),
endedAtMs: null,
totalWatchedMs: 60_000,
activeWatchedMs: 50_000,
linesSeen: 10,
wordsSeen: 100,
tokensSeen: 80,
cardsMined: 2,
lookupCount: 5,
lookupHits: 4,
},
];
const DAILY_ROLLUPS = [
{
rollupDayOrMonth: Math.floor(Date.now() / 86_400_000),
videoId: 1,
totalSessions: 1,
totalActiveMin: 10,
totalLinesSeen: 10,
totalWordsSeen: 100,
totalTokensSeen: 80,
totalCards: 2,
cardsPerHour: 12,
wordsPerMin: 10,
lookupHitRate: 0.8,
},
];
const VOCABULARY_STATS = [
{
wordId: 1,
headword: 'する',
word: 'する',
reading: 'する',
partOfSpeech: 'verb',
pos1: '動詞',
pos2: '自立',
pos3: null,
frequency: 100,
firstSeen: Date.now(),
lastSeen: Date.now(),
},
];
const KANJI_STATS = [
{
kanjiId: 1,
kanji: '日',
frequency: 50,
firstSeen: Date.now(),
lastSeen: Date.now(),
},
];
const OCCURRENCES = [
{
animeId: 1,
animeTitle: 'Little Witch Academia',
videoId: 2,
videoTitle: 'Episode 4',
sessionId: 3,
lineIndex: 7,
segmentStartMs: 12_000,
segmentEndMs: 14_500,
text: '猫 猫 日 日 は 知っている',
occurrenceCount: 2,
},
];
const ANIME_LIBRARY = [
{
animeId: 1,
canonicalTitle: 'Little Witch Academia',
anilistId: 21858,
totalSessions: 3,
totalActiveMs: 180_000,
totalCards: 5,
totalWordsSeen: 300,
episodeCount: 2,
episodesTotal: 25,
lastWatchedMs: Date.now(),
},
];
const ANIME_DETAIL = {
animeId: 1,
canonicalTitle: 'Little Witch Academia',
anilistId: 21858,
titleRomaji: 'Little Witch Academia',
titleEnglish: 'Little Witch Academia',
titleNative: 'リトルウィッチアカデミア',
totalSessions: 3,
totalActiveMs: 180_000,
totalCards: 5,
totalWordsSeen: 300,
totalLinesSeen: 50,
totalLookupCount: 20,
totalLookupHits: 15,
episodeCount: 2,
lastWatchedMs: Date.now(),
};
const ANIME_WORDS = [
{
wordId: 1,
headword: '魔法',
word: '魔法',
reading: 'まほう',
partOfSpeech: 'noun',
frequency: 42,
},
];
const EPISODES_PER_DAY = [
{ epochDay: Math.floor(Date.now() / 86_400_000) - 1, episodeCount: 3 },
{ epochDay: Math.floor(Date.now() / 86_400_000), episodeCount: 1 },
];
const NEW_ANIME_PER_DAY = [
{ epochDay: Math.floor(Date.now() / 86_400_000) - 2, newAnimeCount: 2 },
];
const WATCH_TIME_PER_ANIME = [
{
epochDay: Math.floor(Date.now() / 86_400_000) - 1,
animeId: 1,
animeTitle: 'Little Witch Academia',
totalActiveMin: 25,
},
];
const ANIME_EPISODES = [
{
animeId: 1,
videoId: 1,
canonicalTitle: 'Episode 1',
parsedTitle: 'Little Witch Academia',
season: 1,
episode: 1,
totalSessions: 1,
totalActiveMs: 90_000,
totalCards: 3,
totalWordsSeen: 150,
lastWatchedMs: Date.now(),
},
];
const WORD_DETAIL = {
wordId: 1,
headword: '猫',
word: '猫',
reading: 'ねこ',
partOfSpeech: 'noun',
pos1: '名詞',
pos2: '一般',
pos3: null,
frequency: 42,
firstSeen: Date.now() - 100_000,
lastSeen: Date.now(),
};
const WORD_ANIME_APPEARANCES = [
{ animeId: 1, animeTitle: 'Little Witch Academia', occurrenceCount: 12 },
];
const SIMILAR_WORDS = [
{ wordId: 2, headword: '猫耳', word: '猫耳', reading: 'ねこみみ', frequency: 5 },
];
const KANJI_DETAIL = {
kanjiId: 1,
kanji: '日',
frequency: 50,
firstSeen: Date.now() - 100_000,
lastSeen: Date.now(),
};
const KANJI_ANIME_APPEARANCES = [
{ animeId: 1, animeTitle: 'Little Witch Academia', occurrenceCount: 30 },
];
const KANJI_WORDS = [
{ wordId: 3, headword: '日本', word: '日本', reading: 'にほん', frequency: 20 },
];
const EPISODE_CARD_EVENTS = [
{ eventId: 1, sessionId: 1, tsMs: Date.now(), cardsDelta: 1, noteIds: [12345] },
];
function createMockTracker(
overrides: Partial<ImmersionTrackerService> = {},
): ImmersionTrackerService {
return {
getSessionSummaries: async () => SESSION_SUMMARIES,
getDailyRollups: async () => DAILY_ROLLUPS,
getMonthlyRollups: async () => [],
getQueryHints: async () => ({ totalSessions: 5, activeSessions: 1, episodesToday: 2, activeAnimeCount: 3 }),
getSessionTimeline: async () => [],
getSessionEvents: async () => [],
getVocabularyStats: async () => VOCABULARY_STATS,
getKanjiStats: async () => KANJI_STATS,
getWordOccurrences: async () => OCCURRENCES,
getKanjiOccurrences: async () => OCCURRENCES,
getAnimeLibrary: async () => ANIME_LIBRARY,
getAnimeDetail: async (animeId: number) => (animeId === 1 ? ANIME_DETAIL : null),
getAnimeEpisodes: async () => ANIME_EPISODES,
getAnimeAnilistEntries: async () => [],
getAnimeWords: async () => ANIME_WORDS,
getAnimeDailyRollups: async () => DAILY_ROLLUPS,
getEpisodesPerDay: async () => EPISODES_PER_DAY,
getNewAnimePerDay: async () => NEW_ANIME_PER_DAY,
getWatchTimePerAnime: async () => WATCH_TIME_PER_ANIME,
getStreakCalendar: async () => [
{ epochDay: Math.floor(Date.now() / 86_400_000) - 1, totalActiveMin: 30 },
{ epochDay: Math.floor(Date.now() / 86_400_000), totalActiveMin: 45 },
],
getAnimeCoverArt: async (animeId: number) =>
animeId === 1
? {
videoId: 1,
anilistId: 21858,
coverUrl: 'https://example.com/cover.jpg',
coverBlob: Buffer.from([0xff, 0xd8, 0xff, 0xd9]),
titleRomaji: 'Little Witch Academia',
titleEnglish: 'Little Witch Academia',
episodesTotal: 25,
fetchedAtMs: Date.now(),
}
: null,
getWordDetail: async (wordId: number) => (wordId === 1 ? WORD_DETAIL : null),
getWordAnimeAppearances: async () => WORD_ANIME_APPEARANCES,
getSimilarWords: async () => SIMILAR_WORDS,
getKanjiDetail: async (kanjiId: number) => (kanjiId === 1 ? KANJI_DETAIL : null),
getKanjiAnimeAppearances: async () => KANJI_ANIME_APPEARANCES,
getKanjiWords: async () => KANJI_WORDS,
getEpisodeWords: async () => ANIME_WORDS,
getEpisodeSessions: async () => SESSION_SUMMARIES,
getEpisodeCardEvents: async () => EPISODE_CARD_EVENTS,
...overrides,
} as unknown as ImmersionTrackerService;
}
function withTempDir<T>(fn: (dir: string) => Promise<T> | T): Promise<T> | T {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-stats-server-test-'));
const result = fn(dir);
if (result instanceof Promise) {
return result.finally(() => {
fs.rmSync(dir, { recursive: true, force: true });
});
}
fs.rmSync(dir, { recursive: true, force: true });
return result;
}
describe('stats server API routes', () => {
it('GET /api/stats/overview returns overview data', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/overview');
assert.equal(res.status, 200);
assert.equal(res.headers.get('access-control-allow-origin'), null);
const body = await res.json();
assert.ok(body.sessions);
assert.ok(body.rollups);
assert.ok(body.hints);
assert.equal(body.hints.totalSessions, 5);
assert.equal(body.hints.activeSessions, 1);
assert.equal(body.hints.episodesToday, 2);
assert.equal(body.hints.activeAnimeCount, 3);
});
it('GET /api/stats/sessions returns session list', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/sessions?limit=5');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
});
it('GET /api/stats/vocabulary returns word frequency data', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].headword, 'する');
});
it('GET /api/stats/kanji returns kanji frequency data', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/kanji');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].kanji, '日');
});
it('GET /api/stats/streak-calendar returns streak calendar rows', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/streak-calendar');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body.length, 2);
assert.equal(body[0].totalActiveMin, 30);
assert.equal(body[1].totalActiveMin, 45);
});
it('GET /api/stats/streak-calendar clamps oversized days', async () => {
let seenDays = 0;
const app = createStatsApp(
createMockTracker({
getStreakCalendar: async (days?: number) => {
seenDays = days ?? 0;
return [];
},
}),
);
const res = await app.request('/api/stats/streak-calendar?days=999999');
assert.equal(res.status, 200);
assert.equal(seenDays, 365);
});
it('GET /api/stats/trends/episodes-per-day returns episode count rows', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/trends/episodes-per-day');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body.length, 2);
assert.equal(body[0].episodeCount, 3);
});
it('GET /api/stats/trends/episodes-per-day clamps oversized limits', async () => {
let seenLimit = 0;
const app = createStatsApp(
createMockTracker({
getEpisodesPerDay: async (limit?: number) => {
seenLimit = limit ?? 0;
return EPISODES_PER_DAY;
},
}),
);
const res = await app.request('/api/stats/trends/episodes-per-day?limit=999999');
assert.equal(res.status, 200);
assert.equal(seenLimit, 365);
});
it('GET /api/stats/trends/new-anime-per-day returns new anime rows', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/trends/new-anime-per-day');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body.length, 1);
assert.equal(body[0].newAnimeCount, 2);
});
it('GET /api/stats/trends/new-anime-per-day clamps oversized limits', async () => {
let seenLimit = 0;
const app = createStatsApp(
createMockTracker({
getNewAnimePerDay: async (limit?: number) => {
seenLimit = limit ?? 0;
return NEW_ANIME_PER_DAY;
},
}),
);
const res = await app.request('/api/stats/trends/new-anime-per-day?limit=999999');
assert.equal(res.status, 200);
assert.equal(seenLimit, 365);
});
it('GET /api/stats/trends/watch-time-per-anime returns watch time rows', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/trends/watch-time-per-anime');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body.length, 1);
assert.equal(body[0].animeTitle, 'Little Witch Academia');
assert.equal(body[0].totalActiveMin, 25);
});
it('GET /api/stats/trends/watch-time-per-anime clamps oversized limits', async () => {
let seenLimit = 0;
const app = createStatsApp(
createMockTracker({
getWatchTimePerAnime: async (limit?: number) => {
seenLimit = limit ?? 0;
return WATCH_TIME_PER_ANIME;
},
}),
);
const res = await app.request('/api/stats/trends/watch-time-per-anime?limit=999999');
assert.equal(res.status, 200);
assert.equal(seenLimit, 365);
});
it('GET /api/stats/vocabulary/occurrences returns recent occurrence rows for a word', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getWordOccurrences: async (...args: unknown[]) => {
seenArgs = args;
return OCCURRENCES;
},
}),
);
const res = await app.request(
'/api/stats/vocabulary/occurrences?headword=%E7%8C%AB&word=%E7%8C%AB&reading=%E3%81%AD%E3%81%93&limit=999999&offset=25',
);
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].animeTitle, 'Little Witch Academia');
assert.deepEqual(seenArgs, ['猫', '猫', 'ねこ', 500, 25]);
});
it('GET /api/stats/kanji/occurrences returns recent occurrence rows for a kanji', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getKanjiOccurrences: async (...args: unknown[]) => {
seenArgs = args;
return OCCURRENCES;
},
}),
);
const res = await app.request('/api/stats/kanji/occurrences?kanji=%E6%97%A5&limit=999999&offset=10');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].occurrenceCount, 2);
assert.deepEqual(seenArgs, ['日', 500, 10]);
});
it('GET /api/stats/vocabulary/occurrences rejects missing required params', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary/occurrences?headword=%E7%8C%AB');
assert.equal(res.status, 400);
});
it('GET /api/stats/vocabulary clamps oversized limits', async () => {
let seenLimit = 0;
const app = createStatsApp(
createMockTracker({
getVocabularyStats: async (limit?: number, _excludePos?: string[]) => {
seenLimit = limit ?? 0;
return VOCABULARY_STATS;
},
}),
);
const res = await app.request('/api/stats/vocabulary?limit=999999');
assert.equal(res.status, 200);
assert.equal(seenLimit, 500);
});
it('GET /api/stats/vocabulary passes excludePos to tracker', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getVocabularyStats: async (...args: unknown[]) => {
seenArgs = args;
return VOCABULARY_STATS;
},
}),
);
const res = await app.request('/api/stats/vocabulary?excludePos=particle,auxiliary');
assert.equal(res.status, 200);
assert.deepEqual(seenArgs, [100, ['particle', 'auxiliary']]);
});
it('GET /api/stats/vocabulary returns POS fields', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary');
assert.equal(res.status, 200);
const body = await res.json();
assert.equal(body[0].partOfSpeech, 'verb');
assert.equal(body[0].pos1, '動詞');
assert.equal(body[0].pos2, '自立');
assert.equal(body[0].pos3, null);
});
it('GET /api/stats/anime returns anime library', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].canonicalTitle, 'Little Witch Academia');
});
it('GET /api/stats/anime/:animeId returns anime detail with episodes', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/1');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(body.detail);
assert.equal(body.detail.canonicalTitle, 'Little Witch Academia');
assert.ok(Array.isArray(body.episodes));
assert.equal(body.episodes[0].videoId, 1);
});
it('GET /api/stats/anime/:animeId returns 404 for missing anime', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/99999');
assert.equal(res.status, 404);
});
it('GET /api/stats/anime/:animeId/cover returns cover art', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/1/cover');
assert.equal(res.status, 200);
assert.equal(res.headers.get('content-type'), 'image/jpeg');
assert.equal(res.headers.get('cache-control'), 'public, max-age=86400');
});
it('GET /api/stats/anime/:animeId/cover returns 404 for missing anime', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/99999/cover');
assert.equal(res.status, 404);
});
it('GET /api/stats/anime/:animeId/words returns top words for an anime', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getAnimeWords: async (...args: unknown[]) => {
seenArgs = args;
return ANIME_WORDS;
},
}),
);
const res = await app.request('/api/stats/anime/1/words?limit=25');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].headword, '魔法');
assert.deepEqual(seenArgs, [1, 25]);
});
it('GET /api/stats/anime/:animeId/words rejects invalid animeId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/0/words');
assert.equal(res.status, 400);
});
it('GET /api/stats/anime/:animeId/words clamps oversized limits', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getAnimeWords: async (...args: unknown[]) => {
seenArgs = args;
return ANIME_WORDS;
},
}),
);
const res = await app.request('/api/stats/anime/1/words?limit=999999');
assert.equal(res.status, 200);
assert.deepEqual(seenArgs, [1, 200]);
});
it('GET /api/stats/anime/:animeId/rollups returns daily rollups for an anime', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getAnimeDailyRollups: async (...args: unknown[]) => {
seenArgs = args;
return DAILY_ROLLUPS;
},
}),
);
const res = await app.request('/api/stats/anime/1/rollups?limit=30');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
assert.equal(body[0].totalSessions, 1);
assert.deepEqual(seenArgs, [1, 30]);
});
it('GET /api/stats/anime/:animeId/rollups rejects invalid animeId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anime/-1/rollups');
assert.equal(res.status, 400);
});
it('GET /api/stats/anime/:animeId/rollups clamps oversized limits', async () => {
let seenArgs: unknown[] = [];
const app = createStatsApp(
createMockTracker({
getAnimeDailyRollups: async (...args: unknown[]) => {
seenArgs = args;
return DAILY_ROLLUPS;
},
}),
);
const res = await app.request('/api/stats/anime/1/rollups?limit=999999');
assert.equal(res.status, 200);
assert.deepEqual(seenArgs, [1, 365]);
});
it('GET /api/stats/vocabulary/:wordId/detail returns word detail', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary/1/detail');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(body.detail);
assert.equal(body.detail.headword, '猫');
assert.equal(body.detail.wordId, 1);
assert.ok(Array.isArray(body.animeAppearances));
assert.equal(body.animeAppearances[0].animeTitle, 'Little Witch Academia');
assert.ok(Array.isArray(body.similarWords));
assert.equal(body.similarWords[0].headword, '猫耳');
});
it('GET /api/stats/vocabulary/:wordId/detail returns 404 for missing word', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary/99999/detail');
assert.equal(res.status, 404);
});
it('GET /api/stats/vocabulary/:wordId/detail returns 400 for invalid wordId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/vocabulary/0/detail');
assert.equal(res.status, 400);
});
it('GET /api/stats/kanji/:kanjiId/detail returns kanji detail', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/kanji/1/detail');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(body.detail);
assert.equal(body.detail.kanji, '日');
assert.equal(body.detail.kanjiId, 1);
assert.ok(Array.isArray(body.animeAppearances));
assert.equal(body.animeAppearances[0].animeTitle, 'Little Witch Academia');
assert.ok(Array.isArray(body.words));
assert.equal(body.words[0].headword, '日本');
});
it('GET /api/stats/kanji/:kanjiId/detail returns 404 for missing kanji', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/kanji/99999/detail');
assert.equal(res.status, 404);
});
it('GET /api/stats/kanji/:kanjiId/detail returns 400 for invalid kanjiId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/kanji/0/detail');
assert.equal(res.status, 400);
});
it('GET /api/stats/vocabulary/occurrences still works with detail routes present', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request(
'/api/stats/vocabulary/occurrences?headword=%E7%8C%AB&word=%E7%8C%AB&reading=%E3%81%AD%E3%81%93',
);
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
});
it('GET /api/stats/kanji/occurrences still works with detail routes present', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/kanji/occurrences?kanji=%E6%97%A5');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body));
});
it('GET /api/stats/episode/:videoId/detail returns episode detail', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/episode/1/detail');
assert.equal(res.status, 200);
const body = await res.json();
assert.ok(Array.isArray(body.sessions));
assert.ok(Array.isArray(body.words));
assert.ok(Array.isArray(body.cardEvents));
assert.equal(body.cardEvents[0].noteIds[0], 12345);
});
it('GET /api/stats/episode/:videoId/detail returns 400 for invalid videoId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/episode/0/detail');
assert.equal(res.status, 400);
});
it('POST /api/stats/anki/browse returns 400 for missing noteId', async () => {
const app = createStatsApp(createMockTracker());
const res = await app.request('/api/stats/anki/browse', { method: 'POST' });
assert.equal(res.status, 400);
});
it('serves stats index and asset files from absolute static dir paths', async () => {
await withTempDir(async (dir) => {
const assetDir = path.join(dir, 'assets');
fs.mkdirSync(assetDir, { recursive: true });
fs.writeFileSync(
path.join(dir, 'index.html'),
'<!doctype html><html><body><div id="root"></div><script src="./assets/app.js"></script></body></html>',
);
fs.writeFileSync(path.join(assetDir, 'app.js'), 'console.log("stats ok");');
const app = createStatsApp(createMockTracker(), { staticDir: dir });
const indexRes = await app.request('/');
assert.equal(indexRes.status, 200);
assert.match(await indexRes.text(), /assets\/app\.js/);
const assetRes = await app.request('/assets/app.js');
assert.equal(assetRes.status, 200);
assert.equal(assetRes.headers.get('content-type'), 'text/javascript; charset=utf-8');
assert.match(await assetRes.text(), /stats ok/);
});
});
it('fetches and serves missing cover art on demand', async () => {
let ensureCalls = 0;
let hasCover = false;
const app = createStatsApp(
createMockTracker({
getCoverArt: async () =>
hasCover
? {
videoId: 1,
anilistId: 1,
coverUrl: 'https://example.com/cover.jpg',
coverBlob: Buffer.from([0xff, 0xd8, 0xff, 0xd9]),
titleRomaji: 'Test',
titleEnglish: 'Test',
episodesTotal: 12,
fetchedAtMs: Date.now(),
}
: null,
ensureCoverArt: async () => {
ensureCalls += 1;
hasCover = true;
return true;
},
}),
);
const res = await app.request('/api/stats/media/1/cover');
assert.equal(res.status, 200);
assert.equal(res.headers.get('content-type'), 'image/jpeg');
assert.equal(ensureCalls, 1);
});
});

View File

@@ -16,6 +16,7 @@ test('guessAnilistMediaInfo uses guessit output when available', async () => {
});
assert.deepEqual(result, {
title: 'Guessit Title',
season: null,
episode: 7,
source: 'guessit',
});
@@ -29,6 +30,7 @@ test('guessAnilistMediaInfo falls back to parser when guessit fails', async () =
});
assert.deepEqual(result, {
title: 'My Anime',
season: 1,
episode: 3,
source: 'fallback',
});
@@ -52,6 +54,7 @@ test('guessAnilistMediaInfo uses basename for guessit input', async () => {
]);
assert.deepEqual(result, {
title: 'Rascal Does Not Dream of Bunny Girl Senpai',
season: null,
episode: 1,
source: 'guessit',
});
@@ -67,6 +70,7 @@ test('guessAnilistMediaInfo joins multi-part guessit titles', async () => {
});
assert.deepEqual(result, {
title: 'Rascal Does not Dream of Bunny Girl Senpai',
season: null,
episode: 1,
source: 'guessit',
});

View File

@@ -7,6 +7,7 @@ const ANILIST_GRAPHQL_URL = 'https://graphql.anilist.co';
export interface AnilistMediaGuess {
title: string;
season: number | null;
episode: number | null;
source: 'guessit' | 'fallback';
}
@@ -56,7 +57,7 @@ interface AnilistSaveEntryData {
};
}
function runGuessit(target: string): Promise<string> {
export function runGuessit(target: string): Promise<string> {
return new Promise((resolve, reject) => {
childProcess.execFile(
'guessit',
@@ -73,7 +74,7 @@ function runGuessit(target: string): Promise<string> {
});
}
type GuessAnilistMediaInfoDeps = {
export interface GuessAnilistMediaInfoDeps {
runGuessit: (target: string) => Promise<string>;
};
@@ -215,8 +216,9 @@ export async function guessAnilistMediaInfo(
const parsed = JSON.parse(stdout) as Record<string, unknown>;
const title = readGuessitTitle(parsed.title);
const episode = firstPositiveInteger(parsed.episode);
const season = firstPositiveInteger(parsed.season);
if (title) {
return { title, episode, source: 'guessit' };
return { title, season, episode, source: 'guessit' };
}
} catch {
// Ignore guessit failures and fall back to internal parser.
@@ -230,6 +232,7 @@ export async function guessAnilistMediaInfo(
}
return {
title: parsed.title.trim(),
season: parsed.season,
episode: parsed.episode,
source: 'fallback',
};

View File

@@ -0,0 +1,239 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { createCoverArtFetcher, stripFilenameTags } from './cover-art-fetcher.js';
import { Database } from '../immersion-tracker/sqlite.js';
import { ensureSchema, getOrCreateVideoRecord } from '../immersion-tracker/storage.js';
import { getCoverArt, upsertCoverArt } from '../immersion-tracker/query.js';
import { SOURCE_TYPE_LOCAL } from '../immersion-tracker/types.js';
function makeDbPath(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-cover-art-test-'));
return path.join(dir, 'immersion.sqlite');
}
function cleanupDbPath(dbPath: string): void {
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
}
test('stripFilenameTags normalizes common media-title formats', () => {
assert.equal(
stripFilenameTags('[Jellyfin/direct] The Eminence in Shadow S01E05 I Am...'),
'The Eminence in Shadow',
);
assert.equal(
stripFilenameTags(
'[Foxtrot] Kono Subarashii Sekai ni Shukufuku wo! S2 - 05: Servitude for this Masked Knight!',
),
'Kono Subarashii Sekai ni Shukufuku wo!',
);
assert.equal(
stripFilenameTags('Kono Subarashii Sekai ni Shukufuku wo! E03: A Panty Treasure'),
'Kono Subarashii Sekai ni Shukufuku wo!',
);
assert.equal(
stripFilenameTags(
'Little Witch Academia (2017) - S01E05 - 005 - Pact of the Dragon [Bluray-1080p][10bit][h265][FLAC 2.0][JA]-FumeiRaws.mkv',
),
'Little Witch Academia',
);
});
test('fetchIfMissing backfills a missing blob from an existing cover URL', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
ensureSchema(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cover-fetcher-test.mkv', {
canonicalTitle: 'Cover Fetcher Test',
sourcePath: '/tmp/cover-fetcher-test.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
upsertCoverArt(db, videoId, {
anilistId: 7,
coverUrl: 'https://images.test/cover.jpg',
coverBlob: null,
titleRomaji: 'Test Title',
titleEnglish: 'Test Title',
episodesTotal: 12,
});
const fetchCalls: string[] = [];
const originalFetch = globalThis.fetch;
globalThis.fetch = (async (input: RequestInfo | URL) => {
const url = String(input);
fetchCalls.push(url);
assert.equal(url, 'https://images.test/cover.jpg');
return new Response(new Uint8Array([1, 2, 3, 4]), {
status: 200,
headers: { 'Content-Type': 'image/jpeg' },
});
}) as typeof fetch;
try {
const fetcher = createCoverArtFetcher(
{
acquire: async () => {},
recordResponse: () => {},
},
console,
);
const fetched = await fetcher.fetchIfMissing(
db,
videoId,
'[Jellyfin] Little Witch Academia S02E05 - 025 - Pact of the Dragon (2020) [1080p].mkv',
);
const stored = getCoverArt(db, videoId);
assert.equal(fetched, true);
assert.equal(fetchCalls.length, 1);
assert.equal(stored?.coverBlob?.length, 4);
assert.equal(stored?.titleEnglish, 'Test Title');
} finally {
globalThis.fetch = originalFetch;
db.close();
cleanupDbPath(dbPath);
}
});
function createJsonResponse(payload: unknown): Response {
return new Response(JSON.stringify(payload), {
status: 200,
headers: { 'content-type': 'application/json' },
});
}
test('fetchIfMissing uses guessit primary title and season when available', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
ensureSchema(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cover-fetcher-season-test.mkv', {
canonicalTitle: '[Jellyfin] Little Witch Academia S02E05 - 025 - Pact of the Dragon (2020) [1080p].mkv',
sourcePath: '/tmp/cover-fetcher-season-test.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const searchCalls: Array<{ search: string }> = [];
const originalFetch = globalThis.fetch;
globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => {
const raw = (init?.body as string | undefined) ?? '';
const payload = JSON.parse(raw) as { variables: { search: string } };
const search = payload.variables.search;
searchCalls.push({ search });
if (search.includes('Season 2')) {
return Promise.resolve(createJsonResponse({ data: { Page: { media: [] } } }));
}
return Promise.resolve(
createJsonResponse({
data: {
Page: {
media: [
{
id: 19,
episodes: 24,
coverImage: { large: 'https://images.test/cover.jpg', medium: null },
title: { romaji: 'Little Witch Academia', english: 'Little Witch Academia', native: null },
},
],
},
},
}),
);
}) as typeof fetch;
try {
const fetcher = createCoverArtFetcher(
{
acquire: async () => {},
recordResponse: () => {},
},
console,
{
runGuessit: async () =>
JSON.stringify({ title: 'Little Witch Academia', season: 2, episode: 5 }),
},
);
const fetched = await fetcher.fetchIfMissing(db, videoId, 'School Vlog S01E01');
const stored = getCoverArt(db, videoId);
assert.equal(fetched, true);
assert.equal(searchCalls.length, 2);
assert.equal(searchCalls[0]!.search, 'Little Witch Academia Season 2');
assert.equal(stored?.anilistId, 19);
} finally {
globalThis.fetch = originalFetch;
db.close();
cleanupDbPath(dbPath);
}
});
test('fetchIfMissing falls back to internal parser when guessit throws', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
ensureSchema(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cover-fetcher-fallback-test.mkv', {
canonicalTitle: 'School Vlog S01E01',
sourcePath: '/tmp/cover-fetcher-fallback-test.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
let requestCount = 0;
const originalFetch = globalThis.fetch;
globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => {
requestCount += 1;
const raw = (init?.body as string | undefined) ?? '';
const payload = JSON.parse(raw) as { variables: { search: string } };
assert.equal(payload.variables.search, 'School Vlog');
return Promise.resolve(
createJsonResponse({
data: {
Page: {
media: [
{
id: 21,
episodes: 12,
coverImage: { large: 'https://images.test/fallback-cover.jpg', medium: null },
title: { romaji: 'School Vlog', english: 'School Vlog', native: null },
},
],
},
},
}),
);
}) as typeof fetch;
try {
const fetcher = createCoverArtFetcher(
{
acquire: async () => {},
recordResponse: () => {},
},
console,
{
runGuessit: async () => {
throw new Error('guessit unavailable');
},
},
);
const fetched = await fetcher.fetchIfMissing(db, videoId, 'Ignored Title');
const stored = getCoverArt(db, videoId);
assert.equal(fetched, true);
assert.equal(requestCount, 1);
assert.equal(stored?.anilistId, 21);
} finally {
globalThis.fetch = originalFetch;
db.close();
cleanupDbPath(dbPath);
}
});

View File

@@ -0,0 +1,405 @@
import type { AnilistRateLimiter } from './rate-limiter';
import type { DatabaseSync } from '../immersion-tracker/sqlite';
import { getCoverArt, upsertCoverArt, updateAnimeAnilistInfo } from '../immersion-tracker/query';
import { guessAnilistMediaInfo, runGuessit, type GuessAnilistMediaInfoDeps } from './anilist-updater';
const ANILIST_GRAPHQL_URL = 'https://graphql.anilist.co';
const NO_MATCH_RETRY_MS = 5 * 60 * 1000;
const SEARCH_QUERY = `
query ($search: String!) {
Page(perPage: 5) {
media(search: $search, type: ANIME) {
id
episodes
season
seasonYear
coverImage { large medium }
title { romaji english native }
}
}
}
`;
interface AnilistMedia {
id: number;
episodes: number | null;
season: string | null;
seasonYear: number | null;
coverImage: { large: string | null; medium: string | null } | null;
title: { romaji: string | null; english: string | null; native: string | null } | null;
}
interface AnilistSearchResponse {
data?: {
Page?: {
media?: AnilistMedia[];
};
};
errors?: Array<{ message?: string }>;
}
export interface CoverArtFetcher {
fetchIfMissing(db: DatabaseSync, videoId: number, canonicalTitle: string): Promise<boolean>;
}
interface Logger {
info(msg: string, ...args: unknown[]): void;
warn(msg: string, ...args: unknown[]): void;
error(msg: string, ...args: unknown[]): void;
}
interface CoverArtCandidate {
title: string;
source: 'guessit' | 'fallback';
season: number | null;
episode: number | null;
}
interface CoverArtFetcherOptions {
runGuessit?: GuessAnilistMediaInfoDeps['runGuessit'];
}
export function stripFilenameTags(raw: string): string {
let title = raw.replace(/\.[A-Za-z0-9]{2,4}$/, '');
title = title.replace(/^(?:\s*\[[^\]]*\]\s*)+/, '');
title = title.replace(/[._]+/g, ' ');
// Remove everything from " - S##E##" or " - ###" onward (season/episode markers)
title = title.replace(/\s+-\s+S\d+E\d+.*$/i, '');
title = title.replace(/\s+-\s+\d{2,}(\s+-\s+\d+)?(\s+-.+)?$/, '');
title = title.replace(/\s+S\d+E\d+.*$/i, '');
title = title.replace(/\s+S\d+\s*[- ]\s*\d+[: -].*$/i, '');
title = title.replace(/\s+E\d+[: -].*$/i, '');
title = title.replace(/^S\d+E\d+\s*[- ]\s*/i, '');
// Remove bracketed/parenthesized tags: [WEBDL-1080p], (2022), etc.
title = title.replace(/\s*\[[^\]]*\]\s*/g, ' ');
title = title.replace(/\s*\([^)]*\d{4}[^)]*\)\s*/g, ' ');
// Remove common codec/source tags that may appear without brackets
title = title.replace(
/\b(WEBDL|WEBRip|BluRay|BDRip|HDTV|DVDRip|x264|x265|H\.?264|H\.?265|AV1|AAC|FLAC|Opus|10bit|8bit|1080p|720p|480p|2160p|4K)\b[-.\w]*/gi,
'',
);
// Remove trailing dashes and group tags like "-Retr0"
title = title.replace(/\s*-\s*[\w]+$/, '');
return title.trim().replace(/\s{2,}/g, ' ');
}
function removeSeasonHint(title: string): string {
return title.replace(/\bseason\s*\d+\b/gi, '').replace(/\s{2,}/g, ' ').trim();
}
function normalizeTitle(text: string): string {
return text.trim().toLowerCase().replace(/\s+/g, ' ');
}
function extractCandidateSeasonHints(text: string): Set<number> {
const normalized = normalizeTitle(text);
const matches = [
...normalized.matchAll(/\bseason\s*(\d{1,2})\b/gi),
...normalized.matchAll(/\bs(\d{1,2})(?:\b|\D)/gi),
];
const values = new Set<number>();
for (const match of matches) {
const value = Number.parseInt(match[1]!, 10);
if (Number.isInteger(value)) {
values.add(value);
}
}
return values;
}
function isSeasonMentioned(titles: string[], season: number | null): boolean {
if (!season) {
return false;
}
const hints = titles.flatMap((title) => [...extractCandidateSeasonHints(title)]);
return hints.includes(season);
}
function pickBestSearchResult(
title: string,
episode: number | null,
season: number | null,
media: AnilistMedia[],
): { id: number; title: string } | null {
const cleanedTitle = removeSeasonHint(title);
const targets = [title, cleanedTitle]
.map(normalizeTitle)
.map((value) => value.trim())
.filter((value, index, all) => value.length > 0 && all.indexOf(value) === index);
const filtered = episode === null
? media
: media.filter((item) => {
const total = item.episodes;
return total === null || total >= episode;
});
const candidates = filtered.length > 0 ? filtered : media;
if (candidates.length === 0) {
return null;
}
const scored = candidates.map((item) => {
const candidateTitles = [
item.title?.romaji,
item.title?.english,
item.title?.native,
]
.filter((value): value is string => typeof value === 'string')
.map((value) => normalizeTitle(value));
let score = 0;
for (const target of targets) {
if (candidateTitles.includes(target)) {
score += 120;
continue;
}
if (candidateTitles.some((itemTitle) => itemTitle.includes(target))) {
score += 30;
}
if (candidateTitles.some((itemTitle) => target.includes(itemTitle))) {
score += 10;
}
}
if (episode !== null && item.episodes === episode) {
score += 20;
}
if (season !== null && isSeasonMentioned(candidateTitles, season)) {
score += 15;
}
return { item, score };
});
scored.sort((a, b) => {
if (b.score !== a.score) return b.score - a.score;
return b.item.id - a.item.id;
});
const selected = scored[0]!;
const selectedTitle = selected.item.title?.english ?? selected.item.title?.romaji ?? selected.item.title?.native ?? title;
return { id: selected.item.id, title: selectedTitle };
}
function buildSearchCandidates(parsed: CoverArtCandidate): string[] {
const candidateTitles = [
parsed.title,
...(parsed.source === 'guessit' && parsed.season !== null && parsed.season > 1
? [`${parsed.title} Season ${parsed.season}`]
: []),
];
return candidateTitles
.map((title) => title.trim())
.filter((title, index, all) => title.length > 0 && all.indexOf(title) === index);
}
async function searchAnilist(
rateLimiter: AnilistRateLimiter,
title: string,
): Promise<{ media: AnilistMedia[]; rateLimited: boolean }> {
await rateLimiter.acquire();
const res = await fetch(ANILIST_GRAPHQL_URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ query: SEARCH_QUERY, variables: { search: title } }),
});
rateLimiter.recordResponse(res.headers);
if (res.status === 429) {
return { media: [], rateLimited: true };
}
if (!res.ok) {
throw new Error(`Anilist search failed: ${res.status} ${res.statusText}`);
}
const json = (await res.json()) as AnilistSearchResponse;
const mediaList = json.data?.Page?.media;
if (!mediaList || mediaList.length === 0) {
return { media: [], rateLimited: false };
}
return { media: mediaList, rateLimited: false };
}
async function downloadImage(url: string): Promise<Buffer | null> {
try {
const res = await fetch(url);
if (!res.ok) return null;
const arrayBuf = await res.arrayBuffer();
return Buffer.from(arrayBuf);
} catch {
return null;
}
}
export function createCoverArtFetcher(
rateLimiter: AnilistRateLimiter,
logger: Logger,
options: CoverArtFetcherOptions = {},
): CoverArtFetcher {
const resolveMediaInfo = async (canonicalTitle: string): Promise<CoverArtCandidate | null> => {
const parsed = await guessAnilistMediaInfo(null, canonicalTitle, {
runGuessit: options.runGuessit ?? runGuessit,
});
if (!parsed) {
return null;
}
return {
title: parsed.title,
season: parsed.season,
episode: parsed.episode,
source: parsed.source,
};
};
return {
async fetchIfMissing(db, videoId, canonicalTitle): Promise<boolean> {
const existing = getCoverArt(db, videoId);
if (existing?.coverBlob) {
return true;
}
if (existing?.coverUrl) {
const coverBlob = await downloadImage(existing.coverUrl);
if (coverBlob) {
upsertCoverArt(db, videoId, {
anilistId: existing.anilistId,
coverUrl: existing.coverUrl,
coverBlob,
titleRomaji: existing.titleRomaji,
titleEnglish: existing.titleEnglish,
episodesTotal: existing.episodesTotal,
});
return true;
}
}
if (
existing &&
existing.coverUrl === null &&
existing.anilistId === null &&
Date.now() - existing.fetchedAtMs < NO_MATCH_RETRY_MS
) {
return false;
}
const cleaned = stripFilenameTags(canonicalTitle);
if (!cleaned) {
logger.warn('cover-art: empty title after stripping tags for videoId=%d', videoId);
upsertCoverArt(db, videoId, {
anilistId: null,
coverUrl: null,
coverBlob: null,
titleRomaji: null,
titleEnglish: null,
episodesTotal: null,
});
return false;
}
const parsedInfo = await resolveMediaInfo(canonicalTitle);
const searchBase = parsedInfo?.title ?? cleaned;
const searchCandidates = parsedInfo
? buildSearchCandidates(parsedInfo)
: [cleaned];
const effectiveCandidates = searchCandidates.includes(cleaned)
? searchCandidates
: [...searchCandidates, cleaned];
let selected: AnilistMedia | null = null;
let rateLimited = false;
for (const candidate of effectiveCandidates) {
logger.info('cover-art: searching Anilist for "%s" (videoId=%d)', candidate, videoId);
try {
const result = await searchAnilist(rateLimiter, candidate);
rateLimited = result.rateLimited;
if (result.media.length === 0) {
continue;
}
const picked = pickBestSearchResult(
searchBase,
parsedInfo?.episode ?? null,
parsedInfo?.season ?? null,
result.media,
);
if (picked) {
const match = result.media.find((media) => media.id === picked.id);
if (match) {
selected = match;
break;
}
}
} catch (err) {
logger.error('cover-art: Anilist search error for "%s": %s', candidate, err);
return false;
}
}
if (rateLimited) {
logger.warn('cover-art: rate-limited by Anilist, skipping videoId=%d', videoId);
return false;
}
if (!selected) {
logger.info('cover-art: no Anilist results for "%s", caching no-match', searchBase);
upsertCoverArt(db, videoId, {
anilistId: null,
coverUrl: null,
coverBlob: null,
titleRomaji: null,
titleEnglish: null,
episodesTotal: null,
});
return false;
}
const coverUrl = selected.coverImage?.large ?? selected.coverImage?.medium ?? null;
let coverBlob: Buffer | null = null;
if (coverUrl) {
coverBlob = await downloadImage(coverUrl);
}
upsertCoverArt(db, videoId, {
anilistId: selected.id,
coverUrl,
coverBlob,
titleRomaji: selected.title?.romaji ?? null,
titleEnglish: selected.title?.english ?? null,
episodesTotal: selected.episodes ?? null,
});
updateAnimeAnilistInfo(db, videoId, {
anilistId: selected.id,
titleRomaji: selected.title?.romaji ?? null,
titleEnglish: selected.title?.english ?? null,
titleNative: selected.title?.native ?? null,
episodesTotal: selected.episodes ?? null,
});
logger.info(
'cover-art: cached art for videoId=%d anilistId=%d title="%s"',
videoId,
selected.id,
selected.title?.romaji ?? searchBase,
);
return true;
},
};
}

View File

@@ -0,0 +1,72 @@
const DEFAULT_MAX_PER_MINUTE = 20;
const WINDOW_MS = 60_000;
const SAFETY_REMAINING_THRESHOLD = 5;
export interface AnilistRateLimiter {
acquire(): Promise<void>;
recordResponse(headers: Headers): void;
}
export function createAnilistRateLimiter(
maxPerMinute = DEFAULT_MAX_PER_MINUTE,
): AnilistRateLimiter {
const timestamps: number[] = [];
let pauseUntilMs = 0;
function pruneOld(now: number): void {
const cutoff = now - WINDOW_MS;
while (timestamps.length > 0 && timestamps[0]! < cutoff) {
timestamps.shift();
}
}
return {
async acquire(): Promise<void> {
const now = Date.now();
if (now < pauseUntilMs) {
const waitMs = pauseUntilMs - now;
await new Promise((resolve) => setTimeout(resolve, waitMs));
}
pruneOld(Date.now());
if (timestamps.length >= maxPerMinute) {
const oldest = timestamps[0]!;
const waitMs = oldest + WINDOW_MS - Date.now() + 100;
if (waitMs > 0) {
await new Promise((resolve) => setTimeout(resolve, waitMs));
}
pruneOld(Date.now());
}
timestamps.push(Date.now());
},
recordResponse(headers: Headers): void {
const remaining = headers.get('x-ratelimit-remaining');
if (remaining !== null) {
const n = parseInt(remaining, 10);
if (Number.isFinite(n) && n < SAFETY_REMAINING_THRESHOLD) {
const reset = headers.get('x-ratelimit-reset');
if (reset) {
const resetMs = parseInt(reset, 10) * 1000;
if (Number.isFinite(resetMs)) {
pauseUntilMs = Math.max(pauseUntilMs, resetMs);
}
} else {
pauseUntilMs = Math.max(pauseUntilMs, Date.now() + WINDOW_MS);
}
}
}
const retryAfter = headers.get('retry-after');
if (retryAfter) {
const seconds = parseInt(retryAfter, 10);
if (Number.isFinite(seconds) && seconds > 0) {
pauseUntilMs = Math.max(pauseUntilMs, Date.now() + seconds * 1000);
}
}
},
};
}

View File

@@ -34,6 +34,7 @@ function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
anilistSetup: false,
anilistRetryQueue: false,
dictionary: false,
stats: false,
jellyfin: false,
jellyfinLogin: false,
jellyfinLogout: false,

View File

@@ -34,6 +34,7 @@ function makeArgs(overrides: Partial<CliArgs> = {}): CliArgs {
anilistSetup: false,
anilistRetryQueue: false,
dictionary: false,
stats: false,
jellyfin: false,
jellyfinLogin: false,
jellyfinLogout: false,
@@ -177,6 +178,9 @@ function createDeps(overrides: Partial<CliCommandServiceDeps> = {}) {
mediaTitle: 'Test',
entryCount: 10,
}),
runStatsCommand: async () => {
calls.push('runStatsCommand');
},
runJellyfinCommand: async () => {
calls.push('runJellyfinCommand');
},
@@ -249,6 +253,21 @@ test('handleCliCommand opens first-run setup window for --setup', () => {
assert.equal(calls.includes('openYomitanSettingsDelayed:1000'), false);
});
test('handleCliCommand dispatches stats command without overlay startup', async () => {
const { deps, calls } = createDeps({
runStatsCommand: async () => {
calls.push('runStatsCommand');
},
});
handleCliCommand(makeArgs({ stats: true }), 'initial', deps);
await Promise.resolve();
assert.ok(calls.includes('runStatsCommand'));
assert.equal(calls.includes('initializeOverlayRuntime'), false);
assert.equal(calls.includes('connectMpvClient'), false);
});
test('handleCliCommand applies cli log level for second-instance commands', () => {
const { deps, calls } = createDeps({
setLogLevel: (level) => {

View File

@@ -61,6 +61,7 @@ export interface CliCommandServiceDeps {
mediaTitle: string;
entryCount: number;
}>;
runStatsCommand: (args: CliArgs, source: CliCommandSource) => Promise<void>;
runJellyfinCommand: (args: CliArgs) => Promise<void>;
printHelp: () => void;
hasMainWindow: () => boolean;
@@ -154,6 +155,7 @@ export interface CliCommandDepsRuntimeOptions {
};
jellyfin: {
openSetup: () => void;
runStatsCommand: (args: CliArgs, source: CliCommandSource) => Promise<void>;
runCommand: (args: CliArgs) => Promise<void>;
};
ui: UiCliRuntime;
@@ -222,6 +224,7 @@ export function createCliCommandDepsRuntime(
getAnilistQueueStatus: options.anilist.getQueueStatus,
retryAnilistQueue: options.anilist.retryQueueNow,
generateCharacterDictionary: options.dictionary.generate,
runStatsCommand: options.jellyfin.runStatsCommand,
runJellyfinCommand: options.jellyfin.runCommand,
printHelp: options.ui.printHelp,
hasMainWindow: options.app.hasMainWindow,
@@ -410,6 +413,8 @@ export function handleCliCommand(
deps.stopApp();
}
});
} else if (args.stats) {
void deps.runStatsCommand(args, source);
} else if (args.anilistRetryQueue) {
const queueStatus = deps.getAnilistQueueStatus();
deps.log(

View File

@@ -12,6 +12,7 @@ import {
resolveBoundedInt,
} from './immersion-tracker/reducer';
import type { QueuedWrite } from './immersion-tracker/types';
import { PartOfSpeech, type MergedToken } from '../../types';
type ImmersionTrackerService = import('./immersion-tracker-service').ImmersionTrackerService;
type ImmersionTrackerServiceCtor =
@@ -26,6 +27,34 @@ async function loadTrackerCtor(): Promise<ImmersionTrackerServiceCtor> {
return trackerCtor;
}
async function waitForPendingAnimeMetadata(tracker: ImmersionTrackerService): Promise<void> {
const privateApi = tracker as unknown as {
sessionState: { videoId: number } | null;
pendingAnimeMetadataUpdates?: Map<number, Promise<void>>;
};
const videoId = privateApi.sessionState?.videoId;
if (!videoId) return;
await privateApi.pendingAnimeMetadataUpdates?.get(videoId);
}
function makeMergedToken(overrides: Partial<MergedToken>): MergedToken {
return {
surface: '',
reading: '',
headword: '',
startPos: 0,
endPos: 0,
partOfSpeech: PartOfSpeech.other,
pos1: '',
pos2: '',
pos3: '',
isMerged: true,
isKnown: false,
isNPlusOneTarget: false,
...overrides,
};
}
function makeDbPath(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-immersion-test-'));
return path.join(dir, 'immersion.sqlite');
@@ -222,6 +251,308 @@ test('persists and retrieves minimum immersion tracking fields', async () => {
}
});
test('recordSubtitleLine persists counted allowed tokenized vocabulary rows and subtitle-line occurrences', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
try {
const Ctor = await loadTrackerCtor();
tracker = new Ctor({ dbPath });
tracker.handleMediaChange('/tmp/Little Witch Academia S02E04.mkv', 'Episode 4');
await waitForPendingAnimeMetadata(tracker);
tracker.recordSubtitleLine('猫 猫 日 日 は 知っている', 0, 1, [
makeMergedToken({
surface: '猫',
headword: '猫',
reading: 'ねこ',
partOfSpeech: PartOfSpeech.noun,
pos1: '名詞',
pos2: '一般',
}),
makeMergedToken({
surface: '猫',
headword: '猫',
reading: 'ねこ',
partOfSpeech: PartOfSpeech.noun,
pos1: '名詞',
pos2: '一般',
}),
makeMergedToken({
surface: 'は',
headword: 'は',
reading: 'は',
partOfSpeech: PartOfSpeech.particle,
pos1: '助詞',
pos2: '係助詞',
}),
makeMergedToken({
surface: '知っている',
headword: '知る',
reading: 'しっている',
partOfSpeech: PartOfSpeech.other,
pos1: '動詞',
pos2: '自立',
}),
]);
const privateApi = tracker as unknown as {
flushTelemetry: (force?: boolean) => void;
flushNow: () => void;
};
privateApi.flushTelemetry(true);
privateApi.flushNow();
const db = new Database(dbPath);
const rows = db
.prepare(
`SELECT headword, word, reading, part_of_speech, pos1, pos2, frequency
FROM imm_words
ORDER BY id ASC`,
)
.all() as Array<{
headword: string;
word: string;
reading: string;
part_of_speech: string;
pos1: string;
pos2: string;
frequency: number;
}>;
const lineRows = db
.prepare(
`SELECT video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text
FROM imm_subtitle_lines
ORDER BY line_id ASC`,
)
.all() as Array<{
video_id: number;
anime_id: number | null;
line_index: number;
segment_start_ms: number | null;
segment_end_ms: number | null;
text: string;
}>;
const wordOccurrenceRows = db
.prepare(
`SELECT o.occurrence_count, w.headword, w.word, w.reading
FROM imm_word_line_occurrences o
JOIN imm_words w ON w.id = o.word_id
ORDER BY o.line_id ASC, o.word_id ASC`,
)
.all() as Array<{
occurrence_count: number;
headword: string;
word: string;
reading: string;
}>;
const kanjiOccurrenceRows = db
.prepare(
`SELECT o.occurrence_count, k.kanji
FROM imm_kanji_line_occurrences o
JOIN imm_kanji k ON k.id = o.kanji_id
ORDER BY o.line_id ASC, k.kanji ASC`,
)
.all() as Array<{
occurrence_count: number;
kanji: string;
}>;
db.close();
assert.deepEqual(rows, [
{
headword: '猫',
word: '猫',
reading: 'ねこ',
part_of_speech: PartOfSpeech.noun,
pos1: '名詞',
pos2: '一般',
frequency: 2,
},
{
headword: '知る',
word: '知っている',
reading: 'しっている',
part_of_speech: PartOfSpeech.verb,
pos1: '動詞',
pos2: '自立',
frequency: 1,
},
]);
assert.equal(lineRows.length, 1);
assert.equal(lineRows[0]?.line_index, 1);
assert.equal(lineRows[0]?.segment_start_ms, 0);
assert.equal(lineRows[0]?.segment_end_ms, 1000);
assert.equal(lineRows[0]?.text, '猫 猫 日 日 は 知っている');
assert.ok(lineRows[0]?.video_id);
assert.ok(lineRows[0]?.anime_id);
assert.deepEqual(wordOccurrenceRows, [
{
occurrence_count: 2,
headword: '猫',
word: '猫',
reading: 'ねこ',
},
{
occurrence_count: 1,
headword: '知る',
word: '知っている',
reading: 'しっている',
},
]);
assert.deepEqual(kanjiOccurrenceRows, [
{
occurrence_count: 2,
kanji: '日',
},
{
occurrence_count: 2,
kanji: '猫',
},
{
occurrence_count: 1,
kanji: '知',
},
]);
} finally {
tracker?.destroy();
cleanupDbPath(dbPath);
}
});
test('handleMediaChange links parsed anime metadata on the active video row', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
try {
const Ctor = await loadTrackerCtor();
tracker = new Ctor({ dbPath });
tracker.handleMediaChange('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5');
await waitForPendingAnimeMetadata(tracker);
const privateApi = tracker as unknown as {
db: DatabaseSync;
sessionState: { videoId: number } | null;
};
const videoId = privateApi.sessionState?.videoId;
assert.ok(videoId);
const row = privateApi.db
.prepare(
`
SELECT
v.anime_id,
v.parsed_basename,
v.parsed_title,
v.parsed_season,
v.parsed_episode,
v.parser_source,
a.canonical_title AS anime_title,
a.anilist_id
FROM imm_videos v
LEFT JOIN imm_anime a ON a.anime_id = v.anime_id
WHERE v.video_id = ?
`,
)
.get(videoId) as {
anime_id: number | null;
parsed_basename: string | null;
parsed_title: string | null;
parsed_season: number | null;
parsed_episode: number | null;
parser_source: string | null;
anime_title: string | null;
anilist_id: number | null;
} | null;
assert.ok(row);
assert.ok(row?.anime_id);
assert.equal(row?.parsed_basename, 'Little Witch Academia S02E05.mkv');
assert.equal(row?.parsed_title, 'Little Witch Academia');
assert.equal(row?.parsed_season, 2);
assert.equal(row?.parsed_episode, 5);
assert.ok(row?.parser_source === 'guessit' || row?.parser_source === 'fallback');
assert.equal(row?.anime_title, 'Little Witch Academia');
assert.equal(row?.anilist_id, null);
} finally {
tracker?.destroy();
cleanupDbPath(dbPath);
}
});
test('handleMediaChange reuses the same provisional anime row across matching files', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;
try {
const Ctor = await loadTrackerCtor();
tracker = new Ctor({ dbPath });
tracker.handleMediaChange('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5');
await waitForPendingAnimeMetadata(tracker);
tracker.handleMediaChange('/tmp/Little Witch Academia S02E06.mkv', 'Episode 6');
await waitForPendingAnimeMetadata(tracker);
const privateApi = tracker as unknown as {
db: DatabaseSync;
};
const rows = privateApi.db
.prepare(
`
SELECT
v.source_path,
v.anime_id,
v.parsed_episode,
a.canonical_title AS anime_title,
a.anilist_id
FROM imm_videos v
LEFT JOIN imm_anime a ON a.anime_id = v.anime_id
WHERE v.source_path IN (?, ?)
ORDER BY v.source_path
`,
)
.all('/tmp/Little Witch Academia S02E05.mkv', '/tmp/Little Witch Academia S02E06.mkv') as
Array<{
source_path: string | null;
anime_id: number | null;
parsed_episode: number | null;
anime_title: string | null;
anilist_id: number | null;
}>;
assert.equal(rows.length, 2);
assert.ok(rows[0]?.anime_id);
assert.equal(rows[0]?.anime_id, rows[1]?.anime_id);
assert.deepEqual(
rows.map((row) => ({
sourcePath: row.source_path,
parsedEpisode: row.parsed_episode,
animeTitle: row.anime_title,
anilistId: row.anilist_id,
})),
[
{
sourcePath: '/tmp/Little Witch Academia S02E05.mkv',
parsedEpisode: 5,
animeTitle: 'Little Witch Academia',
anilistId: null,
},
{
sourcePath: '/tmp/Little Witch Academia S02E06.mkv',
parsedEpisode: 6,
animeTitle: 'Little Witch Academia',
anilistId: null,
},
],
);
} finally {
tracker?.destroy();
cleanupDbPath(dbPath);
}
});
test('applies configurable queue, flush, and retention policy', async () => {
const dbPath = makeDbPath();
let tracker: ImmersionTrackerService | null = null;

View File

@@ -1,7 +1,8 @@
import path from 'node:path';
import * as fs from 'node:fs';
import { createLogger } from '../../logger';
import { getLocalVideoMetadata } from './immersion-tracker/metadata';
import type { CoverArtFetcher } from './anilist/cover-art-fetcher';
import { getLocalVideoMetadata, guessAnimeVideoMetadata } from './immersion-tracker/metadata';
import { pruneRetention, runRollupMaintenance } from './immersion-tracker/maintenance';
import { Database, type DatabaseSync } from './immersion-tracker/sqlite';
import { finalizeSessionRecord, startSessionRecord } from './immersion-tracker/session';
@@ -10,23 +11,58 @@ import {
createTrackerPreparedStatements,
ensureSchema,
executeQueuedWrite,
getOrCreateAnimeRecord,
getOrCreateVideoRecord,
linkVideoToAnimeRecord,
type TrackerPreparedStatements,
updateVideoMetadataRecord,
updateVideoTitleRecord,
} from './immersion-tracker/storage';
import {
cleanupVocabularyStats,
getAnimeCoverArt,
getAnimeDailyRollups,
getAnimeAnilistEntries,
getAnimeDetail,
getAnimeEpisodes,
getAnimeLibrary,
getAnimeWords,
getEpisodeCardEvents,
getEpisodeSessions,
getEpisodeWords,
getCoverArt,
getDailyRollups,
getEpisodesPerDay,
getKanjiAnimeAppearances,
getKanjiDetail,
getKanjiWords,
getNewAnimePerDay,
getSimilarWords,
getStreakCalendar,
getKanjiOccurrences,
getKanjiStats,
getMediaDailyRollups,
getMediaDetail,
getMediaLibrary,
getMediaSessions,
getMonthlyRollups,
getQueryHints,
getSessionEvents,
getSessionSummaries,
getSessionTimeline,
getVocabularyStats,
getWatchTimePerAnime,
getWordAnimeAppearances,
getWordDetail,
getWordOccurrences,
getVideoDurationMs,
markVideoWatched,
} from './immersion-tracker/query';
import {
buildVideoKey,
calculateTextMetrics,
extractLineVocabulary,
deriveCanonicalTitle,
isKanji,
isRemoteSource,
normalizeMediaPath,
normalizeText,
@@ -57,19 +93,73 @@ import {
SOURCE_TYPE_LOCAL,
SOURCE_TYPE_REMOTE,
type ImmersionSessionRollupRow,
type EpisodeCardEventRow,
type EpisodesPerDayRow,
type ImmersionTrackerOptions,
type KanjiAnimeAppearanceRow,
type KanjiDetailRow,
type KanjiOccurrenceRow,
type KanjiStatsRow,
type KanjiWordRow,
type LegacyVocabularyPosResolution,
type LegacyVocabularyPosRow,
type AnimeAnilistEntryRow,
type AnimeDetailRow,
type AnimeEpisodeRow,
type AnimeLibraryRow,
type AnimeWordRow,
type MediaArtRow,
type MediaDetailRow,
type MediaLibraryRow,
type NewAnimePerDayRow,
type QueuedWrite,
type SessionEventRow,
type SessionState,
type SessionSummaryQueryRow,
type SessionTimelineRow,
type SimilarWordRow,
type StreakCalendarRow,
type VocabularyCleanupSummary,
type WatchTimePerAnimeRow,
type WordAnimeAppearanceRow,
type WordDetailRow,
type WordOccurrenceRow,
type VocabularyStatsRow,
} from './immersion-tracker/types';
import type { MergedToken } from '../../types';
import { shouldExcludeTokenFromVocabularyPersistence } from './tokenizer/annotation-stage';
import { deriveStoredPartOfSpeech } from './tokenizer/part-of-speech';
export type {
AnimeAnilistEntryRow,
AnimeDetailRow,
AnimeEpisodeRow,
AnimeLibraryRow,
AnimeWordRow,
EpisodeCardEventRow,
EpisodesPerDayRow,
ImmersionSessionRollupRow,
ImmersionTrackerOptions,
ImmersionTrackerPolicy,
KanjiAnimeAppearanceRow,
KanjiDetailRow,
KanjiOccurrenceRow,
KanjiStatsRow,
KanjiWordRow,
MediaArtRow,
MediaDetailRow,
MediaLibraryRow,
NewAnimePerDayRow,
SessionEventRow,
SessionSummaryQueryRow,
SessionTimelineRow,
SimilarWordRow,
StreakCalendarRow,
WatchTimePerAnimeRow,
WordAnimeAppearanceRow,
WordDetailRow,
WordOccurrenceRow,
VocabularyStatsRow,
} from './immersion-tracker/types';
export class ImmersionTrackerService {
@@ -98,9 +188,17 @@ export class ImmersionTrackerService {
private currentVideoKey = '';
private currentMediaPathOrUrl = '';
private readonly preparedStatements: TrackerPreparedStatements;
private coverArtFetcher: CoverArtFetcher | null = null;
private readonly pendingCoverFetches = new Map<number, Promise<boolean>>();
private readonly recordedSubtitleKeys = new Set<string>();
private readonly pendingAnimeMetadataUpdates = new Map<number, Promise<void>>();
private readonly resolveLegacyVocabularyPos:
| ((row: LegacyVocabularyPosRow) => Promise<LegacyVocabularyPosResolution | null>)
| undefined;
constructor(options: ImmersionTrackerOptions) {
this.dbPath = options.dbPath;
this.resolveLegacyVocabularyPos = options.resolveLegacyVocabularyPos;
const parentDir = path.dirname(this.dbPath);
if (!fs.existsSync(parentDir)) {
fs.mkdirSync(parentDir, { recursive: true });
@@ -198,6 +296,8 @@ export class ImmersionTrackerService {
async getQueryHints(): Promise<{
totalSessions: number;
activeSessions: number;
episodesToday: number;
activeAnimeCount: number;
}> {
return getQueryHints(this.db);
}
@@ -210,6 +310,180 @@ export class ImmersionTrackerService {
return getMonthlyRollups(this.db, limit);
}
async getVocabularyStats(limit = 100, excludePos?: string[]): Promise<VocabularyStatsRow[]> {
return getVocabularyStats(this.db, limit, excludePos);
}
async cleanupVocabularyStats(): Promise<VocabularyCleanupSummary> {
return cleanupVocabularyStats(this.db, {
resolveLegacyPos: this.resolveLegacyVocabularyPos,
});
}
async getKanjiStats(limit = 100): Promise<KanjiStatsRow[]> {
return getKanjiStats(this.db, limit);
}
async getWordOccurrences(
headword: string,
word: string,
reading: string,
limit = 100,
offset = 0,
): Promise<WordOccurrenceRow[]> {
return getWordOccurrences(this.db, headword, word, reading, limit, offset);
}
async getKanjiOccurrences(
kanji: string,
limit = 100,
offset = 0,
): Promise<KanjiOccurrenceRow[]> {
return getKanjiOccurrences(this.db, kanji, limit, offset);
}
async getSessionEvents(sessionId: number, limit = 500): Promise<SessionEventRow[]> {
return getSessionEvents(this.db, sessionId, limit);
}
async getMediaLibrary(): Promise<MediaLibraryRow[]> {
return getMediaLibrary(this.db);
}
async getMediaDetail(videoId: number): Promise<MediaDetailRow | null> {
return getMediaDetail(this.db, videoId);
}
async getMediaSessions(videoId: number, limit = 100): Promise<SessionSummaryQueryRow[]> {
return getMediaSessions(this.db, videoId, limit);
}
async getMediaDailyRollups(videoId: number, limit = 90): Promise<ImmersionSessionRollupRow[]> {
return getMediaDailyRollups(this.db, videoId, limit);
}
async getCoverArt(videoId: number): Promise<MediaArtRow | null> {
return getCoverArt(this.db, videoId);
}
async getAnimeLibrary(): Promise<AnimeLibraryRow[]> {
return getAnimeLibrary(this.db);
}
async getAnimeDetail(animeId: number): Promise<AnimeDetailRow | null> {
return getAnimeDetail(this.db, animeId);
}
async getAnimeEpisodes(animeId: number): Promise<AnimeEpisodeRow[]> {
return getAnimeEpisodes(this.db, animeId);
}
async getAnimeAnilistEntries(animeId: number): Promise<AnimeAnilistEntryRow[]> {
return getAnimeAnilistEntries(this.db, animeId);
}
async getAnimeCoverArt(animeId: number): Promise<MediaArtRow | null> {
return getAnimeCoverArt(this.db, animeId);
}
async getAnimeWords(animeId: number, limit = 50): Promise<AnimeWordRow[]> {
return getAnimeWords(this.db, animeId, limit);
}
async getEpisodeWords(videoId: number, limit = 50): Promise<AnimeWordRow[]> {
return getEpisodeWords(this.db, videoId, limit);
}
async getEpisodeSessions(videoId: number): Promise<SessionSummaryQueryRow[]> {
return getEpisodeSessions(this.db, videoId);
}
async setVideoWatched(videoId: number, watched: boolean): Promise<void> {
markVideoWatched(this.db, videoId, watched);
}
async getEpisodeCardEvents(videoId: number): Promise<EpisodeCardEventRow[]> {
return getEpisodeCardEvents(this.db, videoId);
}
async getAnimeDailyRollups(animeId: number, limit = 90): Promise<ImmersionSessionRollupRow[]> {
return getAnimeDailyRollups(this.db, animeId, limit);
}
async getStreakCalendar(days = 90): Promise<StreakCalendarRow[]> {
return getStreakCalendar(this.db, days);
}
async getEpisodesPerDay(limit = 90): Promise<EpisodesPerDayRow[]> {
return getEpisodesPerDay(this.db, limit);
}
async getNewAnimePerDay(limit = 90): Promise<NewAnimePerDayRow[]> {
return getNewAnimePerDay(this.db, limit);
}
async getWatchTimePerAnime(limit = 90): Promise<WatchTimePerAnimeRow[]> {
return getWatchTimePerAnime(this.db, limit);
}
async getWordDetail(wordId: number): Promise<WordDetailRow | null> {
return getWordDetail(this.db, wordId);
}
async getWordAnimeAppearances(wordId: number): Promise<WordAnimeAppearanceRow[]> {
return getWordAnimeAppearances(this.db, wordId);
}
async getSimilarWords(wordId: number, limit = 10): Promise<SimilarWordRow[]> {
return getSimilarWords(this.db, wordId, limit);
}
async getKanjiDetail(kanjiId: number): Promise<KanjiDetailRow | null> {
return getKanjiDetail(this.db, kanjiId);
}
async getKanjiAnimeAppearances(kanjiId: number): Promise<KanjiAnimeAppearanceRow[]> {
return getKanjiAnimeAppearances(this.db, kanjiId);
}
async getKanjiWords(kanjiId: number, limit = 20): Promise<KanjiWordRow[]> {
return getKanjiWords(this.db, kanjiId, limit);
}
setCoverArtFetcher(fetcher: CoverArtFetcher | null): void {
this.coverArtFetcher = fetcher;
}
async ensureCoverArt(videoId: number): Promise<boolean> {
const existing = getCoverArt(this.db, videoId);
if (existing?.coverBlob) {
return true;
}
if (!this.coverArtFetcher) {
return false;
}
const inFlight = this.pendingCoverFetches.get(videoId);
if (inFlight) {
return await inFlight;
}
const fetchPromise = (async () => {
const detail = getMediaDetail(this.db, videoId);
const canonicalTitle = detail?.canonicalTitle?.trim();
if (!canonicalTitle) {
return false;
}
return await this.coverArtFetcher!.fetchIfMissing(this.db, videoId, canonicalTitle);
})();
this.pendingCoverFetches.set(videoId, fetchPromise);
try {
return await fetchPromise;
} finally {
this.pendingCoverFetches.delete(videoId);
}
}
handleMediaChange(mediaPath: string | null, mediaTitle: string | null): void {
const normalizedPath = normalizeMediaPath(mediaPath);
const normalizedTitle = normalizeText(mediaTitle);
@@ -254,6 +528,7 @@ export class ImmersionTrackerService {
`Starting immersion session for path=${normalizedPath} videoId=${sessionInfo.videoId}`,
);
this.startSession(sessionInfo.videoId, sessionInfo.startedAtMs);
this.captureAnimeMetadataAsync(sessionInfo.videoId, normalizedPath, normalizedTitle || null);
this.captureVideoMetadataAsync(sessionInfo.videoId, sourceType, normalizedPath);
}
@@ -265,41 +540,111 @@ export class ImmersionTrackerService {
this.updateVideoTitleForActiveSession(normalizedTitle);
}
recordSubtitleLine(text: string, startSec: number, endSec: number): void {
recordSubtitleLine(
text: string,
startSec: number,
endSec: number,
tokens?: MergedToken[] | null,
): void {
if (!this.sessionState || !text.trim()) return;
const cleaned = normalizeText(text);
if (!cleaned) return;
if (!endSec || endSec <= 0) {
return;
}
const startMs = secToMs(startSec);
const subtitleKey = `${startMs}:${cleaned}`;
if (this.recordedSubtitleKeys.has(subtitleKey)) {
return;
}
this.recordedSubtitleKeys.add(subtitleKey);
const nowMs = Date.now();
const nowSec = nowMs / 1000;
const metrics = calculateTextMetrics(cleaned);
const extractedVocabulary = extractLineVocabulary(cleaned);
this.sessionState.currentLineIndex += 1;
this.sessionState.linesSeen += 1;
this.sessionState.wordsSeen += metrics.words;
this.sessionState.tokensSeen += metrics.tokens;
this.sessionState.pendingTelemetry = true;
for (const { headword, word, reading } of extractedVocabulary.words) {
this.recordWrite({
kind: 'word',
const wordOccurrences = new Map<
string,
{
headword: string;
word: string;
reading: string;
partOfSpeech: string;
pos1: string;
pos2: string;
pos3: string;
occurrenceCount: number;
}
>();
for (const token of tokens ?? []) {
if (shouldExcludeTokenFromVocabularyPersistence(token)) {
continue;
}
const headword = normalizeText(token.headword || token.surface);
const word = normalizeText(token.surface || token.headword);
const reading = normalizeText(token.reading);
if (!headword || !word) {
continue;
}
const wordKey = [
headword,
word,
reading,
firstSeen: nowSec,
lastSeen: nowSec,
].join('\u0000');
const storedPartOfSpeech = deriveStoredPartOfSpeech({
partOfSpeech: token.partOfSpeech,
pos1: token.pos1 ?? '',
});
const existing = wordOccurrences.get(wordKey);
if (existing) {
existing.occurrenceCount += 1;
continue;
}
wordOccurrences.set(wordKey, {
headword,
word,
reading,
partOfSpeech: storedPartOfSpeech,
pos1: token.pos1 ?? '',
pos2: token.pos2 ?? '',
pos3: token.pos3 ?? '',
occurrenceCount: 1,
});
}
for (const kanji of extractedVocabulary.kanji) {
this.recordWrite({
kind: 'kanji',
kanji,
firstSeen: nowSec,
lastSeen: nowSec,
});
const kanjiCounts = new Map<string, number>();
for (const char of cleaned) {
if (!isKanji(char)) {
continue;
}
kanjiCounts.set(char, (kanjiCounts.get(char) ?? 0) + 1);
}
this.recordWrite({
kind: 'subtitleLine',
sessionId: this.sessionState.sessionId,
videoId: this.sessionState.videoId,
lineIndex: this.sessionState.currentLineIndex,
segmentStartMs: secToMs(startSec),
segmentEndMs: secToMs(endSec),
text: cleaned,
wordOccurrences: Array.from(wordOccurrences.values()),
kanjiOccurrences: Array.from(kanjiCounts.entries()).map(([kanji, occurrenceCount]) => ({
kanji,
occurrenceCount,
})),
firstSeen: nowSec,
lastSeen: nowSec,
});
this.recordWrite({
kind: 'event',
sessionId: this.sessionState.sessionId,
@@ -321,6 +666,16 @@ export class ImmersionTrackerService {
});
}
recordMediaDuration(durationSec: number): void {
if (!this.sessionState || !Number.isFinite(durationSec) || durationSec <= 0) return;
const durationMs = Math.round(durationSec * 1000);
const current = getVideoDurationMs(this.db, this.sessionState.videoId);
if (current === 0 || Math.abs(current - durationMs) > 1000) {
this.db.prepare('UPDATE imm_videos SET duration_ms = ?, LAST_UPDATE_DATE = ? WHERE video_id = ?')
.run(durationMs, Date.now(), this.sessionState.videoId);
}
}
recordPlaybackPosition(mediaTimeSec: number | null): void {
if (!this.sessionState || mediaTimeSec === null || !Number.isFinite(mediaTimeSec)) {
return;
@@ -391,6 +746,14 @@ export class ImmersionTrackerService {
this.sessionState.lastWallClockMs = nowMs;
this.sessionState.lastMediaMs = mediaMs;
this.sessionState.pendingTelemetry = true;
if (!this.sessionState.markedWatched) {
const durationMs = getVideoDurationMs(this.db, this.sessionState.videoId);
if (durationMs > 0 && mediaMs >= durationMs * 0.98) {
markVideoWatched(this.db, this.sessionState.videoId, true);
this.sessionState.markedWatched = true;
}
}
}
recordPauseState(isPaused: boolean): void {
@@ -454,7 +817,7 @@ export class ImmersionTrackerService {
});
}
recordCardsMined(count = 1): void {
recordCardsMined(count = 1, noteIds?: number[]): void {
if (!this.sessionState) return;
this.sessionState.cardsMined += count;
this.sessionState.pendingTelemetry = true;
@@ -465,7 +828,10 @@ export class ImmersionTrackerService {
eventType: EVENT_CARD_MINED,
wordsDelta: 0,
cardsDelta: count,
payloadJson: sanitizePayload({ cardsMined: count }, this.maxPayloadBytes),
payloadJson: sanitizePayload(
{ cardsMined: count, ...(noteIds?.length ? { noteIds } : {}) },
this.maxPayloadBytes,
),
});
}
@@ -615,6 +981,7 @@ export class ImmersionTrackerService {
private startSession(videoId: number, startedAtMs?: number): void {
const { sessionId, state } = startSessionRecord(this.db, videoId, startedAtMs);
this.sessionState = state;
this.recordedSubtitleKeys.clear();
this.recordWrite({
kind: 'telemetry',
sessionId,
@@ -673,6 +1040,48 @@ export class ImmersionTrackerService {
})();
}
private captureAnimeMetadataAsync(
videoId: number,
mediaPath: string | null,
mediaTitle: string | null,
): void {
const updatePromise = (async () => {
try {
const parsed = await guessAnimeVideoMetadata(mediaPath, mediaTitle);
if (this.isDestroyed || !parsed?.parsedTitle.trim()) {
return;
}
const animeId = getOrCreateAnimeRecord(this.db, {
parsedTitle: parsed.parsedTitle,
canonicalTitle: parsed.parsedTitle,
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: parsed.parseMetadataJson,
});
linkVideoToAnimeRecord(this.db, videoId, {
animeId,
parsedBasename: parsed.parsedBasename,
parsedTitle: parsed.parsedTitle,
parsedSeason: parsed.parsedSeason,
parsedEpisode: parsed.parsedEpisode,
parserSource: parsed.parserSource,
parserConfidence: parsed.parserConfidence,
parseMetadataJson: parsed.parseMetadataJson,
});
} catch (error) {
this.logger.warn('Unable to capture anime metadata', (error as Error).message);
}
})();
this.pendingAnimeMetadataUpdates.set(videoId, updatePromise);
void updatePromise.finally(() => {
this.pendingAnimeMetadataUpdates.delete(videoId);
});
}
private updateVideoTitleForActiveSession(canonicalTitle: string): void {
if (!this.sessionState) return;
updateVideoTitleRecord(this.db, this.sessionState.videoId, canonicalTitle);

View File

@@ -0,0 +1,976 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { Database } from '../sqlite.js';
import {
createTrackerPreparedStatements,
ensureSchema,
getOrCreateAnimeRecord,
getOrCreateVideoRecord,
linkVideoToAnimeRecord,
} from '../storage.js';
import { startSessionRecord } from '../session.js';
import {
cleanupVocabularyStats,
getAnimeDetail,
getAnimeEpisodes,
getAnimeLibrary,
getKanjiOccurrences,
getSessionSummaries,
getVocabularyStats,
getKanjiStats,
getSessionEvents,
getWordOccurrences,
} from '../query.js';
import { SOURCE_TYPE_LOCAL, EVENT_SUBTITLE_LINE } from '../types.js';
function makeDbPath(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-imm-query-test-'));
return path.join(dir, 'immersion.sqlite');
}
function cleanupDbPath(dbPath: string): void {
const dir = path.dirname(dbPath);
if (!fs.existsSync(dir)) {
return;
}
const bunRuntime = globalThis as typeof globalThis & {
Bun?: {
gc?: (force?: boolean) => void;
};
};
let lastError: NodeJS.ErrnoException | null = null;
for (let attempt = 0; attempt < 3; attempt += 1) {
try {
fs.rmSync(dir, { recursive: true, force: true });
return;
} catch (error) {
const err = error as NodeJS.ErrnoException;
lastError = err;
if (process.platform !== 'win32' || err.code !== 'EBUSY') {
throw error;
}
bunRuntime.Bun?.gc?.(true);
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 25);
}
}
if (lastError) {
throw lastError;
}
}
test('getSessionSummaries returns sessionId and canonicalTitle', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/query-test.mkv', {
canonicalTitle: 'Query Test Episode',
sourcePath: '/tmp/query-test.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const startedAtMs = 1_000_000;
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
stmts.telemetryInsertStmt.run(
sessionId,
startedAtMs + 1_000,
3_000,
2_500,
5,
10,
10,
1,
2,
1,
0,
0,
0,
0,
0,
startedAtMs + 1_000,
startedAtMs + 1_000,
);
const rows = getSessionSummaries(db, 10);
assert.ok(rows.length >= 1);
const row = rows.find((r) => r.sessionId === sessionId);
assert.ok(row, 'expected to find a row for the created session');
assert.equal(typeof row.sessionId, 'number');
assert.equal(row.sessionId, sessionId);
assert.equal(row.canonicalTitle, 'Query Test Episode');
assert.equal(row.videoId, videoId);
assert.ok(row.linesSeen >= 5);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getSessionSummaries with no telemetry returns zero aggregates', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/no-telemetry.mkv', {
canonicalTitle: 'No Telemetry',
sourcePath: '/tmp/no-telemetry.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const { sessionId } = startSessionRecord(db, videoId, 3_000_000);
const rows = getSessionSummaries(db, 10);
const row = rows.find((r) => r.sessionId === sessionId);
assert.ok(row, 'expected to find the session with no telemetry');
assert.equal(row.canonicalTitle, 'No Telemetry');
assert.equal(row.totalWatchedMs, 0);
assert.equal(row.linesSeen, 0);
assert.equal(row.cardsMined, 0);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getVocabularyStats returns rows ordered by frequency descending', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
// Insert words: 猫 twice, 犬 once
stmts.wordUpsertStmt.run('猫', '猫', 'ねこ', 'noun', '名詞', '一般', '', 1_000, 2_000);
stmts.wordUpsertStmt.run('猫', '猫', 'ねこ', 'noun', '名詞', '一般', '', 1_000, 3_000);
stmts.wordUpsertStmt.run('犬', '犬', 'いぬ', 'noun', '名詞', '一般', '', 1_500, 1_500);
const rows = getVocabularyStats(db, 10);
assert.ok(rows.length >= 2);
// First row should be 猫 (frequency 2)
const nekRow = rows.find((r) => r.headword === '猫');
const inuRow = rows.find((r) => r.headword === '犬');
assert.ok(nekRow, 'expected 猫 row');
assert.ok(inuRow, 'expected 犬 row');
assert.equal(nekRow.headword, '猫');
assert.equal(nekRow.word, '猫');
assert.equal(nekRow.reading, 'ねこ');
assert.equal(nekRow.frequency, 2);
assert.equal(typeof nekRow.firstSeen, 'number');
assert.equal(typeof nekRow.lastSeen, 'number');
// Higher frequency should come first
const nekIdx = rows.indexOf(nekRow);
const inuIdx = rows.indexOf(inuRow);
assert.ok(nekIdx < inuIdx, 'higher frequency word should appear first');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getVocabularyStats returns empty array when no words exist', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const rows = getVocabularyStats(db, 10);
assert.deepEqual(rows, []);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('cleanupVocabularyStats repairs stored POS metadata and removes excluded imm_words rows', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
db.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run('猫', '猫', 'ねこ', 'noun', '名詞', '一般', '', 1_000, 1_500, 3);
db.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run('知っている', '知っている', '', 'other', '動詞', '自立', '', 1_025, 1_525, 4);
db.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run('は', 'は', 'は', 'particle', '助詞', '係助詞', '', 1_100, 1_600, 9);
db.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run('旧', '旧', '', '', '', '', '', 900, 950, 1);
db.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run('未解決', '未解決', '', '', '', '', '', 901, 951, 1);
const result = await cleanupVocabularyStats(db, {
resolveLegacyPos: async (row) => {
if (row.headword === '旧') {
return {
partOfSpeech: 'noun',
headword: '旧',
reading: 'きゅう',
pos1: '名詞',
pos2: '一般',
pos3: '',
};
}
if (row.headword === '知っている') {
return {
partOfSpeech: 'verb',
headword: '知る',
reading: 'しっている',
pos1: '動詞',
pos2: '自立',
pos3: '',
};
}
return null;
},
});
const rows = getVocabularyStats(db, 10);
const repairedRows = db
.prepare(
`SELECT headword, word, reading, part_of_speech, pos1, pos2
FROM imm_words
ORDER BY headword ASC, word ASC`,
)
.all() as Array<{
headword: string;
word: string;
reading: string;
part_of_speech: string;
pos1: string;
pos2: string;
}>;
assert.deepEqual(result, { scanned: 5, kept: 3, deleted: 2, repaired: 2 });
assert.deepEqual(
rows.map((row) => ({ headword: row.headword, frequency: row.frequency })),
[
{ headword: '知る', frequency: 4 },
{ headword: '猫', frequency: 3 },
{ headword: '旧', frequency: 1 },
],
);
assert.deepEqual(
repairedRows,
[
{
headword: '旧',
word: '旧',
reading: 'きゅう',
part_of_speech: 'noun',
pos1: '名詞',
pos2: '一般',
},
{
headword: '猫',
word: '猫',
reading: 'ねこ',
part_of_speech: 'noun',
pos1: '名詞',
pos2: '一般',
},
{
headword: '知る',
word: '知っている',
reading: 'しっている',
part_of_speech: 'verb',
pos1: '動詞',
pos2: '自立',
},
],
);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('cleanupVocabularyStats merges repaired duplicates instead of violating the imm_words unique key', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/cleanup-merge.mkv', {
canonicalTitle: 'Cleanup Merge',
sourcePath: '/tmp/cleanup-merge.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const { sessionId } = startSessionRecord(db, videoId, 2_000_000);
const duplicateResult = db
.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run('知る', '知っている', 'しっている', 'verb', '動詞', '自立', '', 2_000, 2_500, 3);
const legacyResult = db
.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run('知っている', '知っている', '', 'other', '動詞', '自立', '', 1_000, 3_000, 4);
const lineResult = db
.prepare(
`INSERT INTO imm_subtitle_lines (
session_id, event_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(sessionId, null, videoId, null, 1, 0, 1000, '知っている', 2_000, 2_000);
const lineId = Number(lineResult.lastInsertRowid);
const duplicateId = Number(duplicateResult.lastInsertRowid);
const legacyId = Number(legacyResult.lastInsertRowid);
db.prepare(
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
VALUES (?, ?, ?)`,
).run(lineId, duplicateId, 2);
db.prepare(
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
VALUES (?, ?, ?)`,
).run(lineId, legacyId, 1);
const result = await cleanupVocabularyStats(db, {
resolveLegacyPos: async (row) => {
if (row.id !== legacyId) {
return null;
}
return {
partOfSpeech: 'verb',
headword: '知る',
reading: 'しっている',
pos1: '動詞',
pos2: '自立',
pos3: '',
};
},
});
const rows = db
.prepare(
`SELECT id, headword, word, reading, frequency, first_seen, last_seen
FROM imm_words
ORDER BY id ASC`,
)
.all() as Array<{
id: number;
headword: string;
word: string;
reading: string;
frequency: number;
first_seen: number;
last_seen: number;
}>;
const occurrences = getWordOccurrences(db, '知る', '知っている', 'しっている', 10);
assert.deepEqual(result, { scanned: 2, kept: 1, deleted: 1, repaired: 1 });
assert.deepEqual(rows, [
{
id: duplicateId,
headword: '知る',
word: '知っている',
reading: 'しっている',
frequency: 7,
first_seen: 1_000,
last_seen: 3_000,
},
]);
assert.deepEqual(occurrences, [
{
animeId: null,
animeTitle: null,
videoId,
videoTitle: 'Cleanup Merge',
sessionId,
lineIndex: 1,
segmentStartMs: 0,
segmentEndMs: 1000,
text: '知っている',
occurrenceCount: 3,
},
]);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getKanjiStats returns rows ordered by frequency descending', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
// Insert kanji: 日 twice, 月 once
stmts.kanjiUpsertStmt.run('日', 1_000, 2_000);
stmts.kanjiUpsertStmt.run('日', 1_000, 3_000);
stmts.kanjiUpsertStmt.run('月', 1_500, 1_500);
const rows = getKanjiStats(db, 10);
assert.ok(rows.length >= 2);
const nichiRow = rows.find((r) => r.kanji === '日');
const tsukiRow = rows.find((r) => r.kanji === '月');
assert.ok(nichiRow, 'expected 日 row');
assert.ok(tsukiRow, 'expected 月 row');
assert.equal(nichiRow.kanji, '日');
assert.equal(nichiRow.frequency, 2);
assert.equal(typeof nichiRow.firstSeen, 'number');
assert.equal(typeof nichiRow.lastSeen, 'number');
// Higher frequency should come first
const nichiIdx = rows.indexOf(nichiRow);
const tsukiIdx = rows.indexOf(tsukiRow);
assert.ok(nichiIdx < tsukiIdx, 'higher frequency kanji should appear first');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getKanjiStats returns empty array when no kanji exist', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const rows = getKanjiStats(db, 10);
assert.deepEqual(rows, []);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getSessionEvents returns events ordered by ts_ms ascending', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/events-test.mkv', {
canonicalTitle: 'Events Test',
sourcePath: '/tmp/events-test.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const startedAtMs = 5_000_000;
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
// Insert two events at different timestamps
stmts.eventInsertStmt.run(
sessionId,
startedAtMs + 2_000,
EVENT_SUBTITLE_LINE,
1,
0,
800,
2,
0,
'{"line":"second"}',
startedAtMs + 2_000,
startedAtMs + 2_000,
);
stmts.eventInsertStmt.run(
sessionId,
startedAtMs + 1_000,
EVENT_SUBTITLE_LINE,
0,
0,
600,
3,
0,
'{"line":"first"}',
startedAtMs + 1_000,
startedAtMs + 1_000,
);
const events = getSessionEvents(db, sessionId, 50);
assert.equal(events.length, 2);
// Should be ordered ASC by ts_ms
assert.equal(events[0]!.tsMs, startedAtMs + 1_000);
assert.equal(events[1]!.tsMs, startedAtMs + 2_000);
assert.equal(events[0]!.eventType, EVENT_SUBTITLE_LINE);
assert.equal(events[0]!.payload, '{"line":"first"}');
assert.equal(events[1]!.payload, '{"line":"second"}');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getSessionEvents returns empty array for session with no events', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const events = getSessionEvents(db, 9999, 50);
assert.deepEqual(events, []);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getSessionEvents respects limit parameter', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/events-limit.mkv', {
canonicalTitle: 'Events Limit Test',
sourcePath: '/tmp/events-limit.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const startedAtMs = 7_000_000;
const { sessionId } = startSessionRecord(db, videoId, startedAtMs);
// Insert 5 events
for (let i = 0; i < 5; i += 1) {
stmts.eventInsertStmt.run(
sessionId,
startedAtMs + i * 1_000,
EVENT_SUBTITLE_LINE,
i,
0,
500,
1,
0,
null,
startedAtMs + i * 1_000,
startedAtMs + i * 1_000,
);
}
const limited = getSessionEvents(db, sessionId, 3);
assert.equal(limited.length, 3);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('anime-level queries group by anime_id and preserve episode-level rows', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
const lwaAnimeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Little Witch Academia',
canonicalTitle: 'Little Witch Academia',
anilistId: 33_435,
titleRomaji: 'Little Witch Academia',
titleEnglish: 'Little Witch Academia',
titleNative: 'リトルウィッチアカデミア',
metadataJson: '{"source":"anilist"}',
});
const frierenAnimeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Frieren',
canonicalTitle: 'Frieren',
anilistId: 52_921,
titleRomaji: 'Sousou no Frieren',
titleEnglish: 'Frieren: Beyond Journey\'s End',
titleNative: '葬送のフリーレン',
metadataJson: '{"source":"anilist"}',
});
const lwaEpisode5 = getOrCreateVideoRecord(db, 'local:/tmp/lwa-s02e05.mkv', {
canonicalTitle: 'Episode 5',
sourcePath: '/tmp/Little Witch Academia S02E05.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const lwaEpisode6 = getOrCreateVideoRecord(db, 'local:/tmp/lwa-s02e06.mkv', {
canonicalTitle: 'Episode 6',
sourcePath: '/tmp/Little Witch Academia S02E06.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const frierenEpisode3 = getOrCreateVideoRecord(db, 'local:/tmp/frieren-03.mkv', {
canonicalTitle: 'Episode 3',
sourcePath: '/tmp/[SubsPlease] Frieren - 03 - Departure.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
linkVideoToAnimeRecord(db, lwaEpisode5, {
animeId: lwaAnimeId,
parsedBasename: 'Little Witch Academia S02E05.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 5,
parserSource: 'fallback',
parserConfidence: 1,
parseMetadataJson: '{"episode":5}',
});
linkVideoToAnimeRecord(db, lwaEpisode6, {
animeId: lwaAnimeId,
parsedBasename: 'Little Witch Academia S02E06.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 6,
parserSource: 'fallback',
parserConfidence: 1,
parseMetadataJson: '{"episode":6}',
});
linkVideoToAnimeRecord(db, frierenEpisode3, {
animeId: frierenAnimeId,
parsedBasename: '[SubsPlease] Frieren - 03 - Departure.mkv',
parsedTitle: 'Frieren',
parsedSeason: 1,
parsedEpisode: 3,
parserSource: 'fallback',
parserConfidence: 0.6,
parseMetadataJson: '{"episode":3}',
});
const sessionA = startSessionRecord(db, lwaEpisode5, 1_000_000);
const sessionB = startSessionRecord(db, lwaEpisode5, 1_010_000);
const sessionC = startSessionRecord(db, lwaEpisode6, 1_020_000);
const sessionD = startSessionRecord(db, frierenEpisode3, 1_030_000);
stmts.telemetryInsertStmt.run(
sessionA.sessionId,
1_001_000,
4_000,
3_000,
10,
25,
25,
1,
3,
2,
0,
0,
0,
0,
0,
1_001_000,
1_001_000,
);
stmts.telemetryInsertStmt.run(
sessionB.sessionId,
1_011_000,
5_000,
4_000,
11,
27,
27,
2,
4,
2,
0,
0,
0,
0,
0,
1_011_000,
1_011_000,
);
stmts.telemetryInsertStmt.run(
sessionC.sessionId,
1_021_000,
6_000,
5_000,
12,
28,
28,
3,
5,
4,
0,
0,
0,
0,
0,
1_021_000,
1_021_000,
);
stmts.telemetryInsertStmt.run(
sessionD.sessionId,
1_031_000,
4_000,
3_500,
8,
20,
20,
1,
2,
1,
0,
0,
0,
0,
0,
1_031_000,
1_031_000,
);
const animeLibrary = getAnimeLibrary(db);
assert.equal(animeLibrary.length, 2);
assert.deepEqual(
animeLibrary.map((row) => ({
animeId: row.animeId,
canonicalTitle: row.canonicalTitle,
totalSessions: row.totalSessions,
totalActiveMs: row.totalActiveMs,
totalCards: row.totalCards,
episodeCount: row.episodeCount,
})),
[
{
animeId: lwaAnimeId,
canonicalTitle: 'Little Witch Academia',
totalSessions: 3,
totalActiveMs: 12_000,
totalCards: 6,
episodeCount: 2,
},
{
animeId: frierenAnimeId,
canonicalTitle: 'Frieren',
totalSessions: 1,
totalActiveMs: 3_500,
totalCards: 1,
episodeCount: 1,
},
],
);
const animeDetail = getAnimeDetail(db, lwaAnimeId);
assert.ok(animeDetail);
assert.equal(animeDetail?.animeId, lwaAnimeId);
assert.equal(animeDetail?.canonicalTitle, 'Little Witch Academia');
assert.equal(animeDetail?.anilistId, 33_435);
assert.equal(animeDetail?.totalSessions, 3);
assert.equal(animeDetail?.totalActiveMs, 12_000);
assert.equal(animeDetail?.totalCards, 6);
assert.equal(animeDetail?.totalWordsSeen, 80);
assert.equal(animeDetail?.totalLinesSeen, 33);
assert.equal(animeDetail?.totalLookupCount, 12);
assert.equal(animeDetail?.totalLookupHits, 8);
assert.equal(animeDetail?.episodeCount, 2);
const episodes = getAnimeEpisodes(db, lwaAnimeId);
assert.deepEqual(
episodes.map((row) => ({
videoId: row.videoId,
season: row.season,
episode: row.episode,
totalSessions: row.totalSessions,
totalActiveMs: row.totalActiveMs,
totalCards: row.totalCards,
})),
[
{
videoId: lwaEpisode5,
season: 2,
episode: 5,
totalSessions: 2,
totalActiveMs: 7_000,
totalCards: 3,
},
{
videoId: lwaEpisode6,
season: 2,
episode: 6,
totalSessions: 1,
totalActiveMs: 5_000,
totalCards: 3,
},
],
);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getWordOccurrences maps a normalized word back to anime, video, and subtitle line context', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const animeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Little Witch Academia',
canonicalTitle: 'Little Witch Academia',
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: '{"source":"test"}',
});
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/lwa-s02e04.mkv', {
canonicalTitle: 'Episode 4',
sourcePath: '/tmp/Little Witch Academia S02E04.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
linkVideoToAnimeRecord(db, videoId, {
animeId,
parsedBasename: 'Little Witch Academia S02E04.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 4,
parserSource: 'fallback',
parserConfidence: 1,
parseMetadataJson: '{"episode":4}',
});
const { sessionId } = startSessionRecord(db, videoId, 1_000_000);
const wordResult = db
.prepare(
`INSERT INTO imm_words (
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run('猫', '猫', 'ねこ', 'noun', '名詞', '一般', '', 1_000, 1_500, 4);
const lineResult = db
.prepare(
`INSERT INTO imm_subtitle_lines (
session_id, event_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(sessionId, null, videoId, animeId, 1, 0, 1000, '猫 猫 日 日 は', 1_000, 1_000);
db.prepare(
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count)
VALUES (?, ?, ?)`,
).run(Number(lineResult.lastInsertRowid), Number(wordResult.lastInsertRowid), 2);
const rows = getWordOccurrences(db, '猫', '猫', 'ねこ', 10);
assert.deepEqual(rows, [
{
animeId,
animeTitle: 'Little Witch Academia',
videoId,
videoTitle: 'Episode 4',
sessionId,
lineIndex: 1,
segmentStartMs: 0,
segmentEndMs: 1000,
text: '猫 猫 日 日 は',
occurrenceCount: 2,
},
]);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('getKanjiOccurrences maps a kanji back to anime, video, and subtitle line context', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const animeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Frieren',
canonicalTitle: 'Frieren',
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: '{"source":"test"}',
});
const videoId = getOrCreateVideoRecord(db, 'local:/tmp/frieren-03.mkv', {
canonicalTitle: 'Episode 3',
sourcePath: '/tmp/[SubsPlease] Frieren - 03 - Departure.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
linkVideoToAnimeRecord(db, videoId, {
animeId,
parsedBasename: '[SubsPlease] Frieren - 03 - Departure.mkv',
parsedTitle: 'Frieren',
parsedSeason: 1,
parsedEpisode: 3,
parserSource: 'fallback',
parserConfidence: 1,
parseMetadataJson: '{"episode":3}',
});
const { sessionId } = startSessionRecord(db, videoId, 2_000_000);
const kanjiResult = db
.prepare(
`INSERT INTO imm_kanji (
kanji, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?)`,
)
.run('日', 2_000, 2_500, 8);
const lineResult = db
.prepare(
`INSERT INTO imm_subtitle_lines (
session_id, event_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(sessionId, null, videoId, animeId, 3, 5000, 6500, '今日は日曜', 2_000, 2_000);
db.prepare(
`INSERT INTO imm_kanji_line_occurrences (line_id, kanji_id, occurrence_count)
VALUES (?, ?, ?)`,
).run(Number(lineResult.lastInsertRowid), Number(kanjiResult.lastInsertRowid), 2);
const rows = getKanjiOccurrences(db, '日', 10);
assert.deepEqual(rows, [
{
animeId,
animeTitle: 'Frieren',
videoId,
videoTitle: 'Episode 3',
sessionId,
lineIndex: 3,
segmentStartMs: 5000,
segmentEndMs: 6500,
text: '今日は日曜',
occurrenceCount: 2,
},
]);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});

View File

@@ -0,0 +1,71 @@
import type { Token } from '../../../types';
import type { LegacyVocabularyPosResolution } from './types';
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
const KATAKANA_TO_HIRAGANA_OFFSET = 0x60;
const KATAKANA_CODEPOINT_START = 0x30a1;
const KATAKANA_CODEPOINT_END = 0x30f6;
function normalizeLookupText(value: string | null | undefined): string {
return typeof value === 'string' ? value.trim() : '';
}
function katakanaToHiragana(text: string): string {
let normalized = '';
for (const char of text) {
const code = char.codePointAt(0);
if (code === undefined) {
continue;
}
if (code >= KATAKANA_CODEPOINT_START && code <= KATAKANA_CODEPOINT_END) {
normalized += String.fromCodePoint(code - KATAKANA_TO_HIRAGANA_OFFSET);
continue;
}
normalized += char;
}
return normalized;
}
function toResolution(token: Token): LegacyVocabularyPosResolution {
return {
headword: normalizeLookupText(token.headword) || normalizeLookupText(token.word),
reading: katakanaToHiragana(normalizeLookupText(token.katakanaReading)),
partOfSpeech: deriveStoredPartOfSpeech({
partOfSpeech: token.partOfSpeech,
pos1: token.pos1,
}),
pos1: normalizeLookupText(token.pos1),
pos2: normalizeLookupText(token.pos2),
pos3: normalizeLookupText(token.pos3),
};
}
export function resolveLegacyVocabularyPosFromTokens(
lookupText: string,
tokens: Token[] | null,
): LegacyVocabularyPosResolution | null {
const normalizedLookup = normalizeLookupText(lookupText);
if (!normalizedLookup || !tokens || tokens.length === 0) {
return null;
}
const exactSurfaceMatches = tokens.filter(
(token) => normalizeLookupText(token.word) === normalizedLookup,
);
if (exactSurfaceMatches.length === 1) {
return toResolution(exactSurfaceMatches[0]!);
}
const exactHeadwordMatches = tokens.filter(
(token) => normalizeLookupText(token.headword) === normalizedLookup,
);
if (exactHeadwordMatches.length === 1) {
return toResolution(exactHeadwordMatches[0]!);
}
if (tokens.length === 1) {
return toResolution(tokens[0]!);
}
return null;
}

View File

@@ -112,35 +112,46 @@ function upsertDailyRollupsForGroups(
words_per_min, lookup_hit_rate, CREATED_DATE, LAST_UPDATE_DATE
)
SELECT
CAST(s.started_at_ms / 86400000 AS INTEGER) AS rollup_day,
CAST(julianday(s.started_at_ms / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER) AS rollup_day,
s.video_id AS video_id,
COUNT(DISTINCT s.session_id) AS total_sessions,
COALESCE(SUM(t.active_watched_ms), 0) / 60000.0 AS total_active_min,
COALESCE(SUM(t.lines_seen), 0) AS total_lines_seen,
COALESCE(SUM(t.words_seen), 0) AS total_words_seen,
COALESCE(SUM(t.tokens_seen), 0) AS total_tokens_seen,
COALESCE(SUM(t.cards_mined), 0) AS total_cards,
COALESCE(SUM(sm.max_active_ms), 0) / 60000.0 AS total_active_min,
COALESCE(SUM(sm.max_lines), 0) AS total_lines_seen,
COALESCE(SUM(sm.max_words), 0) AS total_words_seen,
COALESCE(SUM(sm.max_tokens), 0) AS total_tokens_seen,
COALESCE(SUM(sm.max_cards), 0) AS total_cards,
CASE
WHEN COALESCE(SUM(t.active_watched_ms), 0) > 0
THEN (COALESCE(SUM(t.cards_mined), 0) * 60.0) / (COALESCE(SUM(t.active_watched_ms), 0) / 60000.0)
WHEN COALESCE(SUM(sm.max_active_ms), 0) > 0
THEN (COALESCE(SUM(sm.max_cards), 0) * 60.0) / (COALESCE(SUM(sm.max_active_ms), 0) / 60000.0)
ELSE NULL
END AS cards_per_hour,
CASE
WHEN COALESCE(SUM(t.active_watched_ms), 0) > 0
THEN COALESCE(SUM(t.words_seen), 0) / (COALESCE(SUM(t.active_watched_ms), 0) / 60000.0)
WHEN COALESCE(SUM(sm.max_active_ms), 0) > 0
THEN COALESCE(SUM(sm.max_words), 0) / (COALESCE(SUM(sm.max_active_ms), 0) / 60000.0)
ELSE NULL
END AS words_per_min,
CASE
WHEN COALESCE(SUM(t.lookup_count), 0) > 0
THEN CAST(COALESCE(SUM(t.lookup_hits), 0) AS REAL) / CAST(SUM(t.lookup_count) AS REAL)
WHEN COALESCE(SUM(sm.max_lookups), 0) > 0
THEN CAST(COALESCE(SUM(sm.max_hits), 0) AS REAL) / CAST(SUM(sm.max_lookups) AS REAL)
ELSE NULL
END AS lookup_hit_rate,
? AS CREATED_DATE,
? AS LAST_UPDATE_DATE
FROM imm_sessions s
JOIN imm_session_telemetry t
ON t.session_id = s.session_id
WHERE CAST(s.started_at_ms / 86400000 AS INTEGER) = ? AND s.video_id = ?
JOIN (
SELECT
t.session_id,
MAX(t.active_watched_ms) AS max_active_ms,
MAX(t.lines_seen) AS max_lines,
MAX(t.words_seen) AS max_words,
MAX(t.tokens_seen) AS max_tokens,
MAX(t.cards_mined) AS max_cards,
MAX(t.lookup_count) AS max_lookups,
MAX(t.lookup_hits) AS max_hits
FROM imm_session_telemetry t
GROUP BY t.session_id
) sm ON s.session_id = sm.session_id
WHERE CAST(julianday(s.started_at_ms / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER) = ? AND s.video_id = ?
GROUP BY rollup_day, s.video_id
ON CONFLICT (rollup_day, video_id) DO UPDATE SET
total_sessions = excluded.total_sessions,
@@ -176,20 +187,29 @@ function upsertMonthlyRollupsForGroups(
total_words_seen, total_tokens_seen, total_cards, CREATED_DATE, LAST_UPDATE_DATE
)
SELECT
CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch') AS INTEGER) AS rollup_month,
CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch', 'localtime') AS INTEGER) AS rollup_month,
s.video_id AS video_id,
COUNT(DISTINCT s.session_id) AS total_sessions,
COALESCE(SUM(t.active_watched_ms), 0) / 60000.0 AS total_active_min,
COALESCE(SUM(t.lines_seen), 0) AS total_lines_seen,
COALESCE(SUM(t.words_seen), 0) AS total_words_seen,
COALESCE(SUM(t.tokens_seen), 0) AS total_tokens_seen,
COALESCE(SUM(t.cards_mined), 0) AS total_cards,
COALESCE(SUM(sm.max_active_ms), 0) / 60000.0 AS total_active_min,
COALESCE(SUM(sm.max_lines), 0) AS total_lines_seen,
COALESCE(SUM(sm.max_words), 0) AS total_words_seen,
COALESCE(SUM(sm.max_tokens), 0) AS total_tokens_seen,
COALESCE(SUM(sm.max_cards), 0) AS total_cards,
? AS CREATED_DATE,
? AS LAST_UPDATE_DATE
FROM imm_sessions s
JOIN imm_session_telemetry t
ON t.session_id = s.session_id
WHERE CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch') AS INTEGER) = ? AND s.video_id = ?
JOIN (
SELECT
t.session_id,
MAX(t.active_watched_ms) AS max_active_ms,
MAX(t.lines_seen) AS max_lines,
MAX(t.words_seen) AS max_words,
MAX(t.tokens_seen) AS max_tokens,
MAX(t.cards_mined) AS max_cards
FROM imm_session_telemetry t
GROUP BY t.session_id
) sm ON s.session_id = sm.session_id
WHERE CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch', 'localtime') AS INTEGER) = ? AND s.video_id = ?
GROUP BY rollup_month, s.video_id
ON CONFLICT (rollup_month, video_id) DO UPDATE SET
total_sessions = excluded.total_sessions,
@@ -216,8 +236,8 @@ function getAffectedRollupGroups(
.prepare(
`
SELECT DISTINCT
CAST(s.started_at_ms / 86400000 AS INTEGER) AS rollup_day,
CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch') AS INTEGER) AS rollup_month,
CAST(julianday(s.started_at_ms / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER) AS rollup_day,
CAST(strftime('%Y%m', s.started_at_ms / 1000, 'unixepoch', 'localtime') AS INTEGER) AS rollup_month,
s.video_id AS video_id
FROM imm_session_telemetry t
JOIN imm_sessions s

View File

@@ -4,7 +4,7 @@ import { EventEmitter } from 'node:events';
import test from 'node:test';
import type { spawn as spawnFn } from 'node:child_process';
import { SOURCE_TYPE_LOCAL } from './types';
import { getLocalVideoMetadata, runFfprobe } from './metadata';
import { getLocalVideoMetadata, guessAnimeVideoMetadata, runFfprobe } from './metadata';
type Spawn = typeof spawnFn;
@@ -146,3 +146,79 @@ test('getLocalVideoMetadata derives title and falls back to null hash on read er
assert.equal(hashFallbackMetadata.canonicalTitle, 'Episode 02');
assert.equal(hashFallbackMetadata.hashSha256, null);
});
test('guessAnimeVideoMetadata uses guessit basename output first when available', async () => {
const seenTargets: string[] = [];
const parsed = await guessAnimeVideoMetadata('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5', {
runGuessit: async (target) => {
seenTargets.push(target);
return JSON.stringify({
title: 'Little Witch Academia',
season: 2,
episode: 5,
});
},
});
assert.deepEqual(seenTargets, ['Little Witch Academia S02E05.mkv']);
assert.deepEqual(parsed, {
parsedBasename: 'Little Witch Academia S02E05.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 5,
parserSource: 'guessit',
parserConfidence: 1,
parseMetadataJson: JSON.stringify({
filename: 'Little Witch Academia S02E05.mkv',
source: 'guessit',
}),
});
});
test('guessAnimeVideoMetadata falls back to parser when guessit throws', async () => {
const parsed = await guessAnimeVideoMetadata('/tmp/Little Witch Academia S02E05.mkv', 'Episode 5', {
runGuessit: async () => {
throw new Error('guessit unavailable');
},
});
assert.deepEqual(parsed, {
parsedBasename: 'Little Witch Academia S02E05.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 5,
parserSource: 'fallback',
parserConfidence: 1,
parseMetadataJson: JSON.stringify({
confidence: 'high',
filename: 'Little Witch Academia S02E05.mkv',
rawTitle: 'Little Witch Academia S02E05',
source: 'fallback',
}),
});
});
test('guessAnimeVideoMetadata falls back when guessit output is incomplete', async () => {
const parsed = await guessAnimeVideoMetadata(
'/tmp/[SubsPlease] Frieren - 03 (1080p).mkv',
null,
{
runGuessit: async () => JSON.stringify({ episode: 3 }),
},
);
assert.deepEqual(parsed, {
parsedBasename: '[SubsPlease] Frieren - 03 (1080p).mkv',
parsedTitle: 'Frieren - 03 (1080p)',
parsedSeason: null,
parsedEpisode: null,
parserSource: 'fallback',
parserConfidence: 0.2,
parseMetadataJson: JSON.stringify({
confidence: 'low',
filename: '[SubsPlease] Frieren - 03 (1080p).mkv',
rawTitle: 'Frieren - 03 (1080p)',
source: 'fallback',
}),
});
});

View File

@@ -1,6 +1,13 @@
import crypto from 'node:crypto';
import { spawn as nodeSpawn } from 'node:child_process';
import * as fs from 'node:fs';
import path from 'node:path';
import { parseMediaInfo } from '../../../jimaku/utils';
import {
guessAnilistMediaInfo,
runGuessit,
type GuessAnilistMediaInfoDeps,
} from '../anilist/anilist-updater';
import {
deriveCanonicalTitle,
emptyMetadata,
@@ -8,7 +15,12 @@ import {
parseFps,
toNullableInt,
} from './reducer';
import { SOURCE_TYPE_LOCAL, type ProbeMetadata, type VideoMetadata } from './types';
import {
SOURCE_TYPE_LOCAL,
type ParsedAnimeVideoGuess,
type ProbeMetadata,
type VideoMetadata,
} from './types';
type SpawnFn = typeof nodeSpawn;
@@ -24,6 +36,21 @@ interface MetadataDeps {
fs?: FsDeps;
}
interface GuessAnimeVideoMetadataDeps {
runGuessit?: GuessAnilistMediaInfoDeps['runGuessit'];
}
function mapParserConfidenceToScore(confidence: 'high' | 'medium' | 'low'): number {
switch (confidence) {
case 'high':
return 1;
case 'medium':
return 0.6;
default:
return 0.2;
}
}
export async function computeSha256(
mediaPath: string,
deps: MetadataDeps = {},
@@ -151,3 +178,48 @@ export async function getLocalVideoMetadata(
metadataJson: null,
};
}
export async function guessAnimeVideoMetadata(
mediaPath: string | null,
mediaTitle: string | null,
deps: GuessAnimeVideoMetadataDeps = {},
): Promise<ParsedAnimeVideoGuess | null> {
const parsed = await guessAnilistMediaInfo(mediaPath, mediaTitle, {
runGuessit: deps.runGuessit ?? runGuessit,
});
if (!parsed) {
return null;
}
const parsedBasename = mediaPath ? path.basename(mediaPath) : null;
if (parsed.source === 'guessit') {
return {
parsedBasename,
parsedTitle: parsed.title,
parsedSeason: parsed.season,
parsedEpisode: parsed.episode,
parserSource: 'guessit',
parserConfidence: 1,
parseMetadataJson: JSON.stringify({
filename: parsedBasename,
source: 'guessit',
}),
};
}
const fallbackInfo = parseMediaInfo(mediaPath ?? mediaTitle);
return {
parsedBasename: parsedBasename ?? fallbackInfo.filename ?? null,
parsedTitle: parsed.title,
parsedSeason: parsed.season,
parsedEpisode: parsed.episode,
parserSource: 'fallback',
parserConfidence: mapParserConfidenceToScore(fallbackInfo.confidence),
parseMetadataJson: JSON.stringify({
confidence: fallbackInfo.confidence,
filename: fallbackInfo.filename,
rawTitle: fallbackInfo.rawTitle,
source: 'fallback',
}),
};
}

File diff suppressed because it is too large Load Diff

View File

@@ -30,6 +30,7 @@ export function createInitialSessionState(
lastPauseStartMs: null,
isPaused: false,
pendingTelemetry: true,
markedWatched: false,
};
}

View File

@@ -9,7 +9,9 @@ import {
createTrackerPreparedStatements,
ensureSchema,
executeQueuedWrite,
getOrCreateAnimeRecord,
getOrCreateVideoRecord,
linkVideoToAnimeRecord,
} from './storage';
import { EVENT_SUBTITLE_LINE, SESSION_STATUS_ENDED, SOURCE_TYPE_LOCAL } from './types';
@@ -60,6 +62,7 @@ test('ensureSchema creates immersion core tables', () => {
const tableNames = new Set(rows.map((row) => row.name));
assert.ok(tableNames.has('imm_videos'));
assert.ok(tableNames.has('imm_anime'));
assert.ok(tableNames.has('imm_sessions'));
assert.ok(tableNames.has('imm_session_telemetry'));
assert.ok(tableNames.has('imm_session_events'));
@@ -67,8 +70,28 @@ test('ensureSchema creates immersion core tables', () => {
assert.ok(tableNames.has('imm_monthly_rollups'));
assert.ok(tableNames.has('imm_words'));
assert.ok(tableNames.has('imm_kanji'));
assert.ok(tableNames.has('imm_subtitle_lines'));
assert.ok(tableNames.has('imm_word_line_occurrences'));
assert.ok(tableNames.has('imm_kanji_line_occurrences'));
assert.ok(tableNames.has('imm_rollup_state'));
const videoColumns = new Set(
(
db.prepare('PRAGMA table_info(imm_videos)').all() as Array<{
name: string;
}>
).map((row) => row.name),
);
assert.ok(videoColumns.has('anime_id'));
assert.ok(videoColumns.has('parsed_basename'));
assert.ok(videoColumns.has('parsed_title'));
assert.ok(videoColumns.has('parsed_season'));
assert.ok(videoColumns.has('parsed_episode'));
assert.ok(videoColumns.has('parser_source'));
assert.ok(videoColumns.has('parser_confidence'));
assert.ok(videoColumns.has('parse_metadata_json'));
const rollupStateRow = db
.prepare('SELECT state_value FROM imm_rollup_state WHERE state_key = ?')
.get('last_rollup_sample_ms') as {
@@ -82,6 +105,470 @@ test('ensureSchema creates immersion core tables', () => {
}
});
test('ensureSchema migrates legacy videos and backfills anime metadata from filenames', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
db.exec(`
CREATE TABLE imm_schema_version (
schema_version INTEGER PRIMARY KEY,
applied_at_ms INTEGER NOT NULL
);
INSERT INTO imm_schema_version(schema_version, applied_at_ms) VALUES (4, 1);
CREATE TABLE imm_videos(
video_id INTEGER PRIMARY KEY AUTOINCREMENT,
video_key TEXT NOT NULL UNIQUE,
canonical_title TEXT NOT NULL,
source_type INTEGER NOT NULL,
source_path TEXT,
source_url TEXT,
duration_ms INTEGER NOT NULL CHECK(duration_ms>=0),
file_size_bytes INTEGER CHECK(file_size_bytes>=0),
codec_id INTEGER, container_id INTEGER,
width_px INTEGER, height_px INTEGER, fps_x100 INTEGER,
bitrate_kbps INTEGER, audio_codec_id INTEGER,
hash_sha256 TEXT, screenshot_path TEXT,
metadata_json TEXT,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
);
`);
const insertLegacyVideo = db.prepare(`
INSERT INTO imm_videos (
video_key, canonical_title, source_type, source_path, source_url,
duration_ms, file_size_bytes, codec_id, container_id, width_px, height_px,
fps_x100, bitrate_kbps, audio_codec_id, hash_sha256, screenshot_path,
metadata_json, CREATED_DATE, LAST_UPDATE_DATE
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
insertLegacyVideo.run(
'local:/library/Little Witch Academia S02E05.mkv',
'Episode 5',
SOURCE_TYPE_LOCAL,
'/library/Little Witch Academia S02E05.mkv',
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
1,
1,
);
insertLegacyVideo.run(
'local:/library/Little Witch Academia S02E06.mkv',
'Episode 6',
SOURCE_TYPE_LOCAL,
'/library/Little Witch Academia S02E06.mkv',
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
1,
1,
);
insertLegacyVideo.run(
'local:/library/[SubsPlease] Frieren - 03 - Departure.mkv',
'Episode 3',
SOURCE_TYPE_LOCAL,
'/library/[SubsPlease] Frieren - 03 - Departure.mkv',
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
1,
1,
);
ensureSchema(db);
const videoColumns = new Set(
(
db.prepare('PRAGMA table_info(imm_videos)').all() as Array<{
name: string;
}>
).map((row) => row.name),
);
assert.ok(videoColumns.has('anime_id'));
assert.ok(videoColumns.has('parsed_basename'));
assert.ok(videoColumns.has('parsed_title'));
assert.ok(videoColumns.has('parsed_season'));
assert.ok(videoColumns.has('parsed_episode'));
assert.ok(videoColumns.has('parser_source'));
assert.ok(videoColumns.has('parser_confidence'));
assert.ok(videoColumns.has('parse_metadata_json'));
const animeRows = db
.prepare('SELECT canonical_title FROM imm_anime ORDER BY canonical_title')
.all() as Array<{ canonical_title: string }>;
assert.deepEqual(
animeRows.map((row) => row.canonical_title),
['Frieren', 'Little Witch Academia'],
);
const littleWitchRows = db
.prepare(
`
SELECT
a.canonical_title AS anime_title,
v.parsed_title,
v.parsed_basename,
v.parsed_season,
v.parsed_episode,
v.parser_source,
v.parser_confidence
FROM imm_videos v
JOIN imm_anime a ON a.anime_id = v.anime_id
WHERE v.video_key LIKE 'local:/library/Little Witch Academia%'
ORDER BY v.video_key
`,
)
.all() as Array<{
anime_title: string;
parsed_title: string | null;
parsed_basename: string | null;
parsed_season: number | null;
parsed_episode: number | null;
parser_source: string | null;
parser_confidence: number | null;
}>;
assert.equal(littleWitchRows.length, 2);
assert.deepEqual(
littleWitchRows.map((row) => ({
animeTitle: row.anime_title,
parsedTitle: row.parsed_title,
parsedBasename: row.parsed_basename,
parsedSeason: row.parsed_season,
parsedEpisode: row.parsed_episode,
parserSource: row.parser_source,
})),
[
{
animeTitle: 'Little Witch Academia',
parsedTitle: 'Little Witch Academia',
parsedBasename: 'Little Witch Academia S02E05.mkv',
parsedSeason: 2,
parsedEpisode: 5,
parserSource: 'fallback',
},
{
animeTitle: 'Little Witch Academia',
parsedTitle: 'Little Witch Academia',
parsedBasename: 'Little Witch Academia S02E06.mkv',
parsedSeason: 2,
parsedEpisode: 6,
parserSource: 'fallback',
},
],
);
assert.ok(
littleWitchRows.every(
(row) => typeof row.parser_confidence === 'number' && row.parser_confidence > 0,
),
);
const frierenRow = db
.prepare(
`
SELECT
a.canonical_title AS anime_title,
v.parsed_title,
v.parsed_episode,
v.parser_source
FROM imm_videos v
JOIN imm_anime a ON a.anime_id = v.anime_id
WHERE v.video_key = ?
`,
)
.get('local:/library/[SubsPlease] Frieren - 03 - Departure.mkv') as {
anime_title: string;
parsed_title: string | null;
parsed_episode: number | null;
parser_source: string | null;
} | null;
assert.ok(frierenRow);
assert.equal(frierenRow?.anime_title, 'Frieren');
assert.equal(frierenRow?.parsed_title, 'Frieren');
assert.equal(frierenRow?.parsed_episode, 3);
assert.equal(frierenRow?.parser_source, 'fallback');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('ensureSchema adds subtitle-line occurrence tables to schema version 6 databases', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
db.exec(`
CREATE TABLE imm_schema_version (
schema_version INTEGER PRIMARY KEY,
applied_at_ms INTEGER NOT NULL
);
INSERT INTO imm_schema_version(schema_version, applied_at_ms) VALUES (6, 1);
CREATE TABLE imm_videos(
video_id INTEGER PRIMARY KEY AUTOINCREMENT,
video_key TEXT NOT NULL UNIQUE,
anime_id INTEGER,
canonical_title TEXT NOT NULL,
source_type INTEGER NOT NULL,
source_path TEXT,
source_url TEXT,
parsed_basename TEXT,
parsed_title TEXT,
parsed_season INTEGER,
parsed_episode INTEGER,
parser_source TEXT,
parser_confidence REAL,
parse_metadata_json TEXT,
duration_ms INTEGER NOT NULL CHECK(duration_ms>=0),
file_size_bytes INTEGER CHECK(file_size_bytes>=0),
codec_id INTEGER, container_id INTEGER,
width_px INTEGER, height_px INTEGER, fps_x100 INTEGER,
bitrate_kbps INTEGER, audio_codec_id INTEGER,
hash_sha256 TEXT, screenshot_path TEXT,
metadata_json TEXT,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
);
CREATE TABLE imm_sessions(
session_id INTEGER PRIMARY KEY AUTOINCREMENT,
session_uuid TEXT NOT NULL UNIQUE,
video_id INTEGER NOT NULL,
started_at_ms INTEGER NOT NULL,
ended_at_ms INTEGER,
status INTEGER NOT NULL,
locale_id INTEGER,
target_lang_id INTEGER,
difficulty_tier INTEGER,
subtitle_mode INTEGER,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
);
CREATE TABLE imm_session_events(
event_id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL,
ts_ms INTEGER NOT NULL,
event_type INTEGER NOT NULL,
line_index INTEGER,
segment_start_ms INTEGER,
segment_end_ms INTEGER,
words_delta INTEGER NOT NULL DEFAULT 0,
cards_delta INTEGER NOT NULL DEFAULT 0,
payload_json TEXT,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
);
CREATE TABLE imm_words(
id INTEGER PRIMARY KEY AUTOINCREMENT,
headword TEXT,
word TEXT,
reading TEXT,
part_of_speech TEXT,
pos1 TEXT,
pos2 TEXT,
pos3 TEXT,
first_seen REAL,
last_seen REAL,
frequency INTEGER,
UNIQUE(headword, word, reading)
);
CREATE TABLE imm_kanji(
id INTEGER PRIMARY KEY AUTOINCREMENT,
kanji TEXT,
first_seen REAL,
last_seen REAL,
frequency INTEGER,
UNIQUE(kanji)
);
CREATE TABLE imm_rollup_state(
state_key TEXT PRIMARY KEY,
state_value INTEGER NOT NULL
);
`);
ensureSchema(db);
const tableNames = new Set(
(
db.prepare(`SELECT name FROM sqlite_master WHERE type = 'table' AND name LIKE 'imm_%'`).all() as
Array<{ name: string }>
).map((row) => row.name),
);
assert.ok(tableNames.has('imm_subtitle_lines'));
assert.ok(tableNames.has('imm_word_line_occurrences'));
assert.ok(tableNames.has('imm_kanji_line_occurrences'));
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('anime rows are reused by normalized parsed title and upgraded with AniList metadata', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const firstVideoId = getOrCreateVideoRecord(db, 'local:/tmp/lwa-s02e05.mkv', {
canonicalTitle: 'Episode 5',
sourcePath: '/tmp/Little Witch Academia S02E05.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const secondVideoId = getOrCreateVideoRecord(db, 'local:/tmp/lwa-s02e06.mkv', {
canonicalTitle: 'Episode 6',
sourcePath: '/tmp/Little Witch Academia S02E06.mkv',
sourceUrl: null,
sourceType: SOURCE_TYPE_LOCAL,
});
const provisionalAnimeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Little Witch Academia',
canonicalTitle: 'Little Witch Academia',
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: '{"source":"parsed"}',
});
linkVideoToAnimeRecord(db, firstVideoId, {
animeId: provisionalAnimeId,
parsedBasename: 'Little Witch Academia S02E05.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 5,
parserSource: 'fallback',
parserConfidence: 0.6,
parseMetadataJson: '{"source":"parsed","episode":5}',
});
const reusedAnimeId = getOrCreateAnimeRecord(db, {
parsedTitle: ' little witch academia ',
canonicalTitle: 'Little Witch Academia',
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: '{"source":"parsed"}',
});
linkVideoToAnimeRecord(db, secondVideoId, {
animeId: reusedAnimeId,
parsedBasename: 'Little Witch Academia S02E06.mkv',
parsedTitle: 'Little Witch Academia',
parsedSeason: 2,
parsedEpisode: 6,
parserSource: 'fallback',
parserConfidence: 0.6,
parseMetadataJson: '{"source":"parsed","episode":6}',
});
assert.equal(reusedAnimeId, provisionalAnimeId);
const upgradedAnimeId = getOrCreateAnimeRecord(db, {
parsedTitle: 'Little Witch Academia',
canonicalTitle: 'Little Witch Academia TV',
anilistId: 33_435,
titleRomaji: 'Little Witch Academia',
titleEnglish: 'Little Witch Academia',
titleNative: 'リトルウィッチアカデミア',
metadataJson: '{"source":"anilist"}',
});
assert.equal(upgradedAnimeId, provisionalAnimeId);
const animeRows = db.prepare('SELECT * FROM imm_anime').all() as Array<{
anime_id: number;
normalized_title_key: string;
canonical_title: string;
anilist_id: number | null;
title_romaji: string | null;
title_english: string | null;
title_native: string | null;
metadata_json: string | null;
}>;
assert.equal(animeRows.length, 1);
assert.equal(animeRows[0]?.anime_id, provisionalAnimeId);
assert.equal(animeRows[0]?.normalized_title_key, 'little witch academia');
assert.equal(animeRows[0]?.canonical_title, 'Little Witch Academia TV');
assert.equal(animeRows[0]?.anilist_id, 33_435);
assert.equal(animeRows[0]?.title_romaji, 'Little Witch Academia');
assert.equal(animeRows[0]?.title_english, 'Little Witch Academia');
assert.equal(animeRows[0]?.title_native, 'リトルウィッチアカデミア');
assert.equal(animeRows[0]?.metadata_json, '{"source":"anilist"}');
const linkedVideos = db
.prepare(
`
SELECT anime_id, parsed_title, parsed_season, parsed_episode
FROM imm_videos
WHERE video_id IN (?, ?)
ORDER BY video_id
`,
)
.all(firstVideoId, secondVideoId) as Array<{
anime_id: number | null;
parsed_title: string | null;
parsed_season: number | null;
parsed_episode: number | null;
}>;
assert.deepEqual(linkedVideos, [
{
anime_id: provisionalAnimeId,
parsed_title: 'Little Witch Academia',
parsed_season: 2,
parsed_episode: 5,
},
{
anime_id: provisionalAnimeId,
parsed_title: 'Little Witch Academia',
parsed_season: 2,
parsed_episode: 6,
},
]);
} finally {
db.close();
cleanupDbPath(dbPath);
}
});
test('start/finalize session updates ended_at and status', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
@@ -191,18 +678,22 @@ test('executeQueuedWrite inserts and upserts word and kanji rows', () => {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
stmts.wordUpsertStmt.run('猫', '猫', '', 10.0, 10.0);
stmts.wordUpsertStmt.run('猫', '猫', '', 5.0, 15.0);
stmts.wordUpsertStmt.run('猫', '猫', '', 'noun', '名詞', '一般', '', 10.0, 10.0);
stmts.wordUpsertStmt.run('猫', '猫', '', 'noun', '名詞', '一般', '', 5.0, 15.0);
stmts.kanjiUpsertStmt.run('日', 9.0, 9.0);
stmts.kanjiUpsertStmt.run('日', 8.0, 11.0);
const wordRow = db
.prepare(
'SELECT headword, frequency, first_seen, last_seen FROM imm_words WHERE headword = ?',
`SELECT headword, frequency, part_of_speech, pos1, pos2, first_seen, last_seen
FROM imm_words WHERE headword = ?`,
)
.get('猫') as {
headword: string;
frequency: number;
part_of_speech: string;
pos1: string;
pos2: string;
first_seen: number;
last_seen: number;
} | null;
@@ -218,6 +709,9 @@ test('executeQueuedWrite inserts and upserts word and kanji rows', () => {
assert.ok(wordRow);
assert.ok(kanjiRow);
assert.equal(wordRow?.frequency, 2);
assert.equal(wordRow?.part_of_speech, 'noun');
assert.equal(wordRow?.pos1, '名詞');
assert.equal(wordRow?.pos2, '一般');
assert.equal(kanjiRow?.frequency, 2);
assert.equal(wordRow?.first_seen, 5);
assert.equal(wordRow?.last_seen, 15);
@@ -228,3 +722,34 @@ test('executeQueuedWrite inserts and upserts word and kanji rows', () => {
cleanupDbPath(dbPath);
}
});
test('word upsert replaces legacy other part_of_speech when better POS metadata arrives later', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const stmts = createTrackerPreparedStatements(db);
stmts.wordUpsertStmt.run('知っている', '知っている', 'しっている', 'other', '動詞', '自立', '', 10, 10);
stmts.wordUpsertStmt.run('知っている', '知っている', 'しっている', 'verb', '動詞', '自立', '', 11, 12);
const row = db
.prepare('SELECT frequency, part_of_speech, pos1, pos2 FROM imm_words WHERE headword = ?')
.get('知っている') as {
frequency: number;
part_of_speech: string;
pos1: string;
pos2: string;
} | null;
assert.ok(row);
assert.equal(row?.frequency, 2);
assert.equal(row?.part_of_speech, 'verb');
assert.equal(row?.pos1, '動詞');
assert.equal(row?.pos2, '自立');
} finally {
db.close();
cleanupDbPath(dbPath);
}
});

View File

@@ -1,3 +1,4 @@
import { parseMediaInfo } from '../../../jimaku/utils';
import type { DatabaseSync } from './sqlite';
import { SCHEMA_VERSION } from './types';
import type { QueuedWrite, VideoMetadata } from './types';
@@ -7,6 +8,33 @@ export interface TrackerPreparedStatements {
eventInsertStmt: ReturnType<DatabaseSync['prepare']>;
wordUpsertStmt: ReturnType<DatabaseSync['prepare']>;
kanjiUpsertStmt: ReturnType<DatabaseSync['prepare']>;
subtitleLineInsertStmt: ReturnType<DatabaseSync['prepare']>;
wordIdSelectStmt: ReturnType<DatabaseSync['prepare']>;
kanjiIdSelectStmt: ReturnType<DatabaseSync['prepare']>;
wordLineOccurrenceUpsertStmt: ReturnType<DatabaseSync['prepare']>;
kanjiLineOccurrenceUpsertStmt: ReturnType<DatabaseSync['prepare']>;
videoAnimeIdSelectStmt: ReturnType<DatabaseSync['prepare']>;
}
export interface AnimeRecordInput {
parsedTitle: string;
canonicalTitle: string;
anilistId: number | null;
titleRomaji: string | null;
titleEnglish: string | null;
titleNative: string | null;
metadataJson: string | null;
}
export interface VideoAnimeLinkInput {
animeId: number | null;
parsedBasename: string | null;
parsedTitle: string | null;
parsedSeason: number | null;
parsedEpisode: number | null;
parserSource: string | null;
parserConfidence: number | null;
parseMetadataJson: string | null;
}
function hasColumn(db: DatabaseSync, tableName: string, columnName: string): boolean {
@@ -16,9 +44,14 @@ function hasColumn(db: DatabaseSync, tableName: string, columnName: string): boo
.some((row: unknown) => (row as { name: string }).name === columnName);
}
function addColumnIfMissing(db: DatabaseSync, tableName: string, columnName: string): void {
function addColumnIfMissing(
db: DatabaseSync,
tableName: string,
columnName: string,
columnType = 'INTEGER',
): void {
if (!hasColumn(db, tableName, columnName)) {
db.exec(`ALTER TABLE ${tableName} ADD COLUMN ${columnName} INTEGER`);
db.exec(`ALTER TABLE ${tableName} ADD COLUMN ${columnName} ${columnType}`);
}
}
@@ -35,6 +68,247 @@ export function applyPragmas(db: DatabaseSync): void {
db.exec('PRAGMA busy_timeout = 2500');
}
export function normalizeAnimeIdentityKey(title: string): string {
return title
.normalize('NFKC')
.toLowerCase()
.replace(/[^\p{L}\p{N}]+/gu, ' ')
.trim()
.replace(/\s+/g, ' ');
}
function looksLikeEpisodeOnlyTitle(title: string): boolean {
const normalized = title
.normalize('NFKC')
.toLowerCase()
.replace(/\s+/g, ' ')
.trim();
return /^(episode|ep)\s*\d{1,3}$/.test(normalized) || /^第\s*\d{1,3}\s*話$/.test(normalized);
}
function parserConfidenceToScore(confidence: 'high' | 'medium' | 'low'): number {
switch (confidence) {
case 'high':
return 1;
case 'medium':
return 0.6;
default:
return 0.2;
}
}
function parseLegacyAnimeBackfillCandidate(
sourcePath: string | null,
canonicalTitle: string,
): {
basename: string | null;
title: string;
season: number | null;
episode: number | null;
source: 'fallback';
confidenceScore: number;
metadataJson: string;
} | null {
const fromPath =
sourcePath && sourcePath.trim().length > 0 ? parseMediaInfo(sourcePath.trim()) : null;
if (fromPath?.title && !looksLikeEpisodeOnlyTitle(fromPath.title)) {
return {
basename: fromPath.filename || null,
title: fromPath.title,
season: fromPath.season,
episode: fromPath.episode,
source: 'fallback',
confidenceScore: parserConfidenceToScore(fromPath.confidence),
metadataJson: JSON.stringify({
confidence: fromPath.confidence,
filename: fromPath.filename,
rawTitle: fromPath.rawTitle,
migrationSource: 'source_path',
}),
};
}
const fallbackTitle = canonicalTitle.trim();
if (!fallbackTitle) return null;
const fromTitle = parseMediaInfo(fallbackTitle);
if (!fromTitle.title || looksLikeEpisodeOnlyTitle(fromTitle.title)) {
return null;
}
return {
basename: null,
title: fromTitle.title,
season: fromTitle.season,
episode: fromTitle.episode,
source: 'fallback',
confidenceScore: parserConfidenceToScore(fromTitle.confidence),
metadataJson: JSON.stringify({
confidence: fromTitle.confidence,
filename: fromTitle.filename,
rawTitle: fromTitle.rawTitle,
migrationSource: 'canonical_title',
}),
};
}
export function getOrCreateAnimeRecord(db: DatabaseSync, input: AnimeRecordInput): number {
const normalizedTitleKey = normalizeAnimeIdentityKey(input.parsedTitle);
if (!normalizedTitleKey) {
throw new Error('parsedTitle is required to create or update an anime record');
}
const byAnilistId =
input.anilistId !== null
? (db.prepare('SELECT anime_id FROM imm_anime WHERE anilist_id = ?').get(input.anilistId) as {
anime_id: number;
} | null)
: null;
const byNormalizedTitle = db
.prepare('SELECT anime_id FROM imm_anime WHERE normalized_title_key = ?')
.get(normalizedTitleKey) as { anime_id: number } | null;
const existing = byAnilistId ?? byNormalizedTitle;
if (existing?.anime_id) {
db.prepare(
`
UPDATE imm_anime
SET
canonical_title = COALESCE(NULLIF(?, ''), canonical_title),
anilist_id = COALESCE(?, anilist_id),
title_romaji = COALESCE(?, title_romaji),
title_english = COALESCE(?, title_english),
title_native = COALESCE(?, title_native),
metadata_json = COALESCE(?, metadata_json),
LAST_UPDATE_DATE = ?
WHERE anime_id = ?
`,
).run(
input.canonicalTitle,
input.anilistId,
input.titleRomaji,
input.titleEnglish,
input.titleNative,
input.metadataJson,
Date.now(),
existing.anime_id,
);
return existing.anime_id;
}
const nowMs = Date.now();
const result = db
.prepare(
`
INSERT INTO imm_anime(
normalized_title_key,
canonical_title,
anilist_id,
title_romaji,
title_english,
title_native,
metadata_json,
CREATED_DATE,
LAST_UPDATE_DATE
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
`,
)
.run(
normalizedTitleKey,
input.canonicalTitle,
input.anilistId,
input.titleRomaji,
input.titleEnglish,
input.titleNative,
input.metadataJson,
nowMs,
nowMs,
);
return Number(result.lastInsertRowid);
}
export function linkVideoToAnimeRecord(
db: DatabaseSync,
videoId: number,
input: VideoAnimeLinkInput,
): void {
db.prepare(
`
UPDATE imm_videos
SET
anime_id = ?,
parsed_basename = ?,
parsed_title = ?,
parsed_season = ?,
parsed_episode = ?,
parser_source = ?,
parser_confidence = ?,
parse_metadata_json = ?,
LAST_UPDATE_DATE = ?
WHERE video_id = ?
`,
).run(
input.animeId,
input.parsedBasename,
input.parsedTitle,
input.parsedSeason,
input.parsedEpisode,
input.parserSource,
input.parserConfidence,
input.parseMetadataJson,
Date.now(),
videoId,
);
}
function migrateLegacyAnimeMetadata(db: DatabaseSync): void {
addColumnIfMissing(db, 'imm_videos', 'anime_id', 'INTEGER REFERENCES imm_anime(anime_id)');
addColumnIfMissing(db, 'imm_videos', 'parsed_basename', 'TEXT');
addColumnIfMissing(db, 'imm_videos', 'parsed_title', 'TEXT');
addColumnIfMissing(db, 'imm_videos', 'parsed_season', 'INTEGER');
addColumnIfMissing(db, 'imm_videos', 'parsed_episode', 'INTEGER');
addColumnIfMissing(db, 'imm_videos', 'parser_source', 'TEXT');
addColumnIfMissing(db, 'imm_videos', 'parser_confidence', 'REAL');
addColumnIfMissing(db, 'imm_videos', 'parse_metadata_json', 'TEXT');
const legacyRows = db
.prepare(
`
SELECT video_id, source_path, canonical_title
FROM imm_videos
WHERE anime_id IS NULL
`,
)
.all() as Array<{
video_id: number;
source_path: string | null;
canonical_title: string;
}>;
for (const row of legacyRows) {
const parsed = parseLegacyAnimeBackfillCandidate(row.source_path, row.canonical_title);
if (!parsed) continue;
const animeId = getOrCreateAnimeRecord(db, {
parsedTitle: parsed.title,
canonicalTitle: parsed.title,
anilistId: null,
titleRomaji: null,
titleEnglish: null,
titleNative: null,
metadataJson: parsed.metadataJson,
});
linkVideoToAnimeRecord(db, row.video_id, {
animeId,
parsedBasename: parsed.basename,
parsedTitle: parsed.title,
parsedSeason: parsed.season,
parsedEpisode: parsed.episode,
parserSource: parsed.source,
parserConfidence: parsed.confidenceScore,
parseMetadataJson: parsed.metadataJson,
});
}
}
export function ensureSchema(db: DatabaseSync): void {
db.exec(`
CREATE TABLE IF NOT EXISTS imm_schema_version (
@@ -61,14 +335,38 @@ export function ensureSchema(db: DatabaseSync): void {
return;
}
db.exec(`
CREATE TABLE IF NOT EXISTS imm_anime(
anime_id INTEGER PRIMARY KEY AUTOINCREMENT,
normalized_title_key TEXT NOT NULL UNIQUE,
canonical_title TEXT NOT NULL,
anilist_id INTEGER UNIQUE,
title_romaji TEXT,
title_english TEXT,
title_native TEXT,
episodes_total INTEGER,
metadata_json TEXT,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
);
`);
db.exec(`
CREATE TABLE IF NOT EXISTS imm_videos(
video_id INTEGER PRIMARY KEY AUTOINCREMENT,
video_key TEXT NOT NULL UNIQUE,
anime_id INTEGER,
canonical_title TEXT NOT NULL,
source_type INTEGER NOT NULL,
source_path TEXT,
source_url TEXT,
parsed_basename TEXT,
parsed_title TEXT,
parsed_season INTEGER,
parsed_episode INTEGER,
parser_source TEXT,
parser_confidence REAL,
parse_metadata_json TEXT,
watched INTEGER NOT NULL DEFAULT 0,
duration_ms INTEGER NOT NULL CHECK(duration_ms>=0),
file_size_bytes INTEGER CHECK(file_size_bytes>=0),
codec_id INTEGER, container_id INTEGER,
@@ -77,7 +375,8 @@ export function ensureSchema(db: DatabaseSync): void {
hash_sha256 TEXT, screenshot_path TEXT,
metadata_json TEXT,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER
LAST_UPDATE_DATE INTEGER,
FOREIGN KEY(anime_id) REFERENCES imm_anime(anime_id) ON DELETE SET NULL
);
`);
db.exec(`
@@ -173,6 +472,10 @@ export function ensureSchema(db: DatabaseSync): void {
headword TEXT,
word TEXT,
reading TEXT,
part_of_speech TEXT,
pos1 TEXT,
pos2 TEXT,
pos3 TEXT,
first_seen REAL,
last_seen REAL,
frequency INTEGER,
@@ -189,42 +492,59 @@ export function ensureSchema(db: DatabaseSync): void {
UNIQUE(kanji)
);
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_sessions_video_started
ON imm_sessions(video_id, started_at_ms DESC)
CREATE TABLE IF NOT EXISTS imm_subtitle_lines(
line_id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL,
event_id INTEGER,
video_id INTEGER NOT NULL,
anime_id INTEGER,
line_index INTEGER NOT NULL,
segment_start_ms INTEGER,
segment_end_ms INTEGER,
text TEXT NOT NULL,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER,
FOREIGN KEY(session_id) REFERENCES imm_sessions(session_id) ON DELETE CASCADE,
FOREIGN KEY(event_id) REFERENCES imm_session_events(event_id) ON DELETE SET NULL,
FOREIGN KEY(video_id) REFERENCES imm_videos(video_id) ON DELETE CASCADE,
FOREIGN KEY(anime_id) REFERENCES imm_anime(anime_id) ON DELETE SET NULL
);
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_sessions_status_started
ON imm_sessions(status, started_at_ms DESC)
CREATE TABLE IF NOT EXISTS imm_word_line_occurrences(
line_id INTEGER NOT NULL,
word_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
PRIMARY KEY(line_id, word_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
);
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_telemetry_session_sample
ON imm_session_telemetry(session_id, sample_ms DESC)
CREATE TABLE IF NOT EXISTS imm_kanji_line_occurrences(
line_id INTEGER NOT NULL,
kanji_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
PRIMARY KEY(line_id, kanji_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
);
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_events_session_ts
ON imm_session_events(session_id, ts_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_events_type_ts
ON imm_session_events(event_type, ts_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_rollups_day_video
ON imm_daily_rollups(rollup_day, video_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_rollups_month_video
ON imm_monthly_rollups(rollup_month, video_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_words_headword_word_reading
ON imm_words(headword, word, reading)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_kanji_kanji
ON imm_kanji(kanji)
CREATE TABLE IF NOT EXISTS imm_media_art(
video_id INTEGER PRIMARY KEY,
anilist_id INTEGER,
cover_url TEXT,
cover_blob BLOB,
title_romaji TEXT,
title_english TEXT,
episodes_total INTEGER,
fetched_at_ms INTEGER NOT NULL,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER,
FOREIGN KEY(video_id) REFERENCES imm_videos(video_id) ON DELETE CASCADE
);
`);
if (currentVersion?.schema_version === 1) {
@@ -299,6 +619,134 @@ export function ensureSchema(db: DatabaseSync): void {
dropColumnIfExists(db, 'imm_sessions', 'updated_at_ms');
}
if (currentVersion?.schema_version && currentVersion.schema_version < 5) {
migrateLegacyAnimeMetadata(db);
}
if (currentVersion?.schema_version && currentVersion.schema_version < 6) {
addColumnIfMissing(db, 'imm_words', 'part_of_speech', 'TEXT');
addColumnIfMissing(db, 'imm_words', 'pos1', 'TEXT');
addColumnIfMissing(db, 'imm_words', 'pos2', 'TEXT');
addColumnIfMissing(db, 'imm_words', 'pos3', 'TEXT');
}
if (currentVersion?.schema_version && currentVersion.schema_version < 7) {
db.exec(`
CREATE TABLE IF NOT EXISTS imm_subtitle_lines(
line_id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id INTEGER NOT NULL,
event_id INTEGER,
video_id INTEGER NOT NULL,
anime_id INTEGER,
line_index INTEGER NOT NULL,
segment_start_ms INTEGER,
segment_end_ms INTEGER,
text TEXT NOT NULL,
CREATED_DATE INTEGER,
LAST_UPDATE_DATE INTEGER,
FOREIGN KEY(session_id) REFERENCES imm_sessions(session_id) ON DELETE CASCADE,
FOREIGN KEY(event_id) REFERENCES imm_session_events(event_id) ON DELETE SET NULL,
FOREIGN KEY(video_id) REFERENCES imm_videos(video_id) ON DELETE CASCADE,
FOREIGN KEY(anime_id) REFERENCES imm_anime(anime_id) ON DELETE SET NULL
)
`);
db.exec(`
CREATE TABLE IF NOT EXISTS imm_word_line_occurrences(
line_id INTEGER NOT NULL,
word_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
PRIMARY KEY(line_id, word_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(word_id) REFERENCES imm_words(id) ON DELETE CASCADE
)
`);
db.exec(`
CREATE TABLE IF NOT EXISTS imm_kanji_line_occurrences(
line_id INTEGER NOT NULL,
kanji_id INTEGER NOT NULL,
occurrence_count INTEGER NOT NULL,
PRIMARY KEY(line_id, kanji_id),
FOREIGN KEY(line_id) REFERENCES imm_subtitle_lines(line_id) ON DELETE CASCADE,
FOREIGN KEY(kanji_id) REFERENCES imm_kanji(id) ON DELETE CASCADE
)
`);
}
db.exec(`
CREATE INDEX IF NOT EXISTS idx_anime_normalized_title
ON imm_anime(normalized_title_key)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_anime_anilist_id
ON imm_anime(anilist_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_videos_anime_id
ON imm_videos(anime_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_sessions_video_started
ON imm_sessions(video_id, started_at_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_sessions_status_started
ON imm_sessions(status, started_at_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_telemetry_session_sample
ON imm_session_telemetry(session_id, sample_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_events_session_ts
ON imm_session_events(session_id, ts_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_events_type_ts
ON imm_session_events(event_type, ts_ms DESC)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_rollups_day_video
ON imm_daily_rollups(rollup_day, video_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_rollups_month_video
ON imm_monthly_rollups(rollup_month, video_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_words_headword_word_reading
ON imm_words(headword, word, reading)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_kanji_kanji
ON imm_kanji(kanji)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_session_line
ON imm_subtitle_lines(session_id, line_index)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_video_line
ON imm_subtitle_lines(video_id, line_index)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_subtitle_lines_anime_line
ON imm_subtitle_lines(anime_id, line_index)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_word_line_occurrences_word
ON imm_word_line_occurrences(word_id, line_id)
`);
db.exec(`
CREATE INDEX IF NOT EXISTS idx_kanji_line_occurrences_kanji
ON imm_kanji_line_occurrences(kanji_id, line_id)
`);
if (currentVersion?.schema_version && currentVersion.schema_version < SCHEMA_VERSION) {
db.exec('DELETE FROM imm_daily_rollups');
db.exec('DELETE FROM imm_monthly_rollups');
db.exec(`UPDATE imm_rollup_state SET state_value = 0 WHERE state_key = 'last_rollup_sample_ms'`);
}
db.exec(`
INSERT INTO imm_schema_version(schema_version, applied_at_ms)
VALUES (${SCHEMA_VERSION}, ${Date.now()})
@@ -328,12 +776,21 @@ export function createTrackerPreparedStatements(db: DatabaseSync): TrackerPrepar
`),
wordUpsertStmt: db.prepare(`
INSERT INTO imm_words (
headword, word, reading, first_seen, last_seen, frequency
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (
?, ?, ?, ?, ?, 1
?, ?, ?, ?, ?, ?, ?, ?, ?, 1
)
ON CONFLICT(headword, word, reading) DO UPDATE SET
frequency = COALESCE(frequency, 0) + 1,
part_of_speech = CASE
WHEN COALESCE(NULLIF(imm_words.part_of_speech, ''), 'other') = 'other'
AND COALESCE(NULLIF(excluded.part_of_speech, ''), '') <> ''
THEN excluded.part_of_speech
ELSE imm_words.part_of_speech
END,
pos1 = COALESCE(NULLIF(imm_words.pos1, ''), excluded.pos1),
pos2 = COALESCE(NULLIF(imm_words.pos2, ''), excluded.pos2),
pos3 = COALESCE(NULLIF(imm_words.pos3, ''), excluded.pos3),
first_seen = MIN(COALESCE(first_seen, excluded.first_seen), excluded.first_seen),
last_seen = MAX(COALESCE(last_seen, excluded.last_seen), excluded.last_seen)
`),
@@ -348,9 +805,93 @@ export function createTrackerPreparedStatements(db: DatabaseSync): TrackerPrepar
first_seen = MIN(COALESCE(first_seen, excluded.first_seen), excluded.first_seen),
last_seen = MAX(COALESCE(last_seen, excluded.last_seen), excluded.last_seen)
`),
subtitleLineInsertStmt: db.prepare(`
INSERT INTO imm_subtitle_lines (
session_id, event_id, video_id, anime_id, line_index, segment_start_ms,
segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE
) VALUES (
?, ?, ?, ?, ?, ?, ?, ?, ?, ?
)
`),
wordIdSelectStmt: db.prepare(`
SELECT id FROM imm_words
WHERE headword = ? AND word = ? AND reading = ?
`),
kanjiIdSelectStmt: db.prepare(`
SELECT id FROM imm_kanji
WHERE kanji = ?
`),
wordLineOccurrenceUpsertStmt: db.prepare(`
INSERT INTO imm_word_line_occurrences (
line_id, word_id, occurrence_count
) VALUES (
?, ?, ?
)
ON CONFLICT(line_id, word_id) DO UPDATE SET
occurrence_count = imm_word_line_occurrences.occurrence_count + excluded.occurrence_count
`),
kanjiLineOccurrenceUpsertStmt: db.prepare(`
INSERT INTO imm_kanji_line_occurrences (
line_id, kanji_id, occurrence_count
) VALUES (
?, ?, ?
)
ON CONFLICT(line_id, kanji_id) DO UPDATE SET
occurrence_count = imm_kanji_line_occurrences.occurrence_count + excluded.occurrence_count
`),
videoAnimeIdSelectStmt: db.prepare(`
SELECT anime_id FROM imm_videos
WHERE video_id = ?
`),
};
}
function incrementWordAggregate(
stmts: TrackerPreparedStatements,
occurrence: Extract<QueuedWrite, { kind: 'subtitleLine' }>['wordOccurrences'][number],
firstSeen: number,
lastSeen: number,
): number {
for (let i = 0; i < occurrence.occurrenceCount; i += 1) {
stmts.wordUpsertStmt.run(
occurrence.headword,
occurrence.word,
occurrence.reading,
occurrence.partOfSpeech,
occurrence.pos1,
occurrence.pos2,
occurrence.pos3,
firstSeen,
lastSeen,
);
}
const row = stmts.wordIdSelectStmt.get(
occurrence.headword,
occurrence.word,
occurrence.reading,
) as { id: number } | null;
if (!row?.id) {
throw new Error(`Failed to resolve imm_words id for ${occurrence.headword}`);
}
return row.id;
}
function incrementKanjiAggregate(
stmts: TrackerPreparedStatements,
occurrence: Extract<QueuedWrite, { kind: 'subtitleLine' }>['kanjiOccurrences'][number],
firstSeen: number,
lastSeen: number,
): number {
for (let i = 0; i < occurrence.occurrenceCount; i += 1) {
stmts.kanjiUpsertStmt.run(occurrence.kanji, firstSeen, lastSeen);
}
const row = stmts.kanjiIdSelectStmt.get(occurrence.kanji) as { id: number } | null;
if (!row?.id) {
throw new Error(`Failed to resolve imm_kanji id for ${occurrence.kanji}`);
}
return row.id;
}
export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedStatements): void {
if (write.kind === 'telemetry') {
stmts.telemetryInsertStmt.run(
@@ -379,6 +920,10 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
write.headword,
write.word,
write.reading,
write.partOfSpeech,
write.pos1,
write.pos2,
write.pos3,
write.firstSeen,
write.lastSeen,
);
@@ -388,6 +933,31 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
stmts.kanjiUpsertStmt.run(write.kanji, write.firstSeen, write.lastSeen);
return;
}
if (write.kind === 'subtitleLine') {
const animeRow = stmts.videoAnimeIdSelectStmt.get(write.videoId) as { anime_id: number | null } | null;
const lineResult = stmts.subtitleLineInsertStmt.run(
write.sessionId,
null,
write.videoId,
animeRow?.anime_id ?? null,
write.lineIndex,
write.segmentStartMs ?? null,
write.segmentEndMs ?? null,
write.text,
Date.now(),
Date.now(),
);
const lineId = Number(lineResult.lastInsertRowid);
for (const occurrence of write.wordOccurrences) {
const wordId = incrementWordAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
stmts.wordLineOccurrenceUpsertStmt.run(lineId, wordId, occurrence.occurrenceCount);
}
for (const occurrence of write.kanjiOccurrences) {
const kanjiId = incrementKanjiAggregate(stmts, occurrence, write.firstSeen, write.lastSeen);
stmts.kanjiLineOccurrenceUpsertStmt.run(lineId, kanjiId, occurrence.occurrenceCount);
}
return;
}
stmts.eventInsertStmt.run(
write.sessionId,

View File

@@ -1,4 +1,4 @@
export const SCHEMA_VERSION = 3;
export const SCHEMA_VERSION = 7;
export const DEFAULT_QUEUE_CAP = 1_000;
export const DEFAULT_BATCH_SIZE = 25;
export const DEFAULT_FLUSH_INTERVAL_MS = 500;
@@ -29,6 +29,9 @@ export const EVENT_PAUSE_END = 8;
export interface ImmersionTrackerOptions {
dbPath: string;
policy?: ImmersionTrackerPolicy;
resolveLegacyVocabularyPos?: (
row: LegacyVocabularyPosRow,
) => Promise<LegacyVocabularyPosResolution | null>;
}
export interface ImmersionTrackerPolicy {
@@ -72,6 +75,7 @@ export interface SessionState extends TelemetryAccumulator {
lastPauseStartMs: number | null;
isPaused: boolean;
pendingTelemetry: boolean;
markedWatched: boolean;
}
interface QueuedTelemetryWrite {
@@ -118,6 +122,10 @@ interface QueuedWordWrite {
headword: string;
word: string;
reading: string;
partOfSpeech: string;
pos1: string;
pos2: string;
pos3: string;
firstSeen: number;
lastSeen: number;
}
@@ -129,11 +137,42 @@ interface QueuedKanjiWrite {
lastSeen: number;
}
export interface CountedWordOccurrence {
headword: string;
word: string;
reading: string;
partOfSpeech: string;
pos1: string;
pos2: string;
pos3: string;
occurrenceCount: number;
}
export interface CountedKanjiOccurrence {
kanji: string;
occurrenceCount: number;
}
interface QueuedSubtitleLineWrite {
kind: 'subtitleLine';
sessionId: number;
videoId: number;
lineIndex: number;
segmentStartMs: number | null;
segmentEndMs: number | null;
text: string;
wordOccurrences: CountedWordOccurrence[];
kanjiOccurrences: CountedKanjiOccurrence[];
firstSeen: number;
lastSeen: number;
}
export type QueuedWrite =
| QueuedTelemetryWrite
| QueuedEventWrite
| QueuedWordWrite
| QueuedKanjiWrite;
| QueuedKanjiWrite
| QueuedSubtitleLineWrite;
export interface VideoMetadata {
sourceType: number;
@@ -152,8 +191,33 @@ export interface VideoMetadata {
metadataJson: string | null;
}
export interface ParsedAnimeVideoMetadata {
animeId: number | null;
parsedBasename: string | null;
parsedTitle: string | null;
parsedSeason: number | null;
parsedEpisode: number | null;
parserSource: string | null;
parserConfidence: number | null;
parseMetadataJson: string | null;
}
export interface ParsedAnimeVideoGuess {
parsedBasename: string | null;
parsedTitle: string;
parsedSeason: number | null;
parsedEpisode: number | null;
parserSource: 'guessit' | 'fallback';
parserConfidence: number;
parseMetadataJson: string;
}
export interface SessionSummaryQueryRow {
sessionId: number;
videoId: number | null;
canonicalTitle: string | null;
animeId: number | null;
animeTitle: string | null;
startedAtMs: number;
endedAtMs: number | null;
totalWatchedMs: number;
@@ -166,6 +230,82 @@ export interface SessionSummaryQueryRow {
lookupHits: number;
}
export interface VocabularyStatsRow {
wordId: number;
headword: string;
word: string;
reading: string;
partOfSpeech: string | null;
pos1: string | null;
pos2: string | null;
pos3: string | null;
frequency: number;
firstSeen: number;
lastSeen: number;
}
export interface VocabularyCleanupSummary {
scanned: number;
kept: number;
deleted: number;
repaired: number;
}
export interface LegacyVocabularyPosRow {
headword: string;
word: string;
reading: string | null;
}
export interface LegacyVocabularyPosResolution {
headword: string;
reading: string;
partOfSpeech: string;
pos1: string;
pos2: string;
pos3: string;
}
export interface KanjiStatsRow {
kanjiId: number;
kanji: string;
frequency: number;
firstSeen: number;
lastSeen: number;
}
export interface WordOccurrenceRow {
animeId: number | null;
animeTitle: string | null;
videoId: number;
videoTitle: string;
sessionId: number;
lineIndex: number;
segmentStartMs: number | null;
segmentEndMs: number | null;
text: string;
occurrenceCount: number;
}
export interface KanjiOccurrenceRow {
animeId: number | null;
animeTitle: string | null;
videoId: number;
videoTitle: string;
sessionId: number;
lineIndex: number;
segmentStartMs: number | null;
segmentEndMs: number | null;
text: string;
occurrenceCount: number;
}
export interface SessionEventRow {
eventType: number;
tsMs: number;
payload: string | null;
}
export interface SessionTimelineRow {
sampleMs: number;
totalWatchedMs: number;
@@ -200,3 +340,180 @@ export interface ProbeMetadata {
bitrateKbps: number | null;
audioCodecId: number | null;
}
export interface MediaArtRow {
videoId: number;
anilistId: number | null;
coverUrl: string | null;
coverBlob: Buffer | null;
titleRomaji: string | null;
titleEnglish: string | null;
episodesTotal: number | null;
fetchedAtMs: number;
}
export interface MediaLibraryRow {
videoId: number;
canonicalTitle: string;
totalSessions: number;
totalActiveMs: number;
totalCards: number;
totalWordsSeen: number;
lastWatchedMs: number;
hasCoverArt: number;
}
export interface MediaDetailRow {
videoId: number;
canonicalTitle: string;
totalSessions: number;
totalActiveMs: number;
totalCards: number;
totalWordsSeen: number;
totalLinesSeen: number;
totalLookupCount: number;
totalLookupHits: number;
}
export interface AnimeLibraryRow {
animeId: number;
canonicalTitle: string;
anilistId: number | null;
totalSessions: number;
totalActiveMs: number;
totalCards: number;
totalWordsSeen: number;
episodeCount: number;
episodesTotal: number | null;
lastWatchedMs: number;
}
export interface AnimeDetailRow {
animeId: number;
canonicalTitle: string;
anilistId: number | null;
titleRomaji: string | null;
titleEnglish: string | null;
titleNative: string | null;
totalSessions: number;
totalActiveMs: number;
totalCards: number;
totalWordsSeen: number;
totalLinesSeen: number;
totalLookupCount: number;
totalLookupHits: number;
episodeCount: number;
lastWatchedMs: number;
}
export interface AnimeAnilistEntryRow {
anilistId: number;
titleRomaji: string | null;
titleEnglish: string | null;
season: number | null;
}
export interface AnimeEpisodeRow {
animeId: number;
videoId: number;
canonicalTitle: string;
parsedTitle: string | null;
season: number | null;
episode: number | null;
durationMs: number;
watched: number;
totalSessions: number;
totalActiveMs: number;
totalCards: number;
totalWordsSeen: number;
lastWatchedMs: number;
}
export interface StreakCalendarRow {
epochDay: number;
totalActiveMin: number;
}
export interface AnimeWordRow {
wordId: number;
headword: string;
word: string;
reading: string;
partOfSpeech: string | null;
frequency: number;
}
export interface EpisodesPerDayRow {
epochDay: number;
episodeCount: number;
}
export interface NewAnimePerDayRow {
epochDay: number;
newAnimeCount: number;
}
export interface WatchTimePerAnimeRow {
epochDay: number;
animeId: number;
animeTitle: string;
totalActiveMin: number;
}
export interface WordDetailRow {
wordId: number;
headword: string;
word: string;
reading: string;
partOfSpeech: string | null;
pos1: string | null;
pos2: string | null;
pos3: string | null;
frequency: number;
firstSeen: number;
lastSeen: number;
}
export interface WordAnimeAppearanceRow {
animeId: number;
animeTitle: string;
occurrenceCount: number;
}
export interface SimilarWordRow {
wordId: number;
headword: string;
word: string;
reading: string;
frequency: number;
}
export interface KanjiDetailRow {
kanjiId: number;
kanji: string;
frequency: number;
firstSeen: number;
lastSeen: number;
}
export interface KanjiAnimeAppearanceRow {
animeId: number;
animeTitle: string;
occurrenceCount: number;
}
export interface KanjiWordRow {
wordId: number;
headword: string;
word: string;
reading: string;
frequency: number;
}
export interface EpisodeCardEventRow {
eventId: number;
sessionId: number;
tsMs: number;
cardsDelta: number;
noteIds: number[];
}

View File

@@ -1,7 +1,7 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { createIpcDepsRuntime, registerIpcHandlers } from './ipc';
import { createIpcDepsRuntime, registerIpcHandlers, type IpcServiceDeps } from './ipc';
import { IPC_CHANNELS } from '../../shared/ipc/contracts';
interface FakeIpcRegistrar {
@@ -33,6 +33,90 @@ function createFakeIpcRegistrar(): {
};
}
function createRegisterIpcDeps(overrides: Partial<IpcServiceDeps> = {}): IpcServiceDeps {
return {
onOverlayModalClosed: () => {},
openYomitanSettings: () => {},
quitApp: () => {},
toggleDevTools: () => {},
getVisibleOverlayVisibility: () => false,
toggleVisibleOverlay: () => {},
tokenizeCurrentSubtitle: async () => null,
getCurrentSubtitleRaw: () => '',
getCurrentSubtitleAss: () => '',
getPlaybackPaused: () => false,
getSubtitlePosition: () => null,
getSubtitleStyle: () => null,
saveSubtitlePosition: () => {},
getMecabStatus: () => ({ available: false, enabled: false, path: null }),
setMecabEnabled: () => {},
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () =>
({
enabled: true,
preferredGamepadId: '',
preferredGamepadLabel: '',
smoothScroll: true,
scrollPixelsPerSecond: 960,
horizontalJumpPixels: 160,
stickDeadzone: 0.2,
triggerInputMode: 'auto',
triggerDeadzone: 0.5,
repeatDelayMs: 220,
repeatIntervalMs: 80,
buttonIndices: {
select: 6,
buttonSouth: 0,
buttonEast: 1,
buttonWest: 2,
buttonNorth: 3,
leftShoulder: 4,
rightShoulder: 5,
leftStickPress: 9,
rightStickPress: 10,
leftTrigger: 6,
rightTrigger: 7,
},
bindings: {
toggleLookup: 'buttonSouth',
closeLookup: 'buttonEast',
toggleKeyboardOnlyMode: 'buttonNorth',
mineCard: 'buttonWest',
quitMpv: 'select',
previousAudio: 'leftShoulder',
nextAudio: 'rightShoulder',
playCurrentAudio: 'rightTrigger',
toggleMpvPause: 'leftTrigger',
leftStickHorizontal: 'leftStickX',
leftStickVertical: 'leftStickY',
rightStickHorizontal: 'rightStickX',
rightStickVertical: 'rightStickY',
},
}) as never,
saveControllerPreference: async () => {},
getSecondarySubMode: () => 'hover',
getCurrentSecondarySub: () => '',
focusMainWindow: () => {},
runSubsyncManual: async () => ({ ok: true, message: 'ok' }),
getAnkiConnectStatus: () => false,
getRuntimeOptions: () => [],
setRuntimeOption: () => ({ ok: true }),
cycleRuntimeOption: () => ({ ok: true }),
reportOverlayContentBounds: () => {},
getAnilistStatus: () => ({}),
clearAnilistToken: () => {},
openAnilistSetup: () => {},
getAnilistQueueStatus: () => ({}),
retryAnilistQueueNow: async () => ({ ok: true, message: 'ok' }),
appendClipboardVideoToQueue: () => ({ ok: true, message: 'ok' }),
immersionTracker: null,
...overrides,
};
}
test('createIpcDepsRuntime wires AniList handlers', async () => {
const calls: string[] = [];
const deps = createIpcDepsRuntime({
@@ -53,6 +137,7 @@ test('createIpcDepsRuntime wires AniList handlers', async () => {
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () => ({
enabled: true,
preferredGamepadId: '',
@@ -159,6 +244,7 @@ test('registerIpcHandlers rejects malformed runtime-option payloads', async () =
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () => ({
enabled: true,
preferredGamepadId: '',
@@ -266,6 +352,90 @@ test('registerIpcHandlers rejects malformed runtime-option payloads', async () =
);
});
test('registerIpcHandlers returns empty stats overview shape without a tracker', async () => {
const { registrar, handlers } = createFakeIpcRegistrar();
registerIpcHandlers(createRegisterIpcDeps(), registrar);
const overviewHandler = handlers.handle.get(IPC_CHANNELS.request.statsGetOverview);
assert.ok(overviewHandler);
assert.deepEqual(await overviewHandler!({}), {
sessions: [],
rollups: [],
hints: {
totalSessions: 0,
activeSessions: 0,
},
});
});
test('registerIpcHandlers validates and clamps stats request limits', async () => {
const { registrar, handlers } = createFakeIpcRegistrar();
const calls: Array<[string, number, number?]> = [];
registerIpcHandlers(
createRegisterIpcDeps({
immersionTracker: {
getSessionSummaries: async (limit = 0) => {
calls.push(['sessions', limit]);
return [];
},
getDailyRollups: async (limit = 0) => {
calls.push(['daily', limit]);
return [];
},
getMonthlyRollups: async (limit = 0) => {
calls.push(['monthly', limit]);
return [];
},
getQueryHints: async () => ({ totalSessions: 0, activeSessions: 0, episodesToday: 0, activeAnimeCount: 0 }),
getSessionTimeline: async (sessionId: number, limit = 0) => {
calls.push(['timeline', limit, sessionId]);
return [];
},
getSessionEvents: async (sessionId: number, limit = 0) => {
calls.push(['events', limit, sessionId]);
return [];
},
getVocabularyStats: async (limit = 0) => {
calls.push(['vocabulary', limit]);
return [];
},
getKanjiStats: async (limit = 0) => {
calls.push(['kanji', limit]);
return [];
},
getMediaLibrary: async () => [],
getMediaDetail: async () => null,
getMediaSessions: async () => [],
getMediaDailyRollups: async () => [],
getCoverArt: async () => null,
},
}),
registrar,
);
await handlers.handle.get(IPC_CHANNELS.request.statsGetDailyRollups)!({}, -1);
await handlers.handle.get(IPC_CHANNELS.request.statsGetMonthlyRollups)!(
{},
Number.POSITIVE_INFINITY,
);
await handlers.handle.get(IPC_CHANNELS.request.statsGetSessions)!({}, 9999);
await handlers.handle.get(IPC_CHANNELS.request.statsGetSessionTimeline)!({}, 7, 12.5);
await handlers.handle.get(IPC_CHANNELS.request.statsGetSessionEvents)!({}, 7, 0);
await handlers.handle.get(IPC_CHANNELS.request.statsGetVocabulary)!({}, 1000);
await handlers.handle.get(IPC_CHANNELS.request.statsGetKanji)!({}, NaN);
assert.deepEqual(calls, [
['daily', 60],
['monthly', 24],
['sessions', 500],
['timeline', 200, 7],
['events', 500, 7],
['vocabulary', 500],
['kanji', 100],
]);
});
test('registerIpcHandlers ignores malformed fire-and-forget payloads', () => {
const { registrar, handlers } = createFakeIpcRegistrar();
const saves: unknown[] = [];
@@ -299,6 +469,7 @@ test('registerIpcHandlers ignores malformed fire-and-forget payloads', () => {
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () => ({
enabled: true,
preferredGamepadId: '',
@@ -400,6 +571,7 @@ test('registerIpcHandlers awaits saveControllerPreference through request-respon
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () => ({
enabled: true,
preferredGamepadId: '',
@@ -508,6 +680,7 @@ test('registerIpcHandlers rejects malformed controller preference payloads', asy
handleMpvCommand: () => {},
getKeybindings: () => [],
getConfiguredShortcuts: () => ({}),
getStatsToggleKey: () => 'Backquote',
getControllerConfig: () => ({
enabled: true,
preferredGamepadId: '',

View File

@@ -48,6 +48,7 @@ export interface IpcServiceDeps {
handleMpvCommand: (command: Array<string | number>) => void;
getKeybindings: () => unknown;
getConfiguredShortcuts: () => unknown;
getStatsToggleKey: () => string;
getControllerConfig: () => ResolvedControllerConfig;
saveControllerPreference: (update: ControllerPreferenceUpdate) => void | Promise<void>;
getSecondarySubMode: () => unknown;
@@ -65,6 +66,21 @@ export interface IpcServiceDeps {
getAnilistQueueStatus: () => unknown;
retryAnilistQueueNow: () => Promise<{ ok: boolean; message: string }>;
appendClipboardVideoToQueue: () => { ok: boolean; message: string };
immersionTracker?: {
getSessionSummaries: (limit?: number) => Promise<unknown>;
getDailyRollups: (limit?: number) => Promise<unknown>;
getMonthlyRollups: (limit?: number) => Promise<unknown>;
getQueryHints: () => Promise<{ totalSessions: number; activeSessions: number; episodesToday: number; activeAnimeCount: number }>;
getSessionTimeline: (sessionId: number, limit?: number) => Promise<unknown>;
getSessionEvents: (sessionId: number, limit?: number) => Promise<unknown>;
getVocabularyStats: (limit?: number) => Promise<unknown>;
getKanjiStats: (limit?: number) => Promise<unknown>;
getMediaLibrary: () => Promise<unknown>;
getMediaDetail: (videoId: number) => Promise<unknown>;
getMediaSessions: (videoId: number, limit?: number) => Promise<unknown>;
getMediaDailyRollups: (videoId: number, limit?: number) => Promise<unknown>;
getCoverArt: (videoId: number) => Promise<unknown>;
} | null;
}
interface WindowLike {
@@ -113,6 +129,7 @@ export interface IpcDepsRuntimeOptions {
handleMpvCommand: (command: Array<string | number>) => void;
getKeybindings: () => unknown;
getConfiguredShortcuts: () => unknown;
getStatsToggleKey: () => string;
getControllerConfig: () => ResolvedControllerConfig;
saveControllerPreference: (update: ControllerPreferenceUpdate) => void | Promise<void>;
getSecondarySubMode: () => unknown;
@@ -130,6 +147,7 @@ export interface IpcDepsRuntimeOptions {
getAnilistQueueStatus: () => unknown;
retryAnilistQueueNow: () => Promise<{ ok: boolean; message: string }>;
appendClipboardVideoToQueue: () => { ok: boolean; message: string };
getImmersionTracker?: () => IpcServiceDeps['immersionTracker'];
}
export function createIpcDepsRuntime(options: IpcDepsRuntimeOptions): IpcServiceDeps {
@@ -166,6 +184,7 @@ export function createIpcDepsRuntime(options: IpcDepsRuntimeOptions): IpcService
handleMpvCommand: options.handleMpvCommand,
getKeybindings: options.getKeybindings,
getConfiguredShortcuts: options.getConfiguredShortcuts,
getStatsToggleKey: options.getStatsToggleKey,
getControllerConfig: options.getControllerConfig,
saveControllerPreference: options.saveControllerPreference,
getSecondarySubMode: options.getSecondarySubMode,
@@ -187,10 +206,24 @@ export function createIpcDepsRuntime(options: IpcDepsRuntimeOptions): IpcService
getAnilistQueueStatus: options.getAnilistQueueStatus,
retryAnilistQueueNow: options.retryAnilistQueueNow,
appendClipboardVideoToQueue: options.appendClipboardVideoToQueue,
get immersionTracker() {
return options.getImmersionTracker?.() ?? null;
},
};
}
export function registerIpcHandlers(deps: IpcServiceDeps, ipc: IpcMainRegistrar = ipcMain): void {
const parsePositiveIntLimit = (
value: unknown,
defaultValue: number,
maxValue: number,
): number => {
if (!Number.isInteger(value) || (value as number) < 1) {
return defaultValue;
}
return Math.min(value as number, maxValue);
};
ipc.on(
IPC_CHANNELS.command.setIgnoreMouseEvents,
(event: unknown, ignore: unknown, options: unknown = {}) => {
@@ -299,6 +332,10 @@ export function registerIpcHandlers(deps: IpcServiceDeps, ipc: IpcMainRegistrar
return deps.getConfiguredShortcuts();
});
ipc.handle(IPC_CHANNELS.request.getStatsToggleKey, () => {
return deps.getStatsToggleKey();
});
ipc.handle(IPC_CHANNELS.request.getControllerConfig, () => {
return deps.getControllerConfig();
});
@@ -384,4 +421,106 @@ export function registerIpcHandlers(deps: IpcServiceDeps, ipc: IpcMainRegistrar
ipc.handle(IPC_CHANNELS.request.appendClipboardVideoToQueue, () => {
return deps.appendClipboardVideoToQueue();
});
// Stats request handlers
ipc.handle(IPC_CHANNELS.request.statsGetOverview, async () => {
const tracker = deps.immersionTracker;
if (!tracker) {
return {
sessions: [],
rollups: [],
hints: {
totalSessions: 0,
activeSessions: 0,
},
};
}
const [sessions, rollups, hints] = await Promise.all([
tracker.getSessionSummaries(5),
tracker.getDailyRollups(14),
tracker.getQueryHints(),
]);
return { sessions, rollups, hints };
});
ipc.handle(IPC_CHANNELS.request.statsGetDailyRollups, async (_event, limit: unknown) => {
const parsedLimit = parsePositiveIntLimit(limit, 60, 500);
return deps.immersionTracker?.getDailyRollups(parsedLimit) ?? [];
});
ipc.handle(IPC_CHANNELS.request.statsGetMonthlyRollups, async (_event, limit: unknown) => {
const parsedLimit = parsePositiveIntLimit(limit, 24, 120);
return deps.immersionTracker?.getMonthlyRollups(parsedLimit) ?? [];
});
ipc.handle(IPC_CHANNELS.request.statsGetSessions, async (_event, limit: unknown) => {
const parsedLimit = parsePositiveIntLimit(limit, 50, 500);
return deps.immersionTracker?.getSessionSummaries(parsedLimit) ?? [];
});
ipc.handle(
IPC_CHANNELS.request.statsGetSessionTimeline,
async (_event, sessionId: unknown, limit: unknown) => {
if (typeof sessionId !== 'number') return [];
const parsedLimit = parsePositiveIntLimit(limit, 200, 1000);
return deps.immersionTracker?.getSessionTimeline(sessionId, parsedLimit) ?? [];
},
);
ipc.handle(
IPC_CHANNELS.request.statsGetSessionEvents,
async (_event, sessionId: unknown, limit: unknown) => {
if (typeof sessionId !== 'number') return [];
const parsedLimit = parsePositiveIntLimit(limit, 500, 1000);
return deps.immersionTracker?.getSessionEvents(sessionId, parsedLimit) ?? [];
},
);
ipc.handle(IPC_CHANNELS.request.statsGetVocabulary, async (_event, limit: unknown) => {
const parsedLimit = parsePositiveIntLimit(limit, 100, 500);
return deps.immersionTracker?.getVocabularyStats(parsedLimit) ?? [];
});
ipc.handle(IPC_CHANNELS.request.statsGetKanji, async (_event, limit: unknown) => {
const parsedLimit = parsePositiveIntLimit(limit, 100, 500);
return deps.immersionTracker?.getKanjiStats(parsedLimit) ?? [];
});
ipc.handle(IPC_CHANNELS.request.statsGetMediaLibrary, async () => {
return deps.immersionTracker?.getMediaLibrary() ?? [];
});
ipc.handle(
IPC_CHANNELS.request.statsGetMediaDetail,
async (_event, videoId: unknown) => {
if (typeof videoId !== 'number') return null;
return deps.immersionTracker?.getMediaDetail(videoId) ?? null;
},
);
ipc.handle(
IPC_CHANNELS.request.statsGetMediaSessions,
async (_event, videoId: unknown, limit: unknown) => {
if (typeof videoId !== 'number') return [];
const parsedLimit = parsePositiveIntLimit(limit, 100, 500);
return deps.immersionTracker?.getMediaSessions(videoId, parsedLimit) ?? [];
},
);
ipc.handle(
IPC_CHANNELS.request.statsGetMediaDailyRollups,
async (_event, videoId: unknown, limit: unknown) => {
if (typeof videoId !== 'number') return [];
const parsedLimit = parsePositiveIntLimit(limit, 90, 500);
return deps.immersionTracker?.getMediaDailyRollups(videoId, parsedLimit) ?? [];
},
);
ipc.handle(
IPC_CHANNELS.request.statsGetMediaCover,
async (_event, videoId: unknown) => {
if (typeof videoId !== 'number') return null;
return deps.immersionTracker?.getCoverArt(videoId) ?? null;
},
);
}

View File

@@ -59,9 +59,12 @@ const MPV_SUBTITLE_PROPERTY_OBSERVATIONS: string[] = [
'sub-ass-override',
'sub-use-margins',
'pause',
'duration',
'media-title',
'secondary-sub-visibility',
'sub-visibility',
'sid',
'track-list',
];
const MPV_INITIAL_PROPERTY_REQUESTS: Array<MpvProtocolCommand> = [

View File

@@ -60,6 +60,8 @@ function createDeps(overrides: Partial<MpvProtocolHandleMessageDeps> = {}): {
emitSubtitleAssChange: (payload) => state.events.push(payload),
emitSubtitleTiming: (payload) => state.events.push(payload),
emitSecondarySubtitleChange: (payload) => state.events.push(payload),
emitSubtitleTrackChange: (payload) => state.events.push(payload),
emitSubtitleTrackListChange: (payload) => state.events.push(payload),
getCurrentSubText: () => state.subText,
setCurrentSubText: (text) => {
state.subText = text;
@@ -87,6 +89,7 @@ function createDeps(overrides: Partial<MpvProtocolHandleMessageDeps> = {}): {
getPauseAtTime: () => null,
setPauseAtTime: () => {},
emitTimePosChange: () => {},
emitDurationChange: () => {},
emitPauseChange: () => {},
autoLoadSecondarySubTrack: () => {},
setCurrentVideoPath: () => {},
@@ -119,6 +122,24 @@ test('dispatchMpvProtocolMessage emits subtitle text on property change', async
assert.deepEqual(state.events, [{ text: '字幕', isOverlayVisible: false }]);
});
test('dispatchMpvProtocolMessage emits subtitle track changes', async () => {
const { deps, state } = createDeps({
emitSubtitleTrackChange: (payload) => state.events.push(payload),
emitSubtitleTrackListChange: (payload) => state.events.push(payload),
});
await dispatchMpvProtocolMessage(
{ event: 'property-change', name: 'sid', data: '3' },
deps,
);
await dispatchMpvProtocolMessage(
{ event: 'property-change', name: 'track-list', data: [{ type: 'sub', id: 3 }] },
deps,
);
assert.deepEqual(state.events, [{ sid: 3 }, { trackList: [{ type: 'sub', id: 3 }] }]);
});
test('dispatchMpvProtocolMessage enforces sub-visibility hidden when overlay suppression is enabled', async () => {
const { deps, state } = createDeps({
isVisibleOverlayVisible: () => true,

View File

@@ -52,6 +52,8 @@ export interface MpvProtocolHandleMessageDeps {
emitSubtitleAssChange: (payload: { text: string }) => void;
emitSubtitleTiming: (payload: { text: string; start: number; end: number }) => void;
emitSecondarySubtitleChange: (payload: { text: string }) => void;
emitSubtitleTrackChange: (payload: { sid: number | null }) => void;
emitSubtitleTrackListChange: (payload: { trackList: unknown[] | null }) => void;
getCurrentSubText: () => string;
setCurrentSubText: (text: string) => void;
setCurrentSubStart: (value: number) => void;
@@ -61,6 +63,7 @@ export interface MpvProtocolHandleMessageDeps {
emitMediaPathChange: (payload: { path: string }) => void;
emitMediaTitleChange: (payload: { title: string | null }) => void;
emitTimePosChange: (payload: { time: number }) => void;
emitDurationChange: (payload: { duration: number }) => void;
emitPauseChange: (payload: { paused: boolean }) => void;
emitSubtitleMetricsChange: (payload: Partial<MpvSubtitleRenderMetrics>) => void;
setCurrentSecondarySubText: (text: string) => void;
@@ -159,6 +162,18 @@ export async function dispatchMpvProtocolMessage(
const nextSubText = (msg.data as string) || '';
deps.setCurrentSecondarySubText(nextSubText);
deps.emitSecondarySubtitleChange({ text: nextSubText });
} else if (msg.name === 'sid') {
const sid =
typeof msg.data === 'number'
? msg.data
: typeof msg.data === 'string'
? Number(msg.data)
: null;
deps.emitSubtitleTrackChange({ sid: sid !== null && Number.isFinite(sid) ? sid : null });
} else if (msg.name === 'track-list') {
deps.emitSubtitleTrackListChange({
trackList: Array.isArray(msg.data) ? (msg.data as unknown[]) : null,
});
} else if (msg.name === 'aid') {
deps.setCurrentAudioTrackId(typeof msg.data === 'number' ? (msg.data as number) : null);
deps.syncCurrentAudioStreamIndex();
@@ -172,6 +187,11 @@ export async function dispatchMpvProtocolMessage(
deps.setPauseAtTime(null);
deps.sendCommand({ command: ['set_property', 'pause', true] });
}
} else if (msg.name === 'duration') {
const duration = typeof msg.data === 'number' ? msg.data : 0;
if (duration > 0) {
deps.emitDurationChange({ duration });
}
} else if (msg.name === 'pause') {
deps.emitPauseChange({ paused: asBoolean(msg.data, false) });
} else if (msg.name === 'media-title') {

Some files were not shown because too many files have changed in this diff Show More