457e6f0f10
- map segmented Yomitan lines into single logical tokens and improve candidate selection heuristics - limit frequency lookup to selected token text with POS-based exclusions and add debug logging hook - add standalone Yomitan parser test script, deterministic utility-script shutdown, and docs/backlog updates
2.6 KiB
id, title, status, assignee, created_date, updated_date, labels, dependencies, priority
| id | title | status | assignee | created_date | updated_date | labels | dependencies | priority |
|---|---|---|---|---|---|---|---|---|
| TASK-63 | Add runtime toggle to log selected Yomitan token groups | Done | 2026-02-16 23:38 | 2026-02-16 23:41 | low |
Description
Provide an in-app debug toggle that logs the selected Yomitan token grouping for each subtitle parse so users can verify token boundaries live without rebuilding.
Acceptance Criteria
- #1 A runtime option exists to enable/disable Yomitan group debug logging without app restart.
- #2 When enabled, subtitle tokenization logs the selected Yomitan grouped tokens (with enough detail to verify boundaries/headwords).
- #3 When disabled, no additional Yomitan group debug logs are emitted.
- #4 Related tests/build pass for touched modules.
Implementation Plan
- Add a boolean runtime option for Yomitan-group debug logging in the centralized runtime option registry and expose it in config metadata.
- Extend tokenizer dependency wiring so main runtime can pass the current toggle value to tokenization without restart.
- Log selected Yomitan token groups (surface/headword/reading/span) only when the toggle is enabled.
- Add tests for registry presence and enabled/disabled logging behavior, then run build/tests.
Implementation Notes
Added runtime option anki.debugYomitanGroups (Debug Yomitan Groups) with default false, mapped to ankiConnect.behavior.debugYomitanGroups.
Wired main.ts tokenizer deps to read the runtime option value live via RuntimeOptionsManager, with config fallback.
Implemented conditional tokenizer logging (Selected Yomitan token groups) in tokenizer-service and covered enabled/disabled behavior with unit tests.
Validation run: pnpm run build && node dist/core/services/tokenizer-service.test.js && node --test dist/config/config.test.js (all passing).
Final Summary
Implemented a live runtime debug toggle to inspect Yomitan token grouping. Added anki.debugYomitanGroups to the runtime option registry and config defaults, wired it through main.ts into tokenizer deps, and added conditional logging in tokenizer parsing that emits selected groups with surface/headword/reading/span for each parsed subtitle. Logging is gated by the toggle and disabled by default. Added tests for runtime registry presence and tokenizer logging on/off behavior, then validated with build + tokenizer + config tests.