sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-02-28 06:22:45 -08:00
Code Issues Packages Projects Releases Wiki Activity

refactor(tokenizer): remove MeCab fallback tokenization path

Browse Source
This commit is contained in:
Kyle Yasuda
2026-02-22 18:03:38 -08:00
parent f1dc418e2d
commit badb82280a
9 changed files with 212 additions and 480 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
docs/subagents/agents/codex-task88-yomitan-flow-20260223T012755Z-x4m2.md Normal file
View File

@@ -0,0 +1,40 @@
# Agent: `codex-task88-yomitan-flow-20260223T012755Z-x4m2`
- alias: `codex-task88-yomitan-flow`
- mission: `Execute TASK-88 remove MeCab fallback tokenizer and simplify Yomitan token flow via plan-first workflow (no commit).`
- status: `handoff`
- branch: `main`
- started_at: `2026-02-23T01:27:55Z`
- heartbeat_minutes: `5`
## Current Work (newest first)
- [2026-02-23T01:44:16Z] handoff: implementation + docs updates complete for TASK-88 scope; tokenizer fallback removed, parser-selection simplified to scanning-parser-only, focused tokenizer/subtitle tests + build + docs build green.
- [2026-02-23T01:44:16Z] test: `bun test src/core/services/tokenizer/parser-selection-stage.test.ts src/core/services/tokenizer.test.ts` pass (47); `bun test src/core/services/subtitle-processing-controller.test.ts` pass (6); `bun run build` pass; `bun run docs:build` pass.
- [2026-02-23T01:30:00Z] progress: wrote plan at `docs/plans/2026-02-23-task-88-yomitan-only-token-flow.md` via writing-plans skill and executed via executing-plans skill.
- [2026-02-23T01:27:55Z] intent: load backlog context for TASK-88, write plan with writing-plans skill, execute with executing-plans skill, validate via focused/full tests, no commit.
## Files Touched
- `docs/subagents/agents/codex-task88-yomitan-flow-20260223T012755Z-x4m2.md`
- `docs/subagents/INDEX.md`
- `docs/subagents/collaboration.md`
- `docs/plans/2026-02-23-task-88-yomitan-only-token-flow.md`
- `src/core/services/tokenizer.ts`
- `src/core/services/tokenizer/parser-selection-stage.ts`
- `src/core/services/tokenizer/parser-selection-stage.test.ts`
- `src/core/services/tokenizer.test.ts`
- `docs/usage.md`
- `docs/troubleshooting.md`
## Assumptions
- Backlog is initialized and TASK-88 title/context from MCP search is authoritative despite stale `task_view` collision on legacy TASK-88.
## Open Questions / Blockers
- Backlog MCP `task_view TASK-88` resolves to a legacy completed TASK-88 entry; current TASK-88 content had to be read from `backlog/tasks/task-88 - Remove-MeCab-fallback-tokenizer-and-simplify-Yomitan-token-flow.md`.
## Next Step
- If needed, repair duplicate TASK-88 ID collision in Backlog MCP so `task_view`/`task_edit` target the active To Do ticket.