SubMiner/backlog/tasks/task-293 - Fix-interjection-tokens-receiving-subtitle-annotations.md

id, title, status, assignee, created_date, labels, dependencies, priority

id	title	status	assignee	created_date	labels	dependencies	priority
TASK-293	Fix interjection tokens receiving subtitle annotations	In Progress		2026-04-25 22:50	tokenizer bug		medium

Description

Standalone interjections such as あ should remain hoverable dictionary tokens but must not receive N+1, frequency, JLPT, or known-word subtitle annotation metadata.

Acceptance Criteria

• #1 A MeCab 感動詞 token like あ is excluded by the shared subtitle annotation gate.
• #2 annotateTokens strips N+1/frequency/JLPT/known metadata from the interjection while preserving token lookup fields.
• #3 Focused tokenizer regression passes.