fix: exclude kana grammar helper annotations

This commit is contained in:
sudacode committed 2026-04-25 19:41:36 -07:00
1 parent 76b546c8f6
commit 2c01baafc9
5 files changed
+351

No files matched your search

@@ -0,0 +1,4 @@
type: fixed
area: tokenizer
- Stopped kana-only grammar-helper merges such as `ことに` from receiving subtitle annotation metadata like N+1, JLPT, known-word, or frequency highlighting.