sudacode/SubMiner
1
0 Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-11-10 16:46:06 -08:00
Code Issues Packages Projects Releases Wiki Activity

fix(tokenizer): preserve annotation and enrichment behavior

Browse Source
This commit is contained in:
sudacode committed 2026-05-10 22:19:42 -07:00
1 parent 8b26559203
commit 09e10c18d2
6 files changed
+55 -16
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

changes/305-tokenizer-word-class-pos-filtering.md
+6
View File
@@ -0,0 +1,6 @@
type: fixed
area: tokenizer
- Use Yomitan `wordClasses` metadata for subtitle POS filtering.
- Backfill blank MeCab POS detail fields during parser enrichment.
- Keep subtitle annotation metadata stripped from token results.