sudacode/SubMiner
1
0 Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-10-27 11:35:23 -07:00
Code Issues Packages Projects Releases Wiki Activity

refactor(tokenizer): consolidate kana/POS helpers into token-classification (#161)

Browse Source
This commit is contained in:
sudacode authored and GitHub committed 2026-07-12 23:27:25 -07:00
1 parent 14070acceb
commit 21a06c12fb
11 files changed
+302 -396
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

changes/kana-pos-content-classification.md
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: overlay
- Kept kanji vocabulary tagged `名詞/非自立` eligible for N+1 highlighting, consistent with frequency, JLPT, and vocabulary persistence.