sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-07-27 16:49:51 -07:00
Code Issues Packages Projects Releases Wiki Activity

refactor(tokenizer): consolidate kana/POS helpers into token-classification (#161)

Browse Source
This commit is contained in:
Kyle Yasuda
2026-07-12 23:27:25 -07:00
committed by GitHub
parent 14070acceb
commit 21a06c12fb
11 changed files with 302 additions and 396 deletions
Download Patch File Download Diff File Expand all files Collapse all files
changes/kana-pos-content-classification.md
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: overlay
- Kept kanji vocabulary tagged `名詞/非自立` eligible for N+1 highlighting, consistent with frequency, JLPT, and vocabulary persistence.