sudacode/SubMiner
1
0 Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-10-27 11:35:23 -07:00
Code Issues Packages Projects Releases Wiki Activity

fix(tokenizer): keep frequency rank for lexicalized kana expressions (#156)

Browse Source
This commit is contained in:
sudacode authored and GitHub committed 2026-07-10 00:54:17 -07:00
1 parent 7b1a005a65
commit 8712780d08
4 files changed
+41 -3
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

changes/lexicalized-expression-frequency.md
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: overlay
- Fixed lexicalized kana expressions such as `かといって` losing frequency annotations when their merged MeCab parts included particles.