sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-06-24 03:13:30 -07:00
Code Issues Packages Projects Releases Wiki Activity

Fix tokenizer annotations for explanatory contrast ending (#33)

Browse Source
This commit is contained in:
Kyle Yasuda
2026-03-23 09:25:17 -07:00
committed by GitHub
parent 0317c7f011
commit c17f0a4080
5 changed files with 184 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
changes/2026-03-23-tokenizer-explanatory-contrast-ending.md
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: tokenizer
- Fixed subtitle annotation clearing so explanatory contrast endings like `んですけど` are excluded consistently across the shared tokenizer filter and annotation stage.