fix(dictionary): isolate concurrent snapshot writes and shorten zip build blocks

The streamed snapshot writer keyed its temp file on the pid alone, so two
overlapping writes for the same media (a manual generate racing auto-sync)
streamed into one file and tore it; add a per-write sequence suffix.

Term banks were stringified 10k entries at a time, measured at ~38MB and
~135ms per bank on a real merged dictionary. Halving that block matters
more than the write itself: at 2k entries the longest event-loop stall in
a merged build drops from ~135ms to 28ms.
This commit is contained in:
2026-08-17 02:46:27 -07:00
parent 1228ebe622
commit 9df2f37d4b
4 changed files with 125 additions and 2 deletions
+5 -1
View File
@@ -78,7 +78,11 @@ export async function buildDictionaryZip(
};
}
const entriesPerBank = 10_000;
// Each bank is stringified in one shot, so the bank size sets the longest single block in the
// build. 10k entries measured ~38MB and ~135ms per bank on a real merged dictionary; 2k keeps
// every bank under the archive writer's yield budget at ~27ms. Yomitan reads any number of
// term_bank_N.json files, so this only changes how the terms are split across them.
const entriesPerBank = 2_000;
for (let i = 0; i < termEntries.length; i += entriesPerBank) {
yield {
name: `term_bank_${Math.floor(i / entriesPerBank) + 1}.json`,