Compare commits

...
4 Commits
Author SHA1 Message Date
sudacode d8c2315405 perf(stats): speed up dashboard queries on large databases (#322) 2026-11-03 22:04:01 -08:00
sudacode 8976d78390 feat(controller): route status messages through notifications
- Send controller connected and disabled messages over a new IPC channel so they follow ankiConnect.behavior.notificationType
- Remove the separate controller status toast from the overlay
- Only announce connected controllers when controller.enabled is on
- Announce a controller again after it reconnects, and update usage docs
2026-11-03 21:55:55 -08:00
sudacode 1c50d249e0 fix(mpv): reset per-file state when a same-file reload skips the path clear (#323)
mpv only sends the latest value of an observed property, and only after its
IPC thread has handled queued commands and events. A same-file
`loadfile replace` can therefore arrive with no `path=null`, so path
comparisons see no change, the autoplay gate stays released for that path and
drops every ready signal, and playback stays paused.

Track an `end-file` that mpv has not followed with a different path. If it is
still pending at `file-loaded`, replay the clear and read the path back, so
consumers see the same sequence as any other load. `file-loaded` waits for the
reply so plugins can read their media snapshot during the event.

Fixes #320
2026-11-03 21:40:14 -08:00
sudacode 8e29cb3c55 fix(controller): pick up macOS presses while mpv is in front (#321) 2026-11-03 21:37:11 -08:00
48 changed files with 1566 additions and 559 deletions

No files matched your search

+1 -1
View File
@@ -49,7 +49,7 @@ Start here, then leave this file.
- Build/release scripts (`scripts/**`): `bun run test:scripts`
- Internal docs, `AGENTS.md`, or repo skills: `bun run test:docs:kb`
- User-facing `docs-site/`: `bun run docs:test`, then `bun run docs:build`
- macOS mpv window helper (`scripts/get-mpv-window-macos.swift`): compile it on macOS with `bun run build`
- macOS Swift helpers (`scripts/get-mpv-window-macos.swift`, `scripts/macos-controller-watch.swift`): compile them on macOS with `bun run build`
- Test lanes are directory-discovered via `scripts/test-lanes.ts`; never hand-list test files in `package.json`
## Docs Upkeep
@@ -0,0 +1,4 @@
type: changed
area: overlay
- Controller connected and controller-disabled messages are now regular notifications that follow `ankiConnect.behavior.notificationType` (overlay, system, both or none) instead of a separate pill in the corner of the overlay. Connected notices only appear when `controller.enabled` is on.
@@ -0,0 +1,4 @@
type: fixed
area: overlay
- On macOS, controller buttons now work while mpv is in front: a press brings the overlay forward and runs that button's action, instead of doing nothing until you click the overlay.
+4
View File
@@ -0,0 +1,4 @@
type: fixed
area: playback
- Reloading the same file in mpv (`loadfile <current path> replace`) could intermittently leave playback paused forever. SubMiner now treats that reload like any other newly loaded file, so it resets the subtitle state and resumes once the file is ready.
+8
View File
@@ -0,0 +1,8 @@
type: fixed
area: stats
- The stats dashboard loads much faster on large databases. Overview, media and library detail, and trends no longer re-count every recorded word in the whole database on each request, so these views only cost as much as the sessions they show. On a ~1 GB database the overview went from ~1.4 s to ~0.14 s, media detail from ~1.5 s to ~0.08 s, the 30-day trends from ~1.5 s to ~0.06 s, and all-time trends from ~1.6 s to ~0.2 s.
- The Vocabulary tab's summary and word list load several times faster (summary ~2 s to ~0.2 s, word list ~0.6 s to ~0.2 s).
- The Trends tab downloads far less data: the per-title library charts now arrive as daily values and are totalled in the dashboard, so the all-time response shrank from ~3.9 MB to ~0.2 MB.
- Sentence search with headword matching returns several times faster (~0.3 s to ~0.08 s per query).
- Vocabulary totals and new-word charts stay correct after running vocabulary cleanup. Words it repaired could be counted or hidden by mistake; the first launch after updating corrects them in the background.
+3 -1
View File
@@ -160,7 +160,7 @@ On Wayland, the tray icon only appears if your panel provides a StatusNotifier (
You can drive the overlay with a gamepad.
1. Set `controller.enabled` to `true` in your config.
2. Connect a controller. SubMiner uses the first one it sees.
2. Connect a controller and press a button. SubMiner uses the first one it sees and announces it through [`ankiConnect.behavior.notificationType`](/configuration#ankiconnect).
3. Press `Y` on the controller to turn on keyboard-only mode. Most controller actions only work in this mode.
4. Move between words with the left stick, press `A` to look one up, and `X` to mine it.
@@ -195,6 +195,8 @@ The media timing review and duplicate card (field grouping) dialogs have their o
On controllers that report the W3C standard layout, the default quit button lands on `L2` instead of `Select`, and the default pause button on `Start` instead of `L3`. The default right-stick popup jump does nothing on these controllers; click **Learn** on **Popup Jump (Stick)** and move the right stick up or down to set it. Remap anything else with `Alt+C`, which names buttons according to the layout your controller reports. All options are in [Configuration](/configuration#controller-support).
On macOS, a controller only reaches the app in front. When mpv is in front, a controller button brings the overlay to the front and then runs that button's action. With more than one controller connected, that first press only brings the overlay to the front; press the button again to run its action. Stick movements only work once the overlay is in front. Buttons pressed while another app is in front are left alone.
## Development profiles
`--dev` and `--debug` use a separate `SubMiner-dev` profile. They do not read or modify dictionaries and configuration from the installed app unless `SUBMINER_USE_PRODUCTION_PROFILE=1` is explicitly set. See [Development](/development#run-locally).
+3 -2
View File
@@ -17,11 +17,12 @@ Trend charts now consume one chart-oriented backend payload from `/api/stats/tre
- rollup-backed:
- activity charts
- cumulative watch/cards/tokens/sessions trends
- per-anime watch/cards/tokens/episodes series
- per-anime watch/cards/tokens/episodes series (`animeDaily`: sparse per-title values for active days only; `TrendsTab` builds the running totals with `buildAnimeCumulative`, because a dense title x day payload grew to several MB)
- session-metric-backed:
- lookup trends
- lookup rate trends
- watch-time by day-of-week/hour
- rollup word counts and session metrics share one session query per request (`getTrendSessionRows`), so the per-session display-word count runs once instead of once per rollup query
- vocabulary-backed:
- new-words trend reads permanent daily lexical rollups
- rollup rows count only vocabulary-visible tokens and normalize mixed legacy timestamp units
@@ -39,6 +40,6 @@ Stats data crosses the process boundary over HTTP only. Shared JSON models live
The stats preload bridge remains limited to native window behavior such as confirmation-dialog layering. Do not add stats data request channels back to Electron IPC.
The stats UI should treat the trends payload as chart-ready data. Presentation-only work in the client is fine, but rebuilding the main trend datasets from raw sessions should stay out of the render path.
The stats UI should treat the trends payload as chart-ready data. Presentation-only work in the client is fine (including the per-anime running totals, memoized per payload), but rebuilding the main trend datasets from raw sessions should stay out of the render path.
For session detail timelines, omitting `limit` now means "return the full retained session telemetry/history". Explicit `limit` remains available for bounded callers, but the default stats UI path should not trim long sessions to the newest 200 samples.
+4
View File
@@ -185,6 +185,10 @@
{
"from": "dist/scripts/get-mpv-window-macos",
"to": "scripts/get-mpv-window-macos"
},
{
"from": "dist/scripts/macos-controller-watch",
"to": "scripts/macos-controller-watch"
}
],
"files": [
+22 -19
View File
@@ -4,6 +4,8 @@ const path = require('node:path');
const LINUX_FFMPEG_LIBRARY = 'libffmpeg.so';
const MACOS_WINDOW_HELPER = 'get-mpv-window-macos';
const MACOS_CONTROLLER_HELPER = 'macos-controller-watch';
const MACOS_HELPERS = [MACOS_WINDOW_HELPER, MACOS_CONTROLLER_HELPER];
const DEFAULT_MACOS_APP_BUNDLE = 'SubMiner.app';
function resolveMacOSAppBundlePath(context) {
@@ -52,7 +54,7 @@ async function stageLinuxAppImageSharedLibrary(
return true;
}
async function verifyMacOSWindowHelper(
async function verifyMacOSHelpers(
context,
deps = {
access: (filePath, mode) => fs.access(filePath, mode),
@@ -60,27 +62,28 @@ async function verifyMacOSWindowHelper(
) {
if (context.electronPlatformName !== 'darwin') return false;
const appBundlePath = resolveMacOSAppBundlePath(context);
const helperPath = path.join(
appBundlePath,
const scriptsDir = path.join(
resolveMacOSAppBundlePath(context),
'Contents',
'Resources',
'scripts',
MACOS_WINDOW_HELPER,
);
try {
await deps.access(helperPath, fsSync.constants.X_OK);
} catch (error) {
if (error && typeof error === 'object' && error.code === 'ENOENT') {
throw new Error(
`macOS packaging requires ${MACOS_WINDOW_HELPER} at ${helperPath}. Run bun run build:assets on macOS with swiftc available before packaging.`,
);
for (const helperName of MACOS_HELPERS) {
const helperPath = path.join(scriptsDir, helperName);
try {
await deps.access(helperPath, fsSync.constants.X_OK);
} catch (error) {
if (error && typeof error === 'object' && error.code === 'ENOENT') {
throw new Error(
`macOS packaging requires ${helperName} at ${helperPath}. Run bun run build:assets on macOS with swiftc available before packaging.`,
);
}
if (error && typeof error === 'object' && error.code === 'EACCES') {
throw new Error(`macOS helper is not executable: ${helperPath}`);
}
throw error;
}
if (error && typeof error === 'object' && error.code === 'EACCES') {
throw new Error(`macOS window helper is not executable: ${helperPath}`);
}
throw error;
}
return true;
@@ -103,17 +106,17 @@ async function stageBundledBunRuntime(context, deps = {}) {
async function afterPack(context, deps = {}) {
await stageLinuxAppImageSharedLibrary(context);
await verifyMacOSWindowHelper(context);
await verifyMacOSHelpers(context);
await stageBundledBunRuntime(context, deps);
await (deps.auditPackage ?? require('./package-audit.cjs').auditPackage)(context);
}
module.exports = {
LINUX_FFMPEG_LIBRARY,
MACOS_WINDOW_HELPER,
MACOS_HELPERS,
resolveMacOSAppBundlePath,
stageBundledBunRuntime,
stageLinuxAppImageSharedLibrary,
verifyMacOSWindowHelper,
verifyMacOSHelpers,
default: afterPack,
};
+19 -21
View File
@@ -6,13 +6,13 @@ import test from 'node:test';
const {
LINUX_FFMPEG_LIBRARY,
MACOS_WINDOW_HELPER,
MACOS_HELPERS,
default: afterPack,
stageLinuxAppImageSharedLibrary,
verifyMacOSWindowHelper,
verifyMacOSHelpers,
} = require('./electron-builder-after-pack.cjs') as {
LINUX_FFMPEG_LIBRARY: string;
MACOS_WINDOW_HELPER: string;
MACOS_HELPERS: string[];
default: (
context: {
appOutDir: string;
@@ -34,7 +34,7 @@ const {
appOutDir: string;
electronPlatformName: string;
}) => Promise<boolean>;
verifyMacOSWindowHelper: (context: {
verifyMacOSHelpers: (context: {
appOutDir: string;
electronPlatformName: string;
}) => Promise<boolean>;
@@ -88,24 +88,19 @@ test('stageLinuxAppImageSharedLibrary skips non-Linux packaging contexts', async
}
});
test('verifyMacOSWindowHelper accepts packaged macOS helper binary', async () => {
test('verifyMacOSHelpers accepts packaged macOS helper binaries', async () => {
const workspace = createWorkspace('subminer-after-pack-macos-helper');
const appOutDir = path.join(workspace, 'SubMiner-darwin-arm64');
const helperPath = path.join(
appOutDir,
'SubMiner.app',
'Contents',
'Resources',
'scripts',
MACOS_WINDOW_HELPER,
);
const scriptsDir = path.join(appOutDir, 'SubMiner.app', 'Contents', 'Resources', 'scripts');
fs.mkdirSync(path.dirname(helperPath), { recursive: true });
fs.writeFileSync(helperPath, 'compiled helper', 'utf8');
fs.chmodSync(helperPath, 0o755);
fs.mkdirSync(scriptsDir, { recursive: true });
for (const helperName of MACOS_HELPERS) {
fs.writeFileSync(path.join(scriptsDir, helperName), 'compiled helper', 'utf8');
fs.chmodSync(path.join(scriptsDir, helperName), 0o755);
}
try {
const verified = await verifyMacOSWindowHelper({
const verified = await verifyMacOSHelpers({
appOutDir,
electronPlatformName: 'darwin',
});
@@ -116,19 +111,22 @@ test('verifyMacOSWindowHelper accepts packaged macOS helper binary', async () =>
}
});
test('verifyMacOSWindowHelper throws when macOS helper binary is missing', async () => {
test('verifyMacOSHelpers throws when a macOS helper binary is missing', async () => {
const workspace = createWorkspace('subminer-after-pack-macos-helper-missing');
const appOutDir = path.join(workspace, 'SubMiner-darwin-arm64');
const scriptsDir = path.join(appOutDir, 'SubMiner.app', 'Contents', 'Resources', 'scripts');
fs.mkdirSync(path.join(appOutDir, 'SubMiner.app', 'Contents', 'Resources'), { recursive: true });
fs.mkdirSync(scriptsDir, { recursive: true });
fs.writeFileSync(path.join(scriptsDir, 'get-mpv-window-macos'), 'compiled helper', 'utf8');
fs.chmodSync(path.join(scriptsDir, 'get-mpv-window-macos'), 0o755);
try {
await assert.rejects(
verifyMacOSWindowHelper({
verifyMacOSHelpers({
appOutDir,
electronPlatformName: 'darwin',
}),
/macOS packaging requires get-mpv-window-macos/,
/macOS packaging requires macos-controller-watch/,
);
} finally {
fs.rmSync(workspace, { recursive: true, force: true });
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env swift
//
// macos-controller-watch.swift
// SubMiner - Report controller button presses on macOS while SubMiner is in the background
//
// macOS only delivers GameController input to the frontmost app, so the overlay sees no
// presses while mpv is in front. This helper opts in to background events and prints one
// line per press so SubMiner can focus the overlay and replay the press there.
//
// Usage: macos-controller-watch (long-running; exits when stdin closes)
// Output: "press <index> <connected pads>" per button press, using the W3C standard gamepad
// button indices. SubMiner only replays a press when one pad is connected: nothing identifies
// a pad to both GameController and Chromium, so with several it could hit the wrong one.
//
import Cocoa
import Foundation
import GameController
private func emit(_ line: String) {
FileHandle.standardOutput.write(Data((line + "\n").utf8))
}
/// Standard gamepad mapping (https://w3c.github.io/gamepad/#remapping), matching the
/// indices Chromium reports for the same pad.
private func standardButtons(_ pad: GCExtendedGamepad) -> [(Int, GCControllerButtonInput?)] {
return [
(0, pad.buttonA),
(1, pad.buttonB),
(2, pad.buttonX),
(3, pad.buttonY),
(4, pad.leftShoulder),
(5, pad.rightShoulder),
(6, pad.leftTrigger),
(7, pad.rightTrigger),
(8, pad.buttonOptions),
(9, pad.buttonMenu),
(10, pad.leftThumbstickButton),
(11, pad.rightThumbstickButton),
(12, pad.dpad.up),
(13, pad.dpad.down),
(14, pad.dpad.left),
(15, pad.dpad.right),
(16, pad.buttonHome),
]
}
private func connectedPadCount() -> Int {
return GCController.controllers().filter { $0.extendedGamepad != nil }.count
}
private func watch(_ controller: GCController) {
guard let pad = controller.extendedGamepad else { return }
for (index, button) in standardButtons(pad) {
button?.pressedChangedHandler = { _, _, pressed in
if pressed {
emit("press \(index) \(connectedPadCount())")
}
}
}
}
// Exit with SubMiner: it holds our stdin open for as long as it runs.
private func exitWhenStdinCloses() {
Thread.detachNewThread {
while !FileHandle.standardInput.availableData.isEmpty {}
exit(0)
}
}
let app = NSApplication.shared
app.setActivationPolicy(.prohibited)
GCController.shouldMonitorBackgroundEvents = true
NotificationCenter.default.addObserver(
forName: .GCControllerDidConnect,
object: nil,
queue: .main
) { notification in
if let controller = notification.object as? GCController {
watch(controller)
}
}
GCController.controllers().forEach(watch)
exitWhenStdinCloses()
app.run()
+21 -21
View File
@@ -14,9 +14,8 @@ const settingsOutputDir = path.join(repoRoot, 'dist', 'settings');
const syncUiSourceDir = path.join(repoRoot, 'src', 'syncui');
const syncUiOutputDir = path.join(repoRoot, 'dist', 'syncui');
const scriptsOutputDir = path.join(repoRoot, 'dist', 'scripts');
const macosHelperSourcePath = path.join(scriptDir, 'get-mpv-window-macos.swift');
const macosHelperBinaryPath = path.join(scriptsOutputDir, 'get-mpv-window-macos');
const macosHelperSourceCopyPath = path.join(scriptsOutputDir, 'get-mpv-window-macos.swift');
// Swift helpers compiled on macOS and shipped in Resources/scripts.
const MACOS_HELPERS = ['get-mpv-window-macos', 'macos-controller-watch'];
function ensureDir(dirPath) {
fs.mkdirSync(dirPath, { recursive: true });
@@ -45,9 +44,10 @@ function copySyncUiAssets() {
copyAssets(syncUiSourceDir, syncUiOutputDir, 'syncui');
}
function fallbackToMacosSource() {
copyFile(macosHelperSourcePath, macosHelperSourceCopyPath);
process.stdout.write(`Staged macOS helper source fallback: ${macosHelperSourceCopyPath}\n`);
function fallbackToMacosSource(helperName) {
const sourceCopyPath = path.join(scriptsOutputDir, `${helperName}.swift`);
copyFile(path.join(scriptDir, `${helperName}.swift`), sourceCopyPath);
process.stdout.write(`Staged macOS helper source fallback: ${sourceCopyPath}\n`);
}
// Pin the minimum macOS to the app's own floor (Electron's `minos`). Without an
@@ -64,34 +64,34 @@ function shouldSkipMacosHelperBuild() {
return process.env.SUBMINER_SKIP_MACOS_HELPER_BUILD === '1';
}
function buildMacosHelper() {
function buildMacosHelper(helperName) {
if (shouldSkipMacosHelperBuild()) {
process.stdout.write('Skipping macOS helper build (SUBMINER_SKIP_MACOS_HELPER_BUILD=1)\n');
fallbackToMacosSource();
fallbackToMacosSource(helperName);
return;
}
if (process.platform !== 'darwin') {
process.stdout.write('Skipping macOS helper build (not on macOS)\n');
fallbackToMacosSource();
fallbackToMacosSource(helperName);
return;
}
ensureDir(scriptsOutputDir);
const sourcePath = path.join(scriptDir, `${helperName}.swift`);
const binaryPath = path.join(scriptsOutputDir, helperName);
try {
execFileSync(
'swiftc',
['-O', '-target', macosHelperTarget(), macosHelperSourcePath, '-o', macosHelperBinaryPath],
{
stdio: 'inherit',
},
);
fs.chmodSync(macosHelperBinaryPath, 0o755);
process.stdout.write(`Built macOS helper: ${macosHelperBinaryPath}\n`);
execFileSync('swiftc', ['-O', '-target', macosHelperTarget(), sourcePath, '-o', binaryPath], {
stdio: 'inherit',
});
fs.chmodSync(binaryPath, 0o755);
process.stdout.write(`Built macOS helper: ${binaryPath}\n`);
} catch (error) {
process.stdout.write('Failed to compile macOS helper; using source fallback.\n');
fallbackToMacosSource();
process.stdout.write(`Failed to compile macOS helper ${helperName}; using source fallback.\n`);
// A binary from an earlier build would otherwise pass the after-pack check and ship stale.
fs.rmSync(binaryPath, { force: true });
fallbackToMacosSource(helperName);
if (error instanceof Error) {
process.stderr.write(`${error.message}\n`);
}
@@ -117,7 +117,7 @@ function main() {
copyRendererAssets();
copySettingsAssets();
copySyncUiAssets();
buildMacosHelper();
MACOS_HELPERS.forEach(buildMacosHelper);
stageIntegrationKeys();
}
@@ -189,7 +189,7 @@ const TRENDS_DASHBOARD = {
lastWatched: 20_000,
},
],
animeCumulative: {
animeDaily: {
watchTime: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 25 }],
episodes: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 1 }],
cards: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 5 }],
@@ -745,7 +745,10 @@ test('getTrendsDashboard returns chart-ready aggregated series', () => {
assert.equal(dashboard.ratios.readingSpeed[0]?.value, +(120 / 30).toFixed(1));
assert.equal(dashboard.ratios.readingSpeed[1]?.value, +(140 / 45).toFixed(1));
assert.equal(dashboard.librarySummary[0]?.title, 'Trend Dashboard Anime');
assert.equal(dashboard.animeCumulative.watchTime[1]?.value, 75);
assert.deepEqual(
dashboard.animeDaily.watchTime.map((point) => point.value).sort((a, b) => a - b),
[30, 45],
);
assert.equal(
dashboard.patterns.watchTimeByDayOfWeek.reduce((sum, point) => sum + point.value, 0),
75,
@@ -819,7 +822,7 @@ test('getTrendsDashboard redacts legacy Jellyfin stream titles', () => {
const dashboard = getTrendsDashboard(db, 'all', 'day');
const titles = [
...dashboard.animeCumulative.watchTime.map((point) => point.animeTitle),
...dashboard.animeDaily.watchTime.map((point) => point.animeTitle),
...dashboard.librarySummary.map((row) => row.title),
];
@@ -1521,6 +1524,8 @@ test('word-count read models use filtered persisted occurrences with raw fallbac
const startedAtMs = 1_700_000_000_000;
const withOccurrences = startSessionRecord(db, videoId, startedAtMs);
const fallbackOnly = startSessionRecord(db, videoId, startedAtMs + 60_000);
// Still running, so its words stay out of the all-time total.
const stillActive = startSessionRecord(db, videoId, startedAtMs + 120_000);
db.prepare(
`
@@ -1543,6 +1548,14 @@ test('word-count read models use filtered persisted occurrences with raw fallbac
occurrenceCount: 2,
startedAtMs,
});
insertFilteredWordOccurrence(db, {
sessionId: stillActive.sessionId,
videoId,
lineIndex: 3,
occurrenceCount: 4,
startedAtMs: startedAtMs + 120_000,
headword: '犬',
});
insertFilteredWordOccurrence(db, {
sessionId: withOccurrences.sessionId,
videoId,
@@ -5075,7 +5088,10 @@ test('getTrendsDashboard rollup word metrics keep persisted totals over partial
assert.equal(dashboard.ratios.lookupsPerHundred[0]?.value, 200);
assert.equal(dashboard.librarySummary[0]?.words, 30);
assert.equal(dashboard.librarySummary[0]?.lookupsPerHundred, 33.3);
assert.equal(dashboard.animeCumulative.words.at(-1)?.value, 30);
assert.equal(
dashboard.animeDaily.words.reduce((sum, point) => sum + point.value, 0),
30,
);
} finally {
db.close();
cleanupDbPath(dbPath);
@@ -14,9 +14,11 @@ import {
getVocabularySummary,
replaceStatsExcludedWords,
} from './query-lexical';
import { cleanupVocabularyStats } from './query-maintenance';
import { Database } from './sqlite';
import type { DatabaseSync } from './sqlite';
import { createTrackerPreparedStatements, ensureSchema, executeQueuedWrite } from './storage';
import { isVocabularyStatsRowVisible, type VocabularyVisibilityRow } from './vocabulary-visibility';
import { BASE_MS, createDb, seedEndedSession, seedVideo } from './__tests__/lifetime-test-fixtures';
function makeDbPath(): string {
@@ -384,7 +386,7 @@ test('current lexical rollup readiness accepts legacy integer state storage', ()
state_value INTEGER NOT NULL
);
INSERT INTO imm_rollup_state(state_key, state_value)
VALUES ('lexical_daily_rollups_version', 3);
VALUES ('lexical_daily_rollups_version', 4);
`);
assert.equal(areLexicalDailyRollupsReady(db), true);
@@ -394,6 +396,185 @@ test('current lexical rollup readiness accepts legacy integer state storage', ()
}
});
function insertWord(
db: DatabaseSync,
word: {
headword: string;
word: string;
reading: string;
partOfSpeech: string;
pos1: string;
pos2: string;
},
firstSeen: number,
): number {
const result = db
.prepare(
`INSERT INTO imm_words(
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen, frequency
) VALUES (?, ?, ?, ?, ?, ?, '', ?, ?, 1)`,
)
.run(
word.headword,
word.word,
word.reading,
word.partOfSpeech,
word.pos1,
word.pos2,
firstSeen,
firstSeen,
);
return Number(result.lastInsertRowid);
}
function assertPersistedVisibilityMatchesPredicate(db: DatabaseSync): void {
const rows = db
.prepare(
`SELECT headword, word, reading, part_of_speech AS partOfSpeech, pos1, pos2, pos3,
frequency_rank AS frequencyRank, vocabulary_visible AS vocabularyVisible
FROM imm_words`,
)
.all() as Array<VocabularyVisibilityRow & { vocabularyVisible: number }>;
assert.ok(rows.length > 0);
assert.deepEqual(
rows.map((row) => [row.headword, row.vocabularyVisible === 1]),
rows.map((row) => [row.headword, isVocabularyStatsRowVisible(row)]),
);
}
test('cleanupVocabularyStats re-derives visibility for a repaired word', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const epochDay = 19_500;
const firstSeen = epochDay * 86_400 + 43_200;
const legacyWord = {
headword: '旧',
word: '旧',
reading: '',
partOfSpeech: 'particle',
pos1: '助詞',
pos2: '係助詞',
};
const legacyId = insertWord(db, legacyWord, firstSeen);
rebuildLexicalDailyRollups(db);
assert.equal(getVocabularySummary(db, null).uniqueWords, 0);
await cleanupVocabularyStats(db, {
resolveLegacyPos: async (row) =>
row.id === legacyId
? {
headword: '旧',
reading: 'きゅう',
partOfSpeech: 'noun',
pos1: '名詞',
pos2: '一般',
pos3: '',
}
: null,
});
assertPersistedVisibilityMatchesPredicate(db);
assert.equal(getVocabularySummary(db, null).uniqueWords, 1);
assert.equal(getLexicalDailyRollups(db)[0]?.wordCount, 1);
} finally {
db.close();
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
}
});
test('cleanupVocabularyStats re-derives visibility for a merged duplicate', async () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const firstSeen = 19_500 * 86_400 + 43_200;
// The merge fills these blank POS fields from the repair, which hides the word.
insertWord(
db,
{
headword: '知る',
word: '知っている',
reading: 'しっている',
partOfSpeech: 'noun',
pos1: '',
pos2: '',
},
firstSeen,
);
const legacyId = insertWord(
db,
{
headword: '知っている',
word: '知っている',
reading: '',
partOfSpeech: 'verb',
pos1: '動詞',
pos2: '自立',
},
firstSeen,
);
rebuildLexicalDailyRollups(db);
assert.equal(getVocabularySummary(db, null).uniqueWords, 2);
await cleanupVocabularyStats(db, {
resolveLegacyPos: async (row) =>
row.id === legacyId
? {
headword: '知る',
reading: 'しっている',
partOfSpeech: 'particle',
pos1: '助詞',
pos2: '係助詞',
pos3: '',
}
: null,
});
assertPersistedVisibilityMatchesPredicate(db);
assert.equal(getVocabularySummary(db, null).uniqueWords, 0);
assert.deepEqual(getLexicalDailyRollups(db), []);
} finally {
db.close();
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
}
});
test('a queued word write persists visibility from the stored row', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
try {
ensureSchema(db);
const firstSeen = 19_500 * 86_400 + 43_200;
executeQueuedWrite(
{
kind: 'word',
headword: 'は',
word: 'は',
reading: 'は',
partOfSpeech: 'particle',
pos1: '助詞',
pos2: '係助詞',
pos3: '',
firstSeen,
lastSeen: firstSeen,
frequencyRank: null,
},
createTrackerPreparedStatements(db),
);
assertPersistedVisibilityMatchesPredicate(db);
assert.equal(getVocabularySummary(db, null).uniqueWords, 0);
} finally {
db.close();
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
}
});
test('imm_words persists vocabulary visibility for rollup maintenance', () => {
const dbPath = makeDbPath();
const db = new Database(dbPath);
@@ -19,8 +19,9 @@ const LOCAL_EPOCH_DAY_SQL = `
`;
// v3 re-derives visibility for rows a re-seen word forced visible and repairs
// last_seen values left behind by session deletes.
const LEXICAL_DAILY_ROLLUP_VERSION = '3';
// last_seen values left behind by session deletes. v4 re-derives visibility for
// rows vocabulary cleanup repaired without updating it.
const LEXICAL_DAILY_ROLLUP_VERSION = '4';
const LEXICAL_DAILY_ROLLUP_VERSION_KEY = 'lexical_daily_rollups_version';
const VOCABULARY_VISIBILITY_SCAN_BATCH_SIZE = 5_000;
@@ -16,7 +16,7 @@ import type {
WordDetailRow,
WordOccurrenceRow,
} from './types';
import { fromDbTimestamp, toDbTimestamp } from './query-shared';
import { forEachIdChunk, fromDbTimestamp, makePlaceholders, toDbTimestamp } from './query-shared';
import { nowMs } from './time';
import { narrowFullwidthKana, widenHalfwidthKana } from '../../utils/halfwidth-kana';
import {
@@ -106,42 +106,58 @@ export function getVocabularyStats(
const whereClause = hasExclude
? `WHERE (part_of_speech IS NULL OR part_of_speech NOT IN (${placeholders}))`
: '';
// The page is selected before the join so `animeCount` is only computed for the
// rows being returned. Aggregating first made every request walk each word's
// entire occurrence history — seconds of blocked event loop on a large library,
// because only the ordering, not the aggregate, decides which rows survive.
// Rows are paged and filtered before `animeCount` is computed, and only the
// rows being returned are counted. The count walks each word's entire
// occurrence history, and the most frequent words hold most occurrences, so
// counting the oversampled page or aggregating first cost seconds of blocked
// event loop on a large library.
const stmt = db.prepare(`
WITH page AS (
SELECT id, headword, word, reading, part_of_speech, pos1, pos2, pos3,
frequency, frequency_rank, first_seen, last_seen
FROM imm_words
${whereClause}
ORDER BY frequency DESC, id
LIMIT ? OFFSET ?
)
SELECT p.id AS wordId, p.headword, p.word, p.reading,
p.part_of_speech AS partOfSpeech, p.pos1, p.pos2, p.pos3,
p.frequency, p.frequency_rank AS frequencyRank,
p.first_seen AS firstSeen, p.last_seen AS lastSeen,
COUNT(DISTINCT sl.anime_id) AS animeCount
FROM page p
LEFT JOIN imm_word_line_occurrences o ON o.word_id = p.id
LEFT JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id AND sl.anime_id IS NOT NULL
GROUP BY p.id
ORDER BY p.frequency DESC, p.id
SELECT id AS wordId, headword, word, reading,
part_of_speech AS partOfSpeech, pos1, pos2, pos3,
frequency, frequency_rank AS frequencyRank,
first_seen AS firstSeen, last_seen AS lastSeen
FROM imm_words
${whereClause}
ORDER BY frequency DESC, id
LIMIT ? OFFSET ?
`);
const visibleRows: VocabularyStatsRow[] = [];
const visibleRows: Array<Omit<VocabularyStatsRow, 'animeCount'>> = [];
let offset = 0;
while (visibleRows.length < limit) {
const params = hasExclude ? [...excludePos, pageSize, offset] : [pageSize, offset];
const page = stmt.all(...params) as VocabularyStatsRow[];
const page = stmt.all(...params) as Array<Omit<VocabularyStatsRow, 'animeCount'>>;
if (page.length === 0) break;
visibleRows.push(...page.filter(isVocabularyStatsRowVisible));
offset += page.length;
}
return visibleRows.slice(0, limit);
const rows = visibleRows.slice(0, limit);
const animeCounts = getWordAnimeCounts(
db,
rows.map((row) => row.wordId),
);
return rows.map((row) => ({ ...row, animeCount: animeCounts.get(row.wordId) ?? 0 }));
}
/** Number of distinct anime each word appears in, keyed by word id. */
function getWordAnimeCounts(db: DatabaseSync, wordIds: number[]): Map<number, number> {
const counts = new Map<number, number>();
forEachIdChunk(wordIds, (chunk) => {
const rows = db
.prepare(
`
SELECT o.word_id AS wordId, COUNT(DISTINCT sl.anime_id) AS animeCount
FROM imm_word_line_occurrences o
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
WHERE o.word_id IN (${makePlaceholders(chunk)}) AND sl.anime_id IS NOT NULL
GROUP BY o.word_id
`,
)
.all(...chunk) as Array<{ wordId: number; animeCount: number }>;
for (const row of rows) counts.set(row.wordId, row.animeCount);
});
return counts;
}
/**
@@ -276,8 +292,12 @@ export function getVocabularySummary(
nowMs: number = Date.now(),
scanBatchSize: number = VOCABULARY_SUMMARY_SCAN_BATCH_SIZE,
): VocabularyStatsSummary {
// Visibility and exclusion rules live in JS, so rows are scanned in id-keyed
// batches to keep memory bounded on large vocabularies.
// Exclusion and known-word rules live in JS, so rows are scanned in id-keyed
// batches to keep memory bounded on large vocabularies. Once the lexical
// rollups are ready, `vocabulary_visible` holds the persisted result of
// isVocabularyStatsRowVisible (the same column the new-word charts count), so
// the scan skips re-running the subtitle filter rules for every word.
const persistedVisibility = areLexicalDailyRollupsReady(db);
const scanStmt = db.prepare(`
SELECT id AS wordId, headword, word, reading,
part_of_speech AS partOfSpeech, pos1, pos2, pos3,
@@ -285,10 +305,11 @@ export function getVocabularySummary(
first_seen AS firstSeen, last_seen AS lastSeen,
0 AS animeCount
FROM imm_words
WHERE id > ?
WHERE id > ?${persistedVisibility ? ' AND vocabulary_visible = 1' : ''}
ORDER BY id
LIMIT ?
`);
const isVisible = persistedVisibility ? () => true : isVocabularyStatsRowVisible;
const excludedAliases = new Set(
getStatsExcludedWords(db).flatMap((word) => excludedVocabularyAliases(word)),
);
@@ -311,7 +332,7 @@ export function getVocabularySummary(
lastId = words[words.length - 1]!.wordId;
for (const word of words) {
if (
!isVocabularyStatsRowVisible(word) ||
!isVisible(word) ||
excludedVocabularyAliases(word).some((alias) => excludedAliases.has(alias))
) {
continue;
@@ -489,15 +510,16 @@ export function searchSubtitleSentences(
narrowFullwidthKana(widenedTerm),
]).map((variant) => `%${escapeLikeTerm(variant)}%`);
const headwords = getHeadwordCandidatesForSentenceSearchTerm(term, options);
// Uncorrelated on purpose: SQLite resolves the headword's lines once through
// the word indexes. A per-line EXISTS probed every subtitle line (~0.2 s).
const headwordClause =
headwords.length > 0
? `
OR EXISTS (
SELECT 1
FROM imm_word_line_occurrences o
JOIN imm_words w ON w.id = o.word_id
WHERE o.line_id = l.line_id
AND w.headword IN (${headwords.map(() => '?').join(', ')})
OR l.line_id IN (
SELECT o.line_id
FROM imm_words w
JOIN imm_word_line_occurrences o ON o.word_id = w.id
WHERE w.headword IN (${headwords.map(() => '?').join(', ')})
)
`
: '';
@@ -18,7 +18,6 @@ import type {
} from './types';
import {
ACTIVE_SESSION_METRICS_CTE,
SESSION_WORD_COUNTS_SELECT,
fromDbTimestamp,
resolvedCoverBlobExpr,
sessionDisplayWordsExpr,
@@ -123,7 +122,7 @@ export function getAnimeAnilistEntries(db: DatabaseSync, animeId: number): Anime
}
export function getAnimeEpisodes(db: DatabaseSync, animeId: number): AnimeEpisodeRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const wordsExpr = sessionDisplayWordsExpr('s', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
@@ -184,7 +183,6 @@ export function getAnimeEpisodes(db: DatabaseSync, animeId: number): AnimeEpisod
FROM imm_videos v
LEFT JOIN imm_sessions s ON s.video_id = v.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE v.anime_id = ?
GROUP BY v.video_id
ORDER BY
@@ -250,7 +248,7 @@ export function getMediaLibrary(db: DatabaseSync): MediaLibraryRow[] {
}
export function getMediaDetail(db: DatabaseSync, videoId: number): MediaDetailRow | null {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const wordsExpr = sessionDisplayWordsExpr('s', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
return db
.prepare(
`
@@ -298,7 +296,6 @@ export function getMediaDetail(db: DatabaseSync, videoId: number): MediaDetailRo
LEFT JOIN imm_youtube_videos yv ON yv.video_id = v.video_id
LEFT JOIN imm_sessions s ON s.video_id = v.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE v.video_id = ?
AND (lm.video_id IS NOT NULL OR s.session_id IS NOT NULL)
GROUP BY v.video_id
@@ -312,7 +309,7 @@ export function getMediaSessions(
videoId: number,
limit = 100,
): SessionSummaryQueryRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const wordsExpr = sessionDisplayWordsExpr('s', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
@@ -333,7 +330,6 @@ export function getMediaSessions(
COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0) AS yomitanLookupCount
FROM imm_sessions s
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
LEFT JOIN imm_videos v ON v.video_id = s.video_id
WHERE s.video_id = ?
ORDER BY s.started_at_ms DESC
@@ -358,12 +354,15 @@ export function getMediaDailyRollups(
videoId: number,
limit = 90,
): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
return db
.prepare(
`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
WITH recent_days AS (
SELECT DISTINCT rollup_day
FROM imm_daily_rollups
WHERE video_id = ?
ORDER BY rollup_day DESC
LIMIT ?
),
daily_word_counts AS (
SELECT
@@ -372,18 +371,15 @@ export function getMediaDailyRollups(
AS INTEGER
) AS rollupDay,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
SUM(${sessionDisplayWordsExpr('s')}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
AND s.video_id = ?
AND CAST(
julianday(CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
AS INTEGER
) >= (SELECT MIN(rollup_day) FROM recent_days)
GROUP BY rollupDay, s.video_id
),
recent_days AS (
SELECT DISTINCT rollup_day
FROM imm_daily_rollups
WHERE video_id = ?
ORDER BY rollup_day DESC
LIMIT ?
)
SELECT
rollup_day AS rollupDayOrMonth,
@@ -416,7 +412,7 @@ export function getMediaDailyRollups(
ORDER BY rollup_day DESC, video_id DESC
`,
)
.all(videoId, limit, videoId) as unknown as ImmersionSessionRollupRow[];
.all(videoId, limit, videoId, videoId) as unknown as ImmersionSessionRollupRow[];
}
export function getAnimeDailyRollups(
@@ -424,12 +420,16 @@ export function getAnimeDailyRollups(
animeId: number,
limit = 90,
): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
return db
.prepare(
`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
WITH recent_days AS (
SELECT DISTINCT r.rollup_day
FROM imm_daily_rollups r
JOIN imm_videos v ON v.video_id = r.video_id
WHERE v.anime_id = ?
ORDER BY r.rollup_day DESC
LIMIT ?
),
daily_word_counts AS (
SELECT
@@ -438,19 +438,15 @@ export function getAnimeDailyRollups(
AS INTEGER
) AS rollupDay,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
SUM(${sessionDisplayWordsExpr('s')}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
AND s.video_id IN (SELECT video_id FROM imm_videos WHERE anime_id = ?)
AND CAST(
julianday(CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
AS INTEGER
) >= (SELECT MIN(rollup_day) FROM recent_days)
GROUP BY rollupDay, s.video_id
),
recent_days AS (
SELECT DISTINCT r.rollup_day
FROM imm_daily_rollups r
JOIN imm_videos v ON v.video_id = r.video_id
WHERE v.anime_id = ?
ORDER BY r.rollup_day DESC
LIMIT ?
)
SELECT r.rollup_day AS rollupDayOrMonth, r.video_id AS videoId,
r.total_sessions AS totalSessions, r.total_active_min AS totalActiveMin,
@@ -481,7 +477,7 @@ export function getAnimeDailyRollups(
ORDER BY r.rollup_day DESC, r.video_id DESC
`,
)
.all(animeId, limit, animeId) as unknown as ImmersionSessionRollupRow[];
.all(animeId, limit, animeId, animeId) as unknown as ImmersionSessionRollupRow[];
}
export function getAnimeCoverArt(db: DatabaseSync, animeId: number): MediaArtRow | null {
@@ -657,7 +653,7 @@ export function getEpisodeWords(db: DatabaseSync, videoId: number, limit = 50):
}
export function getEpisodeSessions(db: DatabaseSync, videoId: number): SessionSummaryQueryRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const wordsExpr = sessionDisplayWordsExpr('s', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
@@ -677,7 +673,6 @@ export function getEpisodeSessions(db: DatabaseSync, videoId: number): SessionSu
FROM imm_sessions s
JOIN imm_videos v ON v.video_id = s.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.video_id = ?
ORDER BY s.started_at_ms DESC
`,
@@ -13,6 +13,7 @@ import { deleteMaintenanceBatch } from './query-delete-maintenance';
import { PartOfSpeech, type MergedToken } from '../../../types';
import { shouldExcludeTokenFromVocabularyPersistence } from '../tokenizer/annotation-stage';
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
import { isVocabularyStatsRowVisible, type VocabularyVisibilityRow } from './vocabulary-visibility';
import {
cleanupUnusedCoverArtBlobHash,
findSharedCoverBlobHash,
@@ -241,6 +242,24 @@ export async function cleanupVocabularyStats(
const deleteOccurrencesStmt = db.prepare(
'DELETE FROM imm_word_line_occurrences WHERE word_id = ?',
);
const selectVisibilityInputsStmt = db.prepare(
`SELECT word, headword, reading, part_of_speech AS partOfSpeech,
pos1, pos2, pos3, frequency_rank AS frequencyRank
FROM imm_words
WHERE id = ?`,
);
const updateVisibilityStmt = db.prepare(
`UPDATE imm_words SET vocabulary_visible = ? WHERE id = ? AND vocabulary_visible IS NOT ?`,
);
// Repairs rewrite the visibility predicate's inputs, and the vocabulary summary
// and new-word rollups read the persisted result, so re-derive it after each one.
const syncWordVisibility = (wordId: number): void => {
// The scan awaits POS resolution, so another writer may have removed the word.
const stored = selectVisibilityInputsStmt.get(wordId) as VocabularyVisibilityRow | null;
if (!stored) return;
const visible = isVocabularyStatsRowVisible(stored) ? 1 : 0;
updateVisibilityStmt.run(visible, wordId, visible);
};
let kept = 0;
let deleted = 0;
let repaired = 0;
@@ -279,6 +298,7 @@ export async function cleanupVocabularyStats(
row.last_seen ?? duplicate.last_seen ?? 0,
duplicate.id,
);
syncWordVisibility(duplicate.id);
deleteStmt.run(row.id);
repaired += 1;
deleted += 1;
@@ -294,6 +314,7 @@ export async function cleanupVocabularyStats(
resolvedPos.pos3,
row.id,
);
syncWordVisibility(row.id);
repaired += 1;
}
@@ -6,9 +6,8 @@ import type {
} from './types';
import {
ACTIVE_SESSION_METRICS_CTE,
SESSION_WORD_COUNTS_CTE,
SESSION_WORD_COUNTS_SELECT,
currentDbTimestamp,
ENDED_SESSIONS_DISPLAY_WORDS_SQL,
fromDbTimestamp,
getLocalEpochDay,
getShiftedLocalDaySec,
@@ -295,32 +294,17 @@ export function getQueryHints(db: DatabaseSync): {
const totalCards = Number(lifetime?.totalCards ?? 0);
const activeDays = Number(lifetime?.activeDays ?? 0);
const lookupWordsExpr = sessionDisplayWordsExpr(
's',
'swc',
'COALESCE(t.tokens_seen, s.tokens_seen)',
);
// Ended sessions are finalized from their last telemetry sample, so their
// own counters already hold the telemetry maxima.
const lookupTotals = db
.prepare(
`
${SESSION_WORD_COUNTS_CTE}
SELECT
COALESCE(SUM(${lookupWordsExpr}), 0) AS totalTokensSeen,
COALESCE(SUM(COALESCE(t.lookup_count, s.lookup_count, 0)), 0) AS totalLookupCount,
COALESCE(SUM(COALESCE(t.lookup_hits, s.lookup_hits, 0)), 0) AS totalLookupHits,
COALESCE(SUM(COALESCE(t.yomitan_lookup_count, s.yomitan_lookup_count, 0)), 0) AS totalYomitanLookupCount
${ENDED_SESSIONS_DISPLAY_WORDS_SQL} AS totalTokensSeen,
COALESCE(SUM(COALESCE(s.lookup_count, 0)), 0) AS totalLookupCount,
COALESCE(SUM(COALESCE(s.lookup_hits, 0)), 0) AS totalLookupHits,
COALESCE(SUM(COALESCE(s.yomitan_lookup_count, 0)), 0) AS totalYomitanLookupCount
FROM imm_sessions s
LEFT JOIN (
SELECT
session_id,
MAX(tokens_seen) AS tokens_seen,
MAX(lookup_count) AS lookup_count,
MAX(lookup_hits) AS lookup_hits,
MAX(yomitan_lookup_count) AS yomitan_lookup_count
FROM imm_session_telemetry
GROUP BY session_id
) t ON t.session_id = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
`,
)
@@ -349,123 +333,157 @@ export function getQueryHints(db: DatabaseSync): {
};
}
export function getDailyRollups(db: DatabaseSync, limit = 60): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
const prepared = db.prepare(`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
),
daily_word_counts AS (
SELECT
CAST(
julianday(CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
AS INTEGER
) AS rollupDay,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
GROUP BY rollupDay, s.video_id
),
recent_days AS (
SELECT DISTINCT rollup_day
FROM imm_daily_rollups
ORDER BY rollup_day DESC
LIMIT ?
)
/** Local calendar day (epoch days) a session row `alias` started on, matching `imm_daily_rollups.rollup_day`. */
export function sessionRollupDaySql(alias: string): string {
return `CAST(julianday(CAST(${alias}.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER)`;
}
/** Local `YYYYMM` month a session row `alias` started in, matching `imm_monthly_rollups.rollup_month`. */
export function sessionRollupMonthSql(alias: string): string {
return `CAST(strftime('%Y%m', CAST(${alias}.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') AS INTEGER)`;
}
/**
* Display-word sums of ended sessions, keyed by `rollupWordCountKey`. Rollup
* rows take these over their persisted token totals when they are larger.
*/
export type RollupWordCounts = Map<string, number>;
export function rollupWordCountKey(rollupDayOrMonth: number, videoId: number | null): string {
return `${rollupDayOrMonth}:${videoId ?? ''}`;
}
export function addRollupWordCount(
counts: RollupWordCounts,
rollupDayOrMonth: number,
videoId: number | null,
words: number,
): void {
const key = rollupWordCountKey(rollupDayOrMonth, videoId);
counts.set(key, (counts.get(key) ?? 0) + words);
}
type RawRollupRow = Omit<ImmersionSessionRollupRow, 'tokensPerMin'>;
/**
* Rollup rows for the `limit` most recent days, without the display-word
* merge. Pair with `applyRollupWordCounts`; `getDailyRollups` does both.
*/
export function selectDailyRollupRows(db: DatabaseSync, limit: number): RawRollupRow[] {
return db
.prepare(
`
SELECT
r.rollup_day AS rollupDayOrMonth,
r.video_id AS videoId,
r.total_sessions AS totalSessions,
r.total_active_min AS totalActiveMin,
r.total_lines_seen AS totalLinesSeen,
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > r.total_tokens_seen THEN dwc.totalTokensSeen
ELSE r.total_tokens_seen
END AS totalTokensSeen,
r.total_tokens_seen AS totalTokensSeen,
r.total_cards AS totalCards,
r.cards_per_hour AS cardsPerHour,
CASE
WHEN r.total_active_min > 0 THEN (
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > r.total_tokens_seen THEN dwc.totalTokensSeen
ELSE r.total_tokens_seen
END
) * 1.0 / r.total_active_min
ELSE NULL
END AS tokensPerMin,
r.lookup_hit_rate AS lookupHitRate
FROM imm_daily_rollups r
LEFT JOIN daily_word_counts dwc
ON dwc.rollupDay = r.rollup_day
AND (
(dwc.videoId IS NULL AND r.video_id IS NULL)
OR dwc.videoId = r.video_id
)
WHERE r.rollup_day IN (SELECT rollup_day FROM recent_days)
WHERE r.rollup_day IN (
SELECT DISTINCT rollup_day FROM imm_daily_rollups ORDER BY rollup_day DESC LIMIT ?
)
ORDER BY r.rollup_day DESC, r.video_id DESC
`);
return prepared.all(limit) as unknown as ImmersionSessionRollupRow[];
`,
)
.all(limit) as unknown as RawRollupRow[];
}
export function getMonthlyRollups(db: DatabaseSync, limit = 24): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
const prepared = db.prepare(`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
),
monthly_word_counts AS (
SELECT
CAST(strftime('%Y%m', CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') AS INTEGER) AS rollupMonth,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
GROUP BY rollupMonth, s.video_id
),
recent_months AS (
SELECT DISTINCT rollup_month
FROM imm_monthly_rollups
ORDER BY rollup_month DESC
LIMIT ?
)
/** Monthly counterpart of `selectDailyRollupRows`. */
export function selectMonthlyRollupRows(db: DatabaseSync, limit: number): RawRollupRow[] {
return db
.prepare(
`
SELECT
r.rollup_month AS rollupDayOrMonth,
r.video_id AS videoId,
r.total_sessions AS totalSessions,
r.total_active_min AS totalActiveMin,
r.total_lines_seen AS totalLinesSeen,
CASE
WHEN mwc.totalTokensSeen IS NOT NULL AND mwc.totalTokensSeen > r.total_tokens_seen THEN mwc.totalTokensSeen
ELSE r.total_tokens_seen
END AS totalTokensSeen,
r.total_tokens_seen AS totalTokensSeen,
r.total_cards AS totalCards,
CASE
WHEN r.total_active_min > 0 THEN (r.total_cards * 60.0) / r.total_active_min
ELSE NULL
END AS cardsPerHour,
CASE
WHEN r.total_active_min > 0 THEN (
CASE
WHEN mwc.totalTokensSeen IS NOT NULL AND mwc.totalTokensSeen > r.total_tokens_seen THEN mwc.totalTokensSeen
ELSE r.total_tokens_seen
END
) * 1.0 / r.total_active_min
ELSE NULL
END AS tokensPerMin,
NULL AS lookupHitRate
FROM imm_monthly_rollups r
LEFT JOIN monthly_word_counts mwc
ON mwc.rollupMonth = r.rollup_month
AND (
(mwc.videoId IS NULL AND r.video_id IS NULL)
OR mwc.videoId = r.video_id
)
WHERE r.rollup_month IN (SELECT rollup_month FROM recent_months)
WHERE r.rollup_month IN (
SELECT DISTINCT rollup_month FROM imm_monthly_rollups ORDER BY rollup_month DESC LIMIT ?
)
ORDER BY r.rollup_month DESC, r.video_id DESC
`);
return prepared.all(limit) as unknown as ImmersionSessionRollupRow[];
`,
)
.all(limit) as unknown as RawRollupRow[];
}
/**
* Raises each rollup's token total to the display-word count of its sessions
* when that is larger (persisted totals can lag behind cleaned-up lines), and
* derives `tokensPerMin` from the result.
*/
export function applyRollupWordCounts(
rows: RawRollupRow[],
counts: RollupWordCounts,
): ImmersionSessionRollupRow[] {
return rows.map((row) => {
const words = counts.get(rollupWordCountKey(row.rollupDayOrMonth, row.videoId));
const totalTokensSeen =
words !== undefined && words > row.totalTokensSeen ? words : row.totalTokensSeen;
return {
...row,
totalTokensSeen,
tokensPerMin: row.totalActiveMin > 0 ? totalTokensSeen / row.totalActiveMin : null,
};
});
}
function getEndedSessionWordCounts(
db: DatabaseSync,
bucketSql: string,
sinceBucket: number,
): RollupWordCounts {
const rows = db
.prepare(
`
SELECT
${bucketSql} AS bucket,
s.video_id AS videoId,
SUM(${sessionDisplayWordsExpr('s')}) AS words
FROM imm_sessions s
WHERE s.ended_at_ms IS NOT NULL
AND ${bucketSql} >= ?
GROUP BY bucket, s.video_id
`,
)
.all(sinceBucket) as Array<{ bucket: number; videoId: number | null; words: number }>;
const counts: RollupWordCounts = new Map();
for (const row of rows) {
addRollupWordCount(counts, row.bucket, row.videoId, row.words);
}
return counts;
}
function withSessionWordCounts(
db: DatabaseSync,
rows: RawRollupRow[],
bucketSql: string,
): ImmersionSessionRollupRow[] {
if (rows.length === 0) {
return [];
}
const sinceBucket = Math.min(...rows.map((row) => row.rollupDayOrMonth));
return applyRollupWordCounts(rows, getEndedSessionWordCounts(db, bucketSql, sinceBucket));
}
export function getDailyRollups(db: DatabaseSync, limit = 60): ImmersionSessionRollupRow[] {
return withSessionWordCounts(db, selectDailyRollupRows(db, limit), sessionRollupDaySql('s'));
}
export function getMonthlyRollups(db: DatabaseSync, limit = 24): ImmersionSessionRollupRow[] {
return withSessionWordCounts(db, selectMonthlyRollupRows(db, limit), sessionRollupMonthSql('s'));
}
@@ -18,24 +18,10 @@ export function visibleWordSql(wordAlias: string): string {
)`;
}
export function filteredWordOccurrenceCountSql(occurrenceAlias: string, wordAlias: string): string {
return `CASE
WHEN ${occurrenceAlias}.word_id IS NOT NULL AND ${visibleWordSql(wordAlias)}
THEN ${occurrenceAlias}.occurrence_count
ELSE 0
END`;
}
export const SESSION_WORD_COUNTS_SELECT = `
SELECT
sl.session_id AS sessionId,
COUNT(DISTINCT sl.line_id) AS persistedLineCount,
COALESCE(SUM(${filteredWordOccurrenceCountSql('wlo', 'w')}), 0) AS filteredWordsSeen
FROM imm_subtitle_lines sl
LEFT JOIN imm_word_line_occurrences wlo ON wlo.line_id = sl.line_id
LEFT JOIN imm_words w ON w.id = wlo.word_id
GROUP BY sl.session_id
`;
// Hidden words are a small set, so SQLite builds this list once per statement
// and checks occurrences against it instead of re-evaluating the term filter
// for every occurrence row.
const HIDDEN_WORD_IDS_SQL = `SELECT hidden_word.id FROM imm_words hidden_word WHERE NOT ${visibleWordSql('hidden_word')}`;
export const ACTIVE_SESSION_METRICS_CTE = `
WITH active_session_metrics AS (
@@ -53,29 +39,77 @@ export const ACTIVE_SESSION_METRICS_CTE = `
JOIN imm_sessions s ON s.session_id = t.session_id
WHERE s.ended_at_ms IS NULL
GROUP BY t.session_id
),
session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
)
`;
export const SESSION_WORD_COUNTS_CTE = `
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
)
`;
/**
* Display word count for the session row `sessionAlias`: visible persisted word
* occurrences when the session has persisted subtitle lines, otherwise the raw
* token counter `rawTokensExpr`.
*
* The subqueries are correlated on `session_id`, so a query only reads the
* lines of the sessions it actually returns. Do not replace this with a CTE
* aggregated over every subtitle line; that costs a full scan of the
* occurrence table on every stats request.
*/
export function sessionDisplayWordsExpr(
sessionAlias: string,
wordCountAlias: string,
rawTokensExpr = `${sessionAlias}.tokens_seen`,
): string {
return `CASE
WHEN COALESCE(${wordCountAlias}.persistedLineCount, 0) > 0 THEN COALESCE(${wordCountAlias}.filteredWordsSeen, 0)
WHEN EXISTS (
SELECT 1 FROM imm_subtitle_lines display_line
WHERE display_line.session_id = ${sessionAlias}.session_id
) THEN (
SELECT COALESCE(SUM(display_occurrence.occurrence_count), 0)
FROM imm_subtitle_lines display_line
JOIN imm_word_line_occurrences display_occurrence
ON display_occurrence.line_id = display_line.line_id
WHERE display_line.session_id = ${sessionAlias}.session_id
AND display_occurrence.word_id NOT IN (${HIDDEN_WORD_IDS_SQL})
)
ELSE COALESCE(${rawTokensExpr}, 0)
END`;
}
/**
* `SUM(sessionDisplayWordsExpr(s))` over every ended session, as one scalar
* expression that avoids walking each session's lines: all visible
* occurrences, minus those on lines outside ended sessions, plus the raw token
* counter of ended sessions without persisted lines. Relies on occurrences
* cascading with their line (foreign keys are on for every connection), so
* every occurrence belongs to an existing line.
*/
export const ENDED_SESSIONS_DISPLAY_WORDS_SQL = `(
(SELECT COALESCE(SUM(occurrence_count), 0) FROM imm_word_line_occurrences)
- (
SELECT COALESCE(SUM(hidden_occurrence.occurrence_count), 0)
FROM imm_word_line_occurrences hidden_occurrence
WHERE hidden_occurrence.word_id IN (${HIDDEN_WORD_IDS_SQL})
)
- (
SELECT COALESCE(SUM(open_occurrence.occurrence_count), 0)
FROM imm_subtitle_lines open_line
JOIN imm_word_line_occurrences open_occurrence ON open_occurrence.line_id = open_line.line_id
WHERE (
open_line.session_id IS NULL
OR open_line.session_id NOT IN (
SELECT ended.session_id FROM imm_sessions ended WHERE ended.ended_at_ms IS NOT NULL
)
)
AND open_occurrence.word_id NOT IN (${HIDDEN_WORD_IDS_SQL})
)
+ (
SELECT COALESCE(SUM(COALESCE(lineless.tokens_seen, 0)), 0)
FROM imm_sessions lineless
WHERE lineless.ended_at_ms IS NOT NULL
AND NOT EXISTS (
SELECT 1 FROM imm_subtitle_lines lineless_line
WHERE lineless_line.session_id = lineless.session_id
)
)
)`;
export function makePlaceholders(values: number[]): string {
return values.map(() => '?').join(',');
}
@@ -3,16 +3,22 @@ import type { ImmersionSessionRollupRow } from './types';
import {
ACTIVE_SESSION_METRICS_CTE,
currentDbTimestamp,
getLocalDayOfWeek,
getLocalEpochDay,
getLocalHourOfDay,
getLocalMonthKey,
getShiftedLocalDayTimestamp,
makePlaceholders,
sessionDisplayWordsExpr,
toDbTimestamp,
} from './query-shared';
import { getDailyRollups, getMonthlyRollups } from './query-sessions';
import {
addRollupWordCount,
applyRollupWordCounts,
selectDailyRollupRows,
selectMonthlyRollupRows,
sessionRollupDaySql,
sessionRollupMonthSql,
type RollupWordCounts,
} from './query-sessions';
import { areLexicalDailyRollupsReady, getLexicalDailyRollups } from './lexical-rollups';
type TrendRange = '7d' | '30d' | '90d' | '365d' | 'all';
@@ -43,7 +49,6 @@ export interface LibrarySummaryRow {
}
interface TrendSessionMetricRow {
startedAtMs: number;
epochDay: number;
monthKey: number;
dayOfWeek: number;
@@ -78,7 +83,8 @@ export interface TrendsDashboardQueryResult {
cardsPerHour: TrendChartPoint[];
readingSpeed: TrendChartPoint[];
};
animeCumulative: {
/** Sparse per-title values for each active day; the stats UI builds the running totals. */
animeDaily: {
watchTime: TrendPerAnimePoint[];
episodes: TrendPerAnimePoint[];
cards: TrendPerAnimePoint[];
@@ -387,41 +393,6 @@ function buildLookupsPerHundredWords(
return fillAxisPoints(axis, ratioByBucket);
}
function buildCumulativePerAnime(points: TrendPerAnimePoint[]): TrendPerAnimePoint[] {
const byAnime = new Map<string, Map<number, number>>();
const allDays = new Set<number>();
for (const point of points) {
const dayMap = byAnime.get(point.animeTitle) ?? new Map();
dayMap.set(point.epochDay, (dayMap.get(point.epochDay) ?? 0) + point.value);
byAnime.set(point.animeTitle, dayMap);
allDays.add(point.epochDay);
}
const sortedDays = [...allDays].sort((left, right) => left - right);
if (sortedDays.length === 0) {
return [];
}
const minDay = sortedDays[0]!;
const maxDay = sortedDays[sortedDays.length - 1]!;
const result: TrendPerAnimePoint[] = [];
for (const [animeTitle, dayMap] of byAnime) {
const firstDay = Math.min(...dayMap.keys());
let cumulative = 0;
for (let epochDay = minDay; epochDay <= maxDay; epochDay += 1) {
if (epochDay < firstDay) {
continue;
}
cumulative += dayMap.get(epochDay) ?? 0;
result.push({ epochDay, animeTitle, value: cumulative });
}
}
return result;
}
function buildLibrarySummary(
rollups: ImmersionSessionRollupRow[],
sessions: TrendSessionMetricRow[],
@@ -616,44 +587,64 @@ function buildEpisodesSeriesFromRollups(
return fillAxisPoints(axis, counts);
}
function getTrendSessionMetrics(
interface TrendSessionRow extends TrendSessionMetricRow {
ended: number;
inTrendRange: number;
}
/**
* One pass over the sessions the dashboard needs. Rows inside the trend range
* feed the session-metric charts; ended rows also supply the rollup word counts,
* so the per-session word count is computed once per request instead of once
* per rollup query. `minRollupDay` / `minRollupMonth` widen the scan to the
* oldest rollup bucket shown.
*/
function getTrendSessionRows(
db: DatabaseSync,
cutoffMs: string | null,
): TrendSessionMetricRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const whereClause = cutoffMs === null ? '' : 'WHERE s.started_at_ms >= ?';
minRollupDay: number | null,
minRollupMonth: number | null,
): TrendSessionRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const cutoffValue = cutoffMs === null ? null : toDbTimestamp(cutoffMs);
const rangeSql = cutoffValue === null ? '1' : 's.started_at_ms >= ?';
const dayValue = minRollupDay ?? Number.MAX_SAFE_INTEGER;
const monthValue = minRollupMonth ?? Number.MAX_SAFE_INTEGER;
const prepared = db.prepare(`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
s.started_at_ms AS startedAtMs,
s.video_id AS videoId,
v.canonical_title AS canonicalTitle,
a.canonical_title AS animeTitle,
COALESCE(asm.activeWatchedMs, s.active_watched_ms, 0) AS activeWatchedMs,
${wordsExpr} AS tokensSeen,
COALESCE(asm.cardsMined, s.cards_mined, 0) AS cardsMined,
COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0) AS yomitanLookupCount
COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0) AS yomitanLookupCount,
CASE WHEN s.ended_at_ms IS NOT NULL THEN 1 ELSE 0 END AS ended,
CASE WHEN ${rangeSql} THEN 1 ELSE 0 END AS inTrendRange,
${sessionRollupDaySql('s')} AS epochDay,
${sessionRollupMonthSql('s')} AS monthKey,
CAST(strftime('%w', CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') AS INTEGER) AS dayOfWeek,
CAST(strftime('%H', CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') AS INTEGER) AS hourOfDay
FROM imm_sessions s
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
LEFT JOIN imm_videos v ON v.video_id = s.video_id
LEFT JOIN imm_anime a ON a.anime_id = v.anime_id
${whereClause}
WHERE ${rangeSql}
OR (s.ended_at_ms IS NOT NULL AND ${sessionRollupDaySql('s')} >= ?)
OR (s.ended_at_ms IS NOT NULL AND ${sessionRollupMonthSql('s')} >= ?)
ORDER BY s.started_at_ms ASC
`);
const rows = (cutoffValue === null ? prepared.all() : prepared.all(cutoffValue)) as Array<
TrendSessionMetricRow & { startedAtMs: number | string }
>;
return rows.map((row) => ({
...row,
startedAtMs: 0,
epochDay: getLocalEpochDay(db, row.startedAtMs),
monthKey: getLocalMonthKey(db, row.startedAtMs),
dayOfWeek: getLocalDayOfWeek(db, row.startedAtMs),
hourOfDay: getLocalHourOfDay(db, row.startedAtMs),
}));
const params =
cutoffValue === null
? [dayValue, monthValue]
: [cutoffValue, cutoffValue, dayValue, monthValue];
return prepared.all(...params) as unknown as TrendSessionRow[];
}
function minRollupBucket(rows: Array<{ rollupDayOrMonth: number }>): number | null {
return rows.length === 0 ? null : Math.min(...rows.map((row) => row.rollupDayOrMonth));
}
function buildNewWordsPerDay(
@@ -750,8 +741,24 @@ export function getTrendsDashboard(
const monthlyLimit = getTrendMonthlyLimit(db, range);
const cutoffMs = getTrendCutoffMs(db, range);
const useMonthlyBuckets = groupBy === 'month';
const dailyRollups = getDailyRollups(db, dayLimit);
const monthlyRollups = getMonthlyRollups(db, monthlyLimit);
const rawDailyRollups = selectDailyRollupRows(db, dayLimit);
const rawMonthlyRollups = selectMonthlyRollupRows(db, monthlyLimit);
const sessionRows = getTrendSessionRows(
db,
cutoffMs,
minRollupBucket(rawDailyRollups),
minRollupBucket(rawMonthlyRollups),
);
const dailyWordCounts: RollupWordCounts = new Map();
const monthlyWordCounts: RollupWordCounts = new Map();
for (const row of sessionRows) {
if (row.ended !== 1) continue;
addRollupWordCount(dailyWordCounts, row.epochDay, row.videoId, row.tokensSeen);
addRollupWordCount(monthlyWordCounts, row.monthKey, row.videoId, row.tokensSeen);
}
const dailyRollups = applyRollupWordCounts(rawDailyRollups, dailyWordCounts);
const monthlyRollups = applyRollupWordCounts(rawMonthlyRollups, monthlyWordCounts);
const sessions = sessionRows.filter((row) => row.inTrendRange === 1);
// A null axis makes the builders fall back to only the buckets present in the
// data; the contiguous axis zero-fills every calendar bucket in the window.
const bucketAxis = fillEmptyBuckets
@@ -759,7 +766,6 @@ export function getTrendsDashboard(
: null;
const chartRollups = useMonthlyBuckets ? monthlyRollups : dailyRollups;
const sessions = getTrendSessionMetrics(db, cutoffMs);
const titlesByVideoId = getVideoAnimeTitleMap(
db,
dailyRollups.map((rollup) => rollup.videoId),
@@ -820,12 +826,7 @@ export function getTrendsDashboard(
cardsPerHour: efficiency.cardsPerHour,
readingSpeed: efficiency.readingSpeed,
},
animeCumulative: {
watchTime: buildCumulativePerAnime(animePerDay.watchTime),
episodes: buildCumulativePerAnime(animePerDay.episodes),
cards: buildCumulativePerAnime(animePerDay.cards),
words: buildCumulativePerAnime(animePerDay.words),
},
animeDaily: animePerDay,
patterns: {
watchTimeByDayOfWeek: buildWatchTimeByDayOfWeek(sessions),
watchTimeByHour: buildWatchTimeByHour(sessions),
+15 -4
View File
@@ -1849,14 +1849,24 @@ function incrementWordAggregate(
occurrence.frequencyRank ?? null,
);
}
const row = stmts.wordSelectStmt.get(occurrence.headword, occurrence.word, occurrence.reading) as
return syncStoredWordVisibility(stmts, occurrence);
}
/**
* Persists visibility for a word that was just upserted and returns its id. The
* upsert keeps the first-seen POS metadata, so visibility comes from the stored
* row (as the vocabulary summary expects) rather than the incoming token.
*/
function syncStoredWordVisibility(
stmts: TrackerPreparedStatements,
key: Pick<Extract<QueuedWrite, { kind: 'word' }>, 'headword' | 'word' | 'reading'>,
): number {
const row = stmts.wordSelectStmt.get(key.headword, key.word, key.reading) as
| (VocabularyVisibilityRow & { id: number; vocabularyVisible: number })
| null;
if (!row?.id) {
throw new Error(`Failed to resolve imm_words id for ${occurrence.headword}`);
throw new Error(`Failed to resolve imm_words id for ${key.headword}`);
}
// The upsert keeps the first-seen POS metadata, so derive visibility from the
// stored row (as the vocabulary summary does) rather than the incoming token.
const visible = isVocabularyStatsRowVisible(row) ? 1 : 0;
if (row.vocabularyVisible !== visible) {
stmts.wordVisibilityUpdateStmt.run(visible, row.id);
@@ -1939,6 +1949,7 @@ export function executeQueuedWrite(write: QueuedWrite, stmts: TrackerPreparedSta
write.lastSeen,
write.frequencyRank ?? null,
);
syncStoredWordVisibility(stmts, write);
return;
}
if (write.kind === 'kanji') {
+23
View File
@@ -146,6 +146,7 @@ function createRegisterIpcDeps(overrides: Partial<IpcServiceDeps> = {}): IpcServ
saveControllerConfig: async () => {},
reportActiveController: () => {},
getActiveControllerId: () => null,
notifyControllerStatus: () => {},
getSecondarySubMode: () => 'hover',
getCurrentSecondarySub: () => '',
focusMainWindow: () => {},
@@ -257,6 +258,26 @@ test('registerIpcHandlers stores the controller the overlay reports for Alt+C to
assert.equal(await handlers.handle.get(IPC_CHANNELS.request.getActiveControllerId)?.({}), null);
});
test('registerIpcHandlers forwards controller status messages and drops invalid ones', () => {
const { registrar, handlers } = createFakeIpcRegistrar();
const messages: string[] = [];
registerIpcHandlers(
createRegisterIpcDeps({
notifyControllerStatus: (message) => {
messages.push(message);
},
}),
registrar,
);
const notify = handlers.on.get(IPC_CHANNELS.command.notifyControllerStatus);
notify?.({}, 'Controller connected: pad-1');
notify?.({}, 42);
notify?.({}, ' ');
assert.deepEqual(messages, ['Controller connected: pad-1']);
});
test('registerIpcHandlers runs AniList update after manual mark watched succeeds', async () => {
const { registrar, handlers } = createFakeIpcRegistrar();
const calls: string[] = [];
@@ -361,6 +382,7 @@ test('registerIpcHandlers rejects malformed runtime-option payloads', async () =
saveControllerConfig: () => {},
reportActiveController: () => {},
getActiveControllerId: () => null,
notifyControllerStatus: () => {},
getSecondarySubMode: () => 'hover',
getCurrentSecondarySub: () => '',
focusMainWindow: () => {},
@@ -644,6 +666,7 @@ test('registerIpcHandlers ignores malformed fire-and-forget payloads', () => {
saveControllerConfig: () => {},
reportActiveController: () => {},
getActiveControllerId: () => null,
notifyControllerStatus: () => {},
getSecondarySubMode: () => 'hover',
getCurrentSecondarySub: () => '',
focusMainWindow: () => {},
+8
View File
@@ -92,6 +92,7 @@ export interface IpcServiceDeps {
/** The overlay reports the controller it is reading (null when none). */
reportActiveController: (gamepadId: string | null) => void;
getActiveControllerId: () => string | null;
notifyControllerStatus: (message: string) => void;
getSecondarySubMode: () => unknown;
getCurrentSecondarySub: () => string;
focusMainWindow: () => void;
@@ -362,6 +363,7 @@ export interface IpcDepsRuntimeOptions {
/** The overlay reports the controller it is reading (null when none). */
reportActiveController: (gamepadId: string | null) => void;
getActiveControllerId: () => string | null;
notifyControllerStatus: (message: string) => void;
getSecondarySubMode: () => unknown;
getMpvClient: () => MpvClientLike | null;
focusMainWindow: () => void;
@@ -440,6 +442,7 @@ export function createIpcDepsRuntime(options: IpcDepsRuntimeOptions): IpcService
saveControllerConfig: options.saveControllerConfig,
reportActiveController: options.reportActiveController,
getActiveControllerId: options.getActiveControllerId,
notifyControllerStatus: options.notifyControllerStatus,
getSecondarySubMode: options.getSecondarySubMode,
getCurrentSecondarySub: () => options.getMpvClient()?.currentSecondarySubText || '',
focusMainWindow: () => {
@@ -777,6 +780,11 @@ export function registerIpcHandlers(deps: IpcServiceDeps, ipc: IpcMainRegistrar
return deps.getActiveControllerId();
});
ipc.on(IPC_CHANNELS.command.notifyControllerStatus, (_event: unknown, message: unknown) => {
if (typeof message !== 'string' || message.trim().length === 0) return;
deps.notifyControllerStatus(message);
});
ipc.handle(IPC_CHANNELS.request.getSecondarySubMode, () => {
return deps.getSecondarySubMode();
});
+106
View File
@@ -3,6 +3,7 @@ import assert from 'node:assert/strict';
import type { MpvSubtitleRenderMetrics } from '../../types';
import {
dispatchMpvProtocolMessage,
MPV_REQUEST_ID_RELOADED_PATH,
MPV_REQUEST_ID_TRACK_LIST_SECONDARY,
MpvProtocolHandleMessageDeps,
splitMpvMessagesFromBuffer,
@@ -99,7 +100,10 @@ function createDeps(overrides: Partial<MpvProtocolHandleMessageDeps> = {}): {
emitPauseChange: () => {},
emitFullscreenChange: (payload) => state.events.push(payload),
autoLoadSecondarySubTrack: () => {},
getCurrentVideoPath: () => '',
setCurrentVideoPath: () => {},
getAwaitingPathChangeAfterEndFile: () => false,
setAwaitingPathChangeAfterEndFile: () => {},
emitSecondarySubtitleVisibility: (payload) => state.events.push(payload),
setCurrentAudioStreamIndex: () => {},
sendCommand: (payload) => {
@@ -502,3 +506,105 @@ test('dispatchMpvProtocolMessage emits end-file with reason, entry id, and error
{ reason: 'unknown', playlistEntryId: null, error: null },
]);
});
const CLIP = '/media/clip.mkv';
// Plays a reload's message sequence against real path state. The timeline holds each path change
// plus 'file-loaded' with the path listeners can read at that moment.
async function playFileTransition(messages: Array<Record<string, unknown>>) {
const timeline: string[] = [];
let currentPath = CLIP;
let awaitingPathChange = false;
const { deps, state } = createDeps({
getCurrentVideoPath: () => currentPath,
setCurrentVideoPath: (value) => {
currentPath = value;
},
getAwaitingPathChangeAfterEndFile: () => awaitingPathChange,
setAwaitingPathChangeAfterEndFile: (value) => {
awaitingPathChange = value;
},
emitMediaPathChange: ({ path }) => timeline.push(`path:${path}`),
emitFileLoaded: () => timeline.push(`file-loaded:${currentPath}`),
});
for (const message of messages) {
await dispatchMpvProtocolMessage(message, deps);
}
const pathReads = state.commands.filter(
(payload) => (payload as { request_id?: number }).request_id === MPV_REQUEST_ID_RELOADED_PATH,
);
return { timeline, pathReads, currentPath };
}
const endFile = { event: 'end-file', reason: 'stop' };
const fileLoaded = { event: 'file-loaded' };
const pathEvent = (data: string | null) => ({ event: 'property-change', name: 'path', data });
test('a reload whose path clear arrives needs no recovery', async () => {
const { timeline, pathReads } = await playFileTransition([
endFile,
pathEvent(null),
pathEvent(CLIP),
fileLoaded,
]);
assert.deepEqual(timeline, ['path:', `path:${CLIP}`, `file-loaded:${CLIP}`]);
assert.equal(pathReads.length, 0);
});
test('a dropped clear is replayed and file-loaded waits for the restored path', async () => {
// With commands queued or a stalled reader, every queued event comes first, then the latest path.
const { timeline, pathReads, currentPath } = await playFileTransition([
endFile,
fileLoaded,
pathEvent(CLIP),
{ request_id: MPV_REQUEST_ID_RELOADED_PATH, data: CLIP },
]);
assert.deepEqual(timeline, ['path:', `path:${CLIP}`, `file-loaded:${CLIP}`]);
assert.equal(pathReads.length, 1);
assert.equal(currentPath, CLIP);
});
test('an unchanged path event before file-loaded does not hide a dropped clear', async () => {
const { timeline } = await playFileTransition([
endFile,
pathEvent(CLIP),
fileLoaded,
{ request_id: MPV_REQUEST_ID_RELOADED_PATH, data: CLIP },
]);
assert.deepEqual(timeline, [`path:${CLIP}`, 'path:', `path:${CLIP}`, `file-loaded:${CLIP}`]);
});
test('a reload with no path event at all restores the path from the read back', async () => {
const { timeline, currentPath } = await playFileTransition([
endFile,
fileLoaded,
{ request_id: MPV_REQUEST_ID_RELOADED_PATH, data: CLIP },
]);
assert.deepEqual(timeline, ['path:', `path:${CLIP}`, `file-loaded:${CLIP}`]);
assert.equal(currentPath, CLIP);
});
test('a failed path read back still delivers the deferred file-loaded', async () => {
const { timeline } = await playFileTransition([
endFile,
fileLoaded,
{ request_id: MPV_REQUEST_ID_RELOADED_PATH, error: 'property unavailable' },
]);
assert.deepEqual(timeline, ['path:', 'file-loaded:']);
});
test('switching files with a dropped clear stays a plain path change', async () => {
const { timeline, pathReads } = await playFileTransition([
endFile,
pathEvent('/media/next.mkv'),
fileLoaded,
]);
assert.deepEqual(timeline, ['path:/media/next.mkv', 'file-loaded:/media/next.mkv']);
assert.equal(pathReads.length, 0);
});
+51 -11
View File
@@ -55,6 +55,7 @@ export const MPV_REQUEST_ID_SUB_SHADOW_OFFSET = 120;
export const MPV_REQUEST_ID_SUB_ASS_OVERRIDE = 121;
export const MPV_REQUEST_ID_SUB_USE_MARGINS = 122;
export const MPV_REQUEST_ID_PAUSE = 123;
export const MPV_REQUEST_ID_RELOADED_PATH = 124;
export const MPV_REQUEST_ID_TRACK_LIST_SECONDARY = 200;
export const MPV_REQUEST_ID_TRACK_LIST_AUDIO = 201;
export const MPV_REQUEST_ID_MEDIA_TITLE = 202;
@@ -109,7 +110,11 @@ export interface MpvProtocolHandleMessageDeps {
getPauseAtTime: () => number | null;
setPauseAtTime: (value: number | null) => void;
autoLoadSecondarySubTrack: (path: string) => void;
getCurrentVideoPath: () => string;
setCurrentVideoPath: (value: string) => void;
/** True from `end-file` until mpv reports a different path or the next `file-loaded`. */
getAwaitingPathChangeAfterEndFile: () => boolean;
setAwaitingPathChangeAfterEndFile: (value: boolean) => void;
emitSecondarySubtitleVisibility: (payload: { visible: boolean }) => void;
setPreviousSecondarySubVisibility: (visible: boolean) => void;
setCurrentAudioStreamIndex: (
@@ -380,18 +385,10 @@ export async function dispatchMpvProtocolMessage(
applyMediaTitle(deps, msg.data);
} else if (msg.name === 'path') {
const path = (msg.data as string) || '';
deps.setCurrentVideoPath(path);
// A forced title set before loadfile arrives ahead of the path change that clears the
// cached title and never fires again, so read it back once the new path is known.
if (path) {
deps.sendCommand({
command: ['get_property', 'media-title'],
request_id: MPV_REQUEST_ID_MEDIA_TITLE,
});
if (path !== deps.getCurrentVideoPath()) {
deps.setAwaitingPathChangeAfterEndFile(false);
}
deps.emitMediaPathChange({ path });
deps.autoLoadSecondarySubTrack(path);
deps.syncCurrentAudioStreamIndex();
applyMediaPath(deps, path);
} else if (msg.name === 'sub-pos') {
deps.emitSubtitleMetricsChange({ subPos: msg.data as number });
} else if (msg.name === 'sub-font-size') {
@@ -464,9 +461,26 @@ export async function dispatchMpvProtocolMessage(
deps.emitClientMessage?.({ args });
}
} else if (msg.event === 'file-loaded') {
if (deps.getAwaitingPathChangeAfterEndFile()) {
// mpv only sends the latest value of an observed property, once its IPC thread gets to it, so
// a same-file reload can skip the path clear. The reload then looks like nothing happened and
// per-file state (autoplay gate, subtitles) never resets. Replay the clear so consumers see
// the same sequence as any other load. Read the path back because the unchanged value may
// never be reported, and hold file-loaded until the reply so listeners can read the path.
deps.setAwaitingPathChangeAfterEndFile(false);
applyMediaPath(deps, '');
deps.sendCommand({
command: ['get_property', 'path'],
request_id: MPV_REQUEST_ID_RELOADED_PATH,
});
return;
}
deps.emitFileLoaded?.();
} else if (msg.event === 'end-file') {
deps.emitEndFile?.(parseEndFileEvent(msg));
if (deps.getCurrentVideoPath()) {
deps.setAwaitingPathChangeAfterEndFile(true);
}
} else if (msg.event === 'shutdown') {
deps.restorePreviousSecondarySubVisibility();
if (deps.shouldQuitOnMpvShutdown()) {
@@ -477,6 +491,17 @@ export async function dispatchMpvProtocolMessage(
return;
}
// Before the no-data check: an error reply still owes the file-loaded held for this read.
if (msg.request_id === MPV_REQUEST_ID_RELOADED_PATH) {
// The path property event may have restored the path before this reply arrived.
const path = (msg.data as string) || '';
if (path && !deps.getCurrentVideoPath()) {
applyMediaPath(deps, path);
}
deps.emitFileLoaded?.();
return;
}
if (msg.data === undefined) {
return;
}
@@ -618,6 +643,21 @@ function applyMediaTitle(
deps.emitMediaTitleChange({ title });
}
function applyMediaPath(deps: MpvProtocolHandleMessageDeps, path: string): void {
deps.setCurrentVideoPath(path);
// A forced title set before loadfile arrives ahead of the path change that clears the
// cached title and never fires again, so read it back once the new path is known.
if (path) {
deps.sendCommand({
command: ['get_property', 'media-title'],
request_id: MPV_REQUEST_ID_MEDIA_TITLE,
});
}
deps.emitMediaPathChange({ path });
deps.autoLoadSecondarySubTrack(path);
deps.syncCurrentAudioStreamIndex();
}
export function parseVisibilityProperty(value: unknown): boolean | null {
if (typeof value === 'boolean') return value;
if (typeof value !== 'string') return null;
+6
View File
@@ -142,6 +142,7 @@ export class MpvIpcClient implements MpvClient {
private hasConnectedOnce = false;
private socketErrorWarnedForDisconnect = false;
public currentVideoPath = '';
private awaitingPathChangeAfterEndFile = false;
public currentMediaTitle: string | null = null;
public currentTimePos = 0;
public currentSubStart = 0;
@@ -489,10 +490,15 @@ export class MpvIpcClient implements MpvClient {
autoLoadSecondarySubTrack: (path: string) => {
this.autoLoadSecondarySubTrack(path);
},
getCurrentVideoPath: () => this.currentVideoPath,
setCurrentVideoPath: (value: string) => {
this.currentVideoPath = value;
this.currentMediaTitle = null;
},
getAwaitingPathChangeAfterEndFile: () => this.awaitingPathChangeAfterEndFile,
setAwaitingPathChangeAfterEndFile: (value: boolean) => {
this.awaitingPathChangeAfterEndFile = value;
},
emitSecondarySubtitleVisibility: (payload) => {
this.emit('secondary-subtitle-visibility', payload);
},
+44
View File
@@ -35,6 +35,11 @@ import { createDiscordRpcClient } from './main/runtime/discord-rpc-client.js';
import { startAppControlServer } from './main/runtime/app-control-server';
import { getAppControlSocketPath } from './shared/app-control';
import { focusMacOSOverlayWindow } from './main/runtime/macos-overlay-window-focus';
import {
handleBackgroundControllerPress,
resolveMacOSControllerWatchCommand,
startMacOSControllerWatch,
} from './main/runtime/macos-controller-watch';
import { restoreMacOSMpvFocusAfterModalClose } from './main/runtime/macos-modal-focus-handoff';
import {
getDefaultPasswordStore,
@@ -2097,6 +2102,11 @@ const { registerIpcRuntimeHandlers } = composeIpcRuntimeHandlers({
activeControllerIdFromOverlay = gamepadId;
},
getActiveControllerId: () => activeControllerIdFromOverlay,
notifyControllerStatus: (message) => {
overlayNotificationsRuntime.showConfiguredStatusNotification(message, {
id: 'controller-status',
});
},
saveControllerConfig: (update) => {
const currentRawConfig = configService.getRawConfig();
configService.patchRawConfig({
@@ -2417,3 +2427,37 @@ registerSubtitleGenerationIpc({
openModal: () => openSubtitleGenerationModal(overlayModals.createOverlayHostedModalOpenDeps()),
});
app.on('before-quit', () => subtitleGenerationRuntime.cancel());
if (process.platform === 'darwin') {
void app.whenReady().then(() => {
const command = resolveMacOSControllerWatchCommand({
resourcesPath: process.resourcesPath,
mainDirname: __dirname,
});
if (!command) {
logger.warn('macOS controller watch helper not found; controller needs overlay focus');
return;
}
const controllerWatch = startMacOSControllerWatch({
command,
logger,
onPress: (press) =>
handleBackgroundControllerPress(press, {
isControllerEnabled: () => configService.getConfig().controller.enabled,
isMpvFocused: () => appState.windowTracker?.isTargetWindowFocused() ?? false,
getModalWindow: () => overlayManager.getModalWindow(),
getMainWindow: () => overlayManager.getMainWindow(),
focusWindow: (window) =>
focusMacOSOverlayWindow({
platform: process.platform,
getOverlayWindow: () => window,
stealAppFocus: () => app.focus({ steal: true }),
warn: (message, details) => logger.warn(message, details),
}),
replayPress: (window, index) =>
window.webContents.send(IPC_CHANNELS.event.controllerBackgroundPress, index),
}),
});
app.on('will-quit', () => controllerWatch.stop());
});
}
+2
View File
@@ -80,6 +80,7 @@ export interface MainIpcRuntimeServiceDepsParams {
saveControllerConfig: IpcDepsRuntimeOptions['saveControllerConfig'];
reportActiveController: IpcDepsRuntimeOptions['reportActiveController'];
getActiveControllerId: IpcDepsRuntimeOptions['getActiveControllerId'];
notifyControllerStatus: IpcDepsRuntimeOptions['notifyControllerStatus'];
getSecondarySubMode: IpcDepsRuntimeOptions['getSecondarySubMode'];
getMpvClient: IpcDepsRuntimeOptions['getMpvClient'];
runSubsyncManual: IpcDepsRuntimeOptions['runSubsyncManual'];
@@ -260,6 +261,7 @@ export function createMainIpcRuntimeServiceDeps(
saveControllerConfig: params.saveControllerConfig,
reportActiveController: params.reportActiveController,
getActiveControllerId: params.getActiveControllerId,
notifyControllerStatus: params.notifyControllerStatus,
focusMainWindow: params.focusMainWindow ?? (() => {}),
activatePlaybackWindowForOverlayInteraction: params.activatePlaybackWindowForOverlayInteraction,
getSecondarySubMode: params.getSecondarySubMode,
@@ -0,0 +1,96 @@
import assert from 'node:assert/strict';
import path from 'node:path';
import test from 'node:test';
import {
handleBackgroundControllerPress,
parseControllerWatchLine,
resolveMacOSControllerWatchCommand,
} from './macos-controller-watch';
test('parseControllerWatchLine reads press lines only', () => {
assert.deepEqual(parseControllerWatchLine('press 7 2\n'), { buttonIndex: 7, controllerCount: 2 });
assert.equal(parseControllerWatchLine('press 7'), null);
assert.equal(parseControllerWatchLine('release 7 1'), null);
});
function fakeFs(files: Record<string, number>) {
return {
existsSync: (filePath: string) => filePath in files,
statSync: (filePath: string) => ({ mtimeMs: files[filePath] }),
} as unknown as Parameters<typeof resolveMacOSControllerWatchCommand>[1];
}
test('resolveMacOSControllerWatchCommand prefers the packaged binary', () => {
const packaged = path.join('/res', 'scripts', 'macos-controller-watch');
assert.deepEqual(
resolveMacOSControllerWatchCommand(
{ resourcesPath: '/res', mainDirname: '/repo/dist' },
fakeFs({ [packaged]: 1 }),
),
{ command: packaged, args: [] },
);
});
test('resolveMacOSControllerWatchCommand runs the source when the build is stale', () => {
const built = path.join('/repo/dist', 'scripts', 'macos-controller-watch');
const source = path.join('/repo', 'scripts', 'macos-controller-watch.swift');
const options = { resourcesPath: undefined, mainDirname: '/repo/dist' };
assert.deepEqual(
resolveMacOSControllerWatchCommand(options, fakeFs({ [built]: 2, [source]: 1 })),
{
command: built,
args: [],
},
);
assert.deepEqual(
resolveMacOSControllerWatchCommand(options, fakeFs({ [built]: 1, [source]: 2 })),
{
command: 'swift',
args: [source],
},
);
assert.equal(resolveMacOSControllerWatchCommand(options, fakeFs({})), null);
});
function window(name: string, visible = true) {
return { name, isDestroyed: () => false, isVisible: () => visible };
}
function run(overrides: {
controllerCount?: number;
enabled?: boolean;
mpvFocused?: boolean;
modal?: ReturnType<typeof window> | null;
main?: ReturnType<typeof window> | null;
}): string[] {
const calls: string[] = [];
const press = { buttonIndex: 4, controllerCount: overrides.controllerCount ?? 1 };
handleBackgroundControllerPress(press, {
isControllerEnabled: () => overrides.enabled ?? true,
isMpvFocused: () => overrides.mpvFocused ?? true,
getModalWindow: () => overrides.modal ?? null,
getMainWindow: () => (overrides.main === undefined ? window('main') : overrides.main),
focusWindow: (target) => calls.push(`focus:${target.name}`),
replayPress: (target, index) => calls.push(`replay:${target.name}:${index}`),
});
return calls;
}
test('a press while mpv is in front focuses the overlay and replays it', () => {
assert.deepEqual(run({}), ['focus:main', 'replay:main:4']);
});
test('with several pads connected a press focuses the overlay without replaying', () => {
assert.deepEqual(run({ controllerCount: 2 }), ['focus:main']);
});
test('an open modal window takes the press instead of the main overlay', () => {
assert.deepEqual(run({ modal: window('modal') }), ['focus:modal', 'replay:modal:4']);
});
test('presses are ignored unless mpv is in front, the controller is on and an overlay shows', () => {
assert.deepEqual(run({ mpvFocused: false }), []);
assert.deepEqual(run({ enabled: false }), []);
assert.deepEqual(run({ main: window('main', false) }), []);
});
+120
View File
@@ -0,0 +1,120 @@
import { spawn as spawnProcess } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as readline from 'node:readline';
import { isCompiledMacOSHelperCurrent } from '../../window-trackers/macos-tracker';
// macOS only delivers GameController input to the frontmost app, so the overlay's Gamepad API
// sees nothing while mpv is in front. `scripts/macos-controller-watch.swift` opts in to
// background events and reports each button press; SubMiner then focuses the overlay (after
// which Chromium reads the pad itself) and replays that first press there.
const HELPER_NAME = 'macos-controller-watch';
export type MacOSControllerWatchCommand = { command: string; args: string[] };
/**
* Finds the helper: the packaged binary in Resources, a compiled build in dist/scripts that is
* newer than its source, or the Swift source run through `swift` in a dev checkout.
*/
export function resolveMacOSControllerWatchCommand(
options: {
resourcesPath: string | undefined;
/** Directory holding the built main process (dist/ in a checkout). */
mainDirname: string;
},
helperFs: Pick<typeof fs, 'existsSync' | 'statSync'> = fs,
): MacOSControllerWatchCommand | null {
const sourcePath = path.join(options.mainDirname, '..', 'scripts', `${HELPER_NAME}.swift`);
if (options.resourcesPath) {
const packagedPath = path.join(options.resourcesPath, 'scripts', HELPER_NAME);
if (helperFs.existsSync(packagedPath)) return { command: packagedPath, args: [] };
}
const builtPath = path.join(options.mainDirname, 'scripts', HELPER_NAME);
if (isCompiledMacOSHelperCurrent(builtPath, sourcePath, helperFs)) {
return { command: builtPath, args: [] };
}
if (helperFs.existsSync(sourcePath)) return { command: 'swift', args: [sourcePath] };
return null;
}
export type ControllerWatchPress = { buttonIndex: number; controllerCount: number };
/** Parses a helper line ("press <standard-mapping button index> <connected pads>"). */
export function parseControllerWatchLine(line: string): ControllerWatchPress | null {
const match = /^press (\d+) (\d+)$/.exec(line.trim());
return match ? { buttonIndex: Number(match[1]), controllerCount: Number(match[2]) } : null;
}
type ControllerWatchLogger = {
info: (message: string, ...args: unknown[]) => void;
warn: (message: string, ...args: unknown[]) => void;
};
/** Runs the helper until `stop` is called; the helper also exits when SubMiner does. */
export function startMacOSControllerWatch(options: {
command: MacOSControllerWatchCommand;
onPress: (press: ControllerWatchPress) => void;
logger: ControllerWatchLogger;
spawn?: typeof spawnProcess;
}): { stop: () => void } {
const spawn = options.spawn ?? spawnProcess;
const child = spawn(options.command.command, options.command.args, {
stdio: ['pipe', 'pipe', 'pipe'],
});
let stopped = false;
readline.createInterface({ input: child.stdout }).on('line', (line) => {
const press = parseControllerWatchLine(line);
if (press) options.onPress(press);
});
readline.createInterface({ input: child.stderr }).on('line', (line) => {
options.logger.warn(`[${HELPER_NAME}] ${line}`);
});
child.on('error', (error) => {
options.logger.warn(`Failed to start ${HELPER_NAME}`, error);
});
child.on('exit', (code, signal) => {
if (!stopped) options.logger.warn(`${HELPER_NAME} exited`, { code, signal });
});
options.logger.info(`Started ${HELPER_NAME}: ${options.command.command}`);
return {
stop: () => {
stopped = true;
child.stdin?.end();
child.kill();
},
};
}
type FocusableWindow = {
isDestroyed: () => boolean;
isVisible: () => boolean;
};
/**
* A press the overlay missed because mpv was in front: focus the overlay window that owns the
* controller (a visible modal, else the main overlay) and replay the press in it. Presses made
* while another app is in front are left alone, so a pad used elsewhere is not hijacked. With
* several pads connected the press only focuses: it might not be from the pad the overlay reads.
*/
export function handleBackgroundControllerPress<TWindow extends FocusableWindow>(
press: ControllerWatchPress,
deps: {
isControllerEnabled: () => boolean;
isMpvFocused: () => boolean;
getModalWindow: () => TWindow | null;
getMainWindow: () => TWindow | null;
focusWindow: (window: TWindow) => void;
replayPress: (window: TWindow, buttonIndex: number) => void;
},
): void {
if (!deps.isControllerEnabled() || !deps.isMpvFocused()) return;
const target = [deps.getModalWindow(), deps.getMainWindow()].find(
(window): window is TWindow => window !== null && !window.isDestroyed() && window.isVisible(),
);
if (!target) return;
deps.focusWindow(target);
if (press.controllerCount === 1) deps.replayPress(target, press.buttonIndex);
}
+8
View File
@@ -197,6 +197,10 @@ const onPlaybackPauseChangedEvent = createQueuedIpcListenerWithPayload<boolean>(
const onControllerConfigChangedEvent = createQueuedIpcListener(
IPC_CHANNELS.event.controllerConfigChanged,
);
const onControllerBackgroundPressEvent = createQueuedIpcListenerWithPayload<number>(
IPC_CHANNELS.event.controllerBackgroundPress,
(payload) => (typeof payload === 'number' && Number.isInteger(payload) ? payload : -1),
);
const onKeyboardModeToggleRequestedEvent = createQueuedIpcListener(
IPC_CHANNELS.event.keyboardModeToggleRequested,
);
@@ -356,6 +360,9 @@ const electronAPI: ElectronAPI = {
reportActiveController: (gamepadId: string | null): void => {
ipcRenderer.send(IPC_CHANNELS.command.reportActiveController, gamepadId);
},
notifyControllerStatus: (message: string): void => {
ipcRenderer.send(IPC_CHANNELS.command.notifyControllerStatus, message);
},
getActiveControllerId: (): Promise<string | null> =>
ipcRenderer.invoke(IPC_CHANNELS.request.getActiveControllerId),
@@ -509,6 +516,7 @@ const electronAPI: ElectronAPI = {
onKeyboardModeToggleRequested: onKeyboardModeToggleRequestedEvent,
onControllerSuspendedChanged: onControllerSuspendedChangedEvent,
onControllerConfigChanged: onControllerConfigChangedEvent,
onControllerBackgroundPress: onControllerBackgroundPressEvent,
onPlaybackPauseChanged: onPlaybackPauseChangedEvent,
onLookupWindowToggleRequested: onLookupWindowToggleRequestedEvent,
getPlaylistBrowserSnapshot: (): Promise<PlaylistBrowserSnapshot> =>
@@ -3,99 +3,40 @@ import test from 'node:test';
import { createControllerStatusIndicator } from './controller-status-indicator.js';
function createClassList(initialTokens: string[] = []) {
const tokens = new Set(initialTokens);
return {
add: (...entries: string[]) => {
for (const entry of entries) tokens.add(entry);
},
remove: (...entries: string[]) => {
for (const entry of entries) tokens.delete(entry);
},
contains: (entry: string) => tokens.has(entry),
};
}
const pad = (id: string, index = 0) => ({ id, index, mapping: 'standard', connected: true });
test('controller status indicator shows once when a controller is first detected and auto-hides', () => {
let nextTimerId = 1;
const scheduled = new Map<number, () => void>();
const classList = createClassList(['hidden']);
const toast = {
textContent: '',
classList,
};
test('controller status indicator notifies once when a controller is first detected', () => {
const messages: string[] = [];
const indicator = createControllerStatusIndicator((message) => messages.push(message));
const indicator = createControllerStatusIndicator({ controllerStatusToast: toast } as never, {
durationMs: 1500,
setTimeout: (callback: () => void) => {
const id = nextTimerId++;
scheduled.set(id, callback);
return id as never;
},
clearTimeout: (id) => {
scheduled.delete(id as never as number);
},
});
indicator.update({ connectedGamepads: [], activeGamepadId: null });
assert.deepEqual(messages, []);
indicator.update({
connectedGamepads: [],
activeGamepadId: null,
});
assert.equal(classList.contains('hidden'), true);
assert.equal(toast.textContent, '');
indicator.update({
connectedGamepads: [{ id: 'pad-1', index: 0, mapping: 'standard', connected: true }],
activeGamepadId: 'pad-1',
});
assert.equal(classList.contains('hidden'), false);
assert.match(toast.textContent, /controller detected/i);
assert.match(toast.textContent, /pad-1/i);
assert.equal(scheduled.size, 1);
indicator.update({
connectedGamepads: [{ id: 'pad-1', index: 0, mapping: 'standard', connected: true }],
activeGamepadId: 'pad-1',
});
assert.equal(scheduled.size, 1);
const [hide] = scheduled.values();
hide?.();
assert.equal(classList.contains('hidden'), true);
assert.equal(toast.textContent, '');
indicator.update({ connectedGamepads: [pad('pad-1')], activeGamepadId: 'pad-1' });
indicator.update({ connectedGamepads: [pad('pad-1')], activeGamepadId: 'pad-1' });
assert.deepEqual(messages, ['Controller connected: pad-1']);
});
test('controller status indicator announces newly detected controllers after startup', () => {
const toast = {
textContent: '',
classList: createClassList(['hidden']),
};
const indicator = createControllerStatusIndicator({ controllerStatusToast: toast } as never, {
setTimeout: () => 1 as never,
clearTimeout: () => {},
});
const messages: string[] = [];
const indicator = createControllerStatusIndicator((message) => messages.push(message));
indicator.update({ connectedGamepads: [pad('pad-1')], activeGamepadId: 'pad-1' });
indicator.update({
connectedGamepads: [{ id: 'pad-1', index: 0, mapping: 'standard', connected: true }],
connectedGamepads: [pad('pad-1'), pad('pad-2', 1)],
activeGamepadId: 'pad-1',
});
toast.classList.add('hidden');
toast.textContent = '';
assert.deepEqual(messages, ['Controller connected: pad-1', 'Controller connected: pad-2']);
});
indicator.update({
connectedGamepads: [
{ id: 'pad-1', index: 0, mapping: 'standard', connected: true },
{ id: 'pad-2', index: 1, mapping: 'standard', connected: true },
],
activeGamepadId: 'pad-1',
});
test('controller status indicator announces a controller again after it reconnects', () => {
const messages: string[] = [];
const indicator = createControllerStatusIndicator((message) => messages.push(message));
assert.equal(toast.classList.contains('hidden'), false);
assert.match(toast.textContent, /pad-2/i);
indicator.update({ connectedGamepads: [pad('pad-1')], activeGamepadId: 'pad-1' });
indicator.update({ connectedGamepads: [], activeGamepadId: null });
indicator.update({ connectedGamepads: [pad('pad-1')], activeGamepadId: 'pad-1' });
assert.equal(messages.length, 2);
});
+5 -45
View File
@@ -5,67 +5,27 @@ type ControllerSnapshot = {
activeGamepadId: string | null;
};
type ControllerStatusIndicatorOptions = {
durationMs?: number;
setTimeout?: (callback: () => void, delay: number) => ReturnType<typeof setTimeout>;
clearTimeout?: (timer: ReturnType<typeof setTimeout> | number) => void;
};
function getDeviceLabel(device: ControllerDeviceInfo | undefined): string {
if (!device) return 'Controller';
return device.id || `Gamepad ${device.index}`;
}
export function createControllerStatusIndicator(
dom: {
controllerStatusToast: {
textContent: string;
classList: { add: (...entries: string[]) => void; remove: (...entries: string[]) => void };
};
},
options: ControllerStatusIndicatorOptions = {},
) {
const durationMs = options.durationMs ?? 2200;
const scheduleTimeout = options.setTimeout ?? globalThis.setTimeout;
const cancelTimeout =
options.clearTimeout ??
((timer: ReturnType<typeof setTimeout> | number) =>
globalThis.clearTimeout(timer as ReturnType<typeof setTimeout>));
let hideTimeout: ReturnType<typeof setTimeout> | number | null = null;
/** Watches controller snapshots and calls `notify` when a new device appears. */
export function createControllerStatusIndicator(notify: (message: string) => void) {
let previousConnectedIds = new Set<string>();
function show(message: string): void {
if (hideTimeout !== null) {
cancelTimeout(hideTimeout);
hideTimeout = null;
}
dom.controllerStatusToast.textContent = message;
dom.controllerStatusToast.classList.remove('hidden');
hideTimeout = scheduleTimeout(() => {
dom.controllerStatusToast.classList.add('hidden');
dom.controllerStatusToast.textContent = '';
hideTimeout = null;
}, durationMs);
}
function update(snapshot: ControllerSnapshot): void {
const newDevices = snapshot.connectedGamepads.filter(
(device) => !previousConnectedIds.has(device.id),
);
if (newDevices.length > 0) {
const activeDevice = snapshot.connectedGamepads.find(
(device) => device.id === snapshot.activeGamepadId,
);
const announcedDevice =
newDevices.find((device) => device.id === snapshot.activeGamepadId) ??
newDevices[0] ??
activeDevice;
show(`Controller detected: ${getDeviceLabel(announcedDevice)}`);
newDevices.find((device) => device.id === snapshot.activeGamepadId) ?? newDevices[0];
notify(`Controller connected: ${getDeviceLabel(announcedDevice)}`);
}
previousConnectedIds = new Set(snapshot.connectedGamepads.map((device) => device.id));
}
return { show, update };
return { update };
}
@@ -0,0 +1,107 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { createControllerPressReplay } from './controller-press-replay';
import type { GamepadLike } from './gamepad-controller';
function pad(pressed: number[] = [], mapping = 'standard'): GamepadLike {
return {
id: 'pad',
index: 0,
connected: true,
mapping,
axes: [0, 0, 0, 0],
buttons: Array.from({ length: 17 }, (_, index) => ({
value: pressed.includes(index) ? 1 : 0,
pressed: pressed.includes(index),
})),
};
}
const pressedIndices = (gamepad: GamepadLike | null | undefined): number[] =>
gamepad?.buttons.flatMap((button, index) => (button.pressed ? [index] : [])) ?? [];
test('a queued press reads as pressed for exactly one poll', () => {
const replay = createControllerPressReplay();
replay.queue(0, 1000);
assert.deepEqual(pressedIndices(replay.apply([pad()], 1010)[0]), [0]);
assert.deepEqual(pressedIndices(replay.apply([pad()], 1026)[0]), []);
});
test('a press the Gamepad API already reported is not replayed', () => {
const replay = createControllerPressReplay();
replay.apply([pad([1])], 990);
replay.queue(1, 1000);
assert.deepEqual(pressedIndices(replay.apply([pad()], 1010)[0]), []);
});
test('a press seen after the replay was queued is not replayed again', () => {
const replay = createControllerPressReplay();
replay.queue(2, 1000);
assert.deepEqual(pressedIndices(replay.apply([pad([2])], 1010)[0]), [2]);
assert.deepEqual(pressedIndices(replay.apply([pad()], 1026)[0]), []);
});
test('an older press of the same button does not block the replay', () => {
const replay = createControllerPressReplay();
replay.apply([pad([3])], 0);
replay.queue(3, 5000);
assert.deepEqual(pressedIndices(replay.apply([pad()], 5010)[0]), [3]);
});
test('a replayed native pad keeps its fields', () => {
// Like the browser's Gamepad, these fields are prototype getters that object spread skips.
class NativePad implements GamepadLike {
get id() {
return 'pad';
}
get index() {
return 2;
}
get connected() {
return true;
}
get mapping() {
return 'standard';
}
get timestamp() {
return 900;
}
get axes() {
return [0.5, 0];
}
get buttons() {
return pad().buttons;
}
}
const replay = createControllerPressReplay();
replay.queue(0, 1000);
const replayed = replay.apply([new NativePad()], 1010)[0];
assert.deepEqual(pressedIndices(replayed), [0]);
assert.equal(replayed?.id, 'pad');
assert.equal(replayed?.index, 2);
assert.equal(replayed?.connected, true);
assert.equal(replayed?.mapping, 'standard');
assert.equal(replayed?.timestamp, 900);
assert.deepEqual(replayed?.axes, [0.5, 0]);
});
test('nothing is replayed while several pads are connected', () => {
const replay = createControllerPressReplay();
replay.queue(0, 1000);
const pads = replay.apply([pad(), { ...pad(), index: 1 }], 1010);
assert.deepEqual(pads.map(pressedIndices), [[], []]);
assert.deepEqual(pressedIndices(replay.apply([pad()], 1026)[0]), []);
});
test('pads without the standard mapping are left alone', () => {
const replay = createControllerPressReplay();
replay.queue(0, 1000);
assert.deepEqual(pressedIndices(replay.apply([pad([], '')], 1010)[0]), []);
});
@@ -0,0 +1,75 @@
import type { GamepadLike } from './gamepad-controller.js';
/**
* How far back a press the Gamepad API reported itself still counts as the same press the
* main process is replaying. Covers the helper -> main -> renderer IPC hop.
*/
const SAME_PRESS_WINDOW_MS = 500;
/**
* Replays macOS background presses (see `src/main/runtime/macos-controller-watch.ts`) into the
* controller poll. A queued button reads as pressed for one poll, so the controller sees a
* normal press edge; the next poll reads the real pad again, so a button still held does not
* fire twice. A press the Gamepad API already reported is dropped.
*
* Only a lone standard-mapping pad gets the replay: the press carries no pad identity, so with
* several visible it could run an action on the wrong one. With none visible yet, or a pad
* mapped differently, the press only served to focus the overlay.
*/
export function createControllerPressReplay() {
const pendingPresses: Array<{ buttonIndex: number; queuedAt: number }> = [];
const lastRealPressAt = new Map<number, number>();
function recordRealPresses(gamepads: GamepadLike[], now: number): void {
for (const gamepad of gamepads) {
gamepad.buttons.forEach((button, buttonIndex) => {
if (button.pressed) lastRealPressAt.set(buttonIndex, now);
});
}
}
function withPressed(gamepad: GamepadLike, buttonIndices: number[]): GamepadLike {
const buttons = gamepad.buttons.map((button, index) =>
buttonIndices.includes(index) ? { value: 1, pressed: true, touched: true } : button,
);
// Native Gamepad fields are prototype getters, so object spread would drop them.
return {
id: gamepad.id,
index: gamepad.index,
connected: gamepad.connected,
mapping: gamepad.mapping,
timestamp: gamepad.timestamp,
axes: gamepad.axes,
buttons,
};
}
return {
queue(buttonIndex: number, now: number): void {
pendingPresses.push({ buttonIndex, queuedAt: now });
},
/** Called once per poll with the Gamepad API's pads; returns the pads to read. */
apply(gamepads: Array<GamepadLike | null>, now: number): Array<GamepadLike | null> {
const connected = gamepads.filter((gamepad): gamepad is GamepadLike =>
Boolean(gamepad?.connected),
);
recordRealPresses(connected, now);
if (pendingPresses.length === 0) return gamepads;
const replayed = pendingPresses
.filter(({ buttonIndex, queuedAt }) => {
const seenAt = lastRealPressAt.get(buttonIndex);
return seenAt === undefined || seenAt < queuedAt - SAME_PRESS_WINDOW_MS;
})
.map(({ buttonIndex }) => buttonIndex);
pendingPresses.length = 0;
const onlyPad = connected.length === 1 ? connected[0] : undefined;
if (replayed.length === 0 || onlyPad?.mapping !== 'standard') return gamepads;
return gamepads.map((gamepad) =>
gamepad === onlyPad ? withPressed(onlyPad, replayed) : gamepad,
);
},
};
}
-6
View File
@@ -30,12 +30,6 @@
<body>
<!-- Programmatic focus fallback target for Electron/window focus management. -->
<div id="overlay" tabindex="-1">
<div
id="controllerStatusToast"
class="controller-status-toast hidden"
role="status"
aria-live="polite"
></div>
<div
id="overlayErrorToast"
class="overlay-error-toast hidden"
+16 -5
View File
@@ -29,6 +29,7 @@ import type {
import type { OverlayHostedModal } from '../shared/ipc/contracts';
import { createKeyboardHandlers } from './handlers/keyboard.js';
import { createGamepadController } from './handlers/gamepad-controller.js';
import { createControllerPressReplay } from './handlers/controller-press-replay.js';
import { createMouseHandlers } from './handlers/mouse.js';
import { createControllerStatusIndicator } from './controller-status-indicator.js';
import { createControllerDebugModal } from './modals/controller-debug.js';
@@ -268,7 +269,9 @@ const controllerDebugModal = createControllerDebugModal(ctx, {
syncSettingsModalSubtitleSuppression,
notifyControllerDisabled: showControllerDisabledNotice,
});
const controllerStatusIndicator = createControllerStatusIndicator(ctx.dom);
const controllerStatusIndicator = createControllerStatusIndicator((message) => {
window.electronAPI.notifyControllerStatus(message);
});
const sessionHelpModal = createSessionHelpModal(ctx, {
modalStateReader: { isAnyModalOpen },
syncSettingsModalSubtitleSuppression,
@@ -394,6 +397,7 @@ let lastSubtitlePreview = '';
let lastSecondarySubtitlePreview = '';
let overlayErrorToastTimeout: ReturnType<typeof setTimeout> | null = null;
let controllerAnimationFrameId: number | null = null;
const controllerPressReplay = createControllerPressReplay();
function truncateForErrorLog(text: string): string {
const normalized = text.replace(/\s+/g, ' ').trim();
@@ -429,8 +433,9 @@ function applyControllerSnapshot(snapshot: {
rawButtons: Array<{ value: number; pressed: boolean; touched?: boolean }>;
}): void {
// Modal windows are fresh documents that only see the pad after a press; announcing
// that as "Controller detected" on every dialog reads like a reconnection.
if (!ctx.platform.isModalLayer) {
// that as "Controller connected" on every dialog reads like a reconnection. Polling also
// runs with controller support off, where a system notification would just be noise.
if (!ctx.platform.isModalLayer && ctx.state.controllerConfig?.enabled === true) {
controllerStatusIndicator.update({
connectedGamepads: snapshot.connectedGamepads,
activeGamepadId: snapshot.activeGamepadId,
@@ -498,7 +503,7 @@ function withAltCSelectedController(config: ResolvedControllerConfig): ResolvedC
let reportedActiveControllerId: string | null | undefined;
function showControllerDisabledNotice(): void {
controllerStatusIndicator.show(
window.electronAPI.notifyControllerStatus(
'Controller support disabled. Set controller.enabled to true in config to use controller tools.',
);
}
@@ -528,7 +533,8 @@ function startControllerPolling(): void {
}
const gamepadController = createGamepadController({
getGamepads: () => Array.from(navigator.getGamepads?.() ?? []),
getGamepads: () =>
controllerPressReplay.apply(Array.from(navigator.getGamepads?.() ?? []), performance.now()),
getConfig: () =>
withAltCSelectedController(
ctx.state.controllerConfig ?? {
@@ -862,6 +868,11 @@ function registerKeyboardCommandHandlers(): void {
keyboardModeAutoPause.handlePlaybackPauseChanged(paused);
});
// macOS: replay a press made while mpv was in front; main has just focused this window.
window.electronAPI.onControllerBackgroundPress((buttonIndex) => {
if (buttonIndex >= 0) controllerPressReplay.queue(buttonIndex, performance.now());
});
// Alt+C saves in its own window; pick up remaps and the button style without a restart.
window.electronAPI.onControllerConfigChanged(() => {
window.electronAPI
-27
View File
@@ -93,33 +93,6 @@ body:focus-visible,
pointer-events: auto;
}
.controller-status-toast {
position: absolute;
top: 16px;
left: 16px;
max-width: min(360px, calc(100vw - 32px));
padding: 8px 12px;
border-radius: 999px;
border: 1px solid rgba(138, 213, 202, 0.45);
background: linear-gradient(135deg, rgba(10, 44, 40, 0.94), rgba(8, 28, 33, 0.94));
color: rgba(228, 255, 251, 0.98);
font-size: 12px;
font-weight: 700;
letter-spacing: 0.02em;
pointer-events: none;
opacity: 0;
transform: translateY(-6px);
transition:
opacity 160ms ease,
transform 160ms ease;
z-index: 1300;
}
.controller-status-toast:not(.hidden) {
opacity: 1;
transform: translateY(0);
}
.overlay-error-toast {
position: absolute;
top: 16px;
-2
View File
@@ -4,7 +4,6 @@ export type RendererDom = {
overlay: HTMLElement;
overlayNotificationStack: HTMLDivElement;
overlayNotificationHistory: HTMLElement;
controllerStatusToast: HTMLDivElement;
overlayErrorToast: HTMLDivElement;
secondarySubContainer: HTMLElement;
secondarySubRoot: HTMLElement;
@@ -222,7 +221,6 @@ export function resolveRendererDom(): RendererDom {
overlay: getRequiredElement<HTMLElement>('overlay'),
overlayNotificationStack: getRequiredElement<HTMLDivElement>('overlayNotificationStack'),
overlayNotificationHistory: getRequiredElement<HTMLElement>('overlayNotificationHistory'),
controllerStatusToast: getRequiredElement<HTMLDivElement>('controllerStatusToast'),
overlayErrorToast: getRequiredElement<HTMLDivElement>('overlayErrorToast'),
secondarySubContainer: getRequiredElement<HTMLElement>('secondarySubContainer'),
secondarySubRoot: getRequiredElement<HTMLElement>('secondarySubRoot'),
+2
View File
@@ -35,6 +35,7 @@ export const IPC_CHANNELS = {
statsNativeDialogClosed: 'stats:native-dialog-closed',
saveControllerConfig: 'save-controller-config',
reportActiveController: 'controller:report-active',
notifyControllerStatus: 'controller:notify-status',
mpvCommand: 'mpv-command',
refreshKnownWords: 'anki:refresh-known-words',
kikuFieldGroupingRespond: 'kiku:field-grouping-respond',
@@ -162,6 +163,7 @@ export const IPC_CHANNELS = {
keyboardModeToggleRequested: 'keyboard-mode-toggle:requested',
controllerSuspendedChanged: 'controller:suspended-changed',
controllerConfigChanged: 'controller:config-changed',
controllerBackgroundPress: 'controller:background-press',
playbackPauseChanged: 'mpv:pause-changed',
lookupWindowToggleRequested: 'lookup-window-toggle:requested',
sessionHelpOpen: 'session-help:open',
+4
View File
@@ -611,6 +611,8 @@ export interface ElectronAPI {
saveControllerConfig: (update: ControllerConfigUpdate) => Promise<void>;
/** The overlay reports which controller it is reading (null when none). */
reportActiveController: (gamepadId: string | null) => void;
/** Announces a controller status message through the configured notification surfaces. */
notifyControllerStatus: (message: string) => void;
/** The controller the overlay last reported, for Alt+C before it sees a pad itself. */
getActiveControllerId: () => Promise<string | null>;
getJimakuMediaInfo: () => Promise<JimakuMediaInfo>;
@@ -695,6 +697,8 @@ export interface ElectronAPI {
onControllerSuspendedChanged: (callback: (suspended: boolean) => void) => void;
/** Alt+C saved controller settings; reload them with `getControllerConfig`. */
onControllerConfigChanged: (callback: () => void) => void;
/** macOS: a button pressed while mpv was in front, which the Gamepad API never saw. */
onControllerBackgroundPress: (callback: (buttonIndex: number) => void) => void;
/** mpv's pause property changed, from any source. */
onPlaybackPauseChanged: (callback: (paused: boolean) => void) => void;
onLookupWindowToggleRequested: (callback: () => void) => void;
+2 -1
View File
@@ -384,7 +384,8 @@ export interface TrendsDashboardData {
readingSpeed: TrendChartPoint[];
};
librarySummary: LibrarySummaryRow[];
animeCumulative: {
/** Sparse per-title values for each active day; the client accumulates them for charts. */
animeDaily: {
watchTime: TrendPerAnimePoint[];
episodes: TrendPerAnimePoint[];
cards: TrendPerAnimePoint[];
+15 -19
View File
@@ -1,4 +1,4 @@
import { useState } from 'react';
import { useMemo, useState } from 'react';
import { useTrends, type TimeRange, type GroupBy } from '../../hooks/useTrends';
import { DateRangeSelector } from './DateRangeSelector';
import { TrendChart } from './TrendChart';
@@ -20,6 +20,7 @@ import {
type MaxTitlesMode,
} from './anime-visibility';
import { LibrarySummarySection } from './LibrarySummarySection';
import { buildAnimeCumulative } from './anime-cumulative';
const MAX_TITLES_MODE_LABELS: Record<MaxTitlesMode, string> = {
recent: 'most recent',
@@ -156,6 +157,10 @@ export function TrendsTab() {
const [maxTitles, setMaxTitles] = useState<number | null>(() => loadMaxTitles());
const [maxTitlesMode, setMaxTitlesMode] = useState<MaxTitlesMode>(() => loadMaxTitlesMode());
const { data, loading, error } = useTrends(range, groupBy, showEmptyDays);
const animeCumulative = useMemo(
() => (data ? buildAnimeCumulative(data.animeDaily) : null),
[data],
);
const updateHiddenAnime = (next: Set<string>) => {
setHiddenAnime(next);
@@ -187,30 +192,21 @@ export function TrendsTab() {
if (loading) return <div className="text-ctp-overlay2 p-4">Loading...</div>;
if (error) return <div className="text-ctp-red p-4">Error: {error}</div>;
if (!data) return null;
if (!data || !animeCumulative) return null;
const animeTitles = buildAnimeVisibilityOptions([
data.animeCumulative.episodes,
data.animeCumulative.cards,
data.animeCumulative.words,
data.animeCumulative.watchTime,
animeCumulative.episodes,
animeCumulative.cards,
animeCumulative.words,
animeCumulative.watchTime,
]);
const activeHiddenAnime = pruneHiddenAnime(hiddenAnime, animeTitles);
const filteredAnimeProgress = filterHiddenAnimeData(
data.animeCumulative.episodes,
activeHiddenAnime,
);
const filteredCardsProgress = filterHiddenAnimeData(
data.animeCumulative.cards,
activeHiddenAnime,
);
const filteredWordsProgress = filterHiddenAnimeData(
data.animeCumulative.words,
activeHiddenAnime,
);
const filteredAnimeProgress = filterHiddenAnimeData(animeCumulative.episodes, activeHiddenAnime);
const filteredCardsProgress = filterHiddenAnimeData(animeCumulative.cards, activeHiddenAnime);
const filteredWordsProgress = filterHiddenAnimeData(animeCumulative.words, activeHiddenAnime);
const filteredWatchTimeProgress = filterHiddenAnimeData(
data.animeCumulative.watchTime,
animeCumulative.watchTime,
activeHiddenAnime,
);
@@ -0,0 +1,36 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { buildCumulativePerAnime } from './anime-cumulative';
test('buildCumulativePerAnime carries each title forward to the last active day', () => {
const result = buildCumulativePerAnime([
{ epochDay: 103, animeTitle: 'B', value: 4 },
{ epochDay: 100, animeTitle: 'A', value: 10 },
{ epochDay: 102, animeTitle: 'A', value: 5 },
{ epochDay: 102, animeTitle: 'A', value: 1 },
]);
assert.deepEqual(
result
.filter((point) => point.animeTitle === 'A')
.map((point) => [point.epochDay, point.value]),
[
[100, 10],
[101, 10],
[102, 16],
[103, 16],
],
);
// A title starts at its own first active day, not the earliest day overall.
assert.deepEqual(
result
.filter((point) => point.animeTitle === 'B')
.map((point) => [point.epochDay, point.value]),
[[103, 4]],
);
});
test('buildCumulativePerAnime returns nothing for no points', () => {
assert.deepEqual(buildCumulativePerAnime([]), []);
});
@@ -0,0 +1,41 @@
import type { TrendsDashboardData } from '../../types/stats';
import type { PerAnimeDataPoint } from './StackedTrendChart';
type AnimeSeries = TrendsDashboardData['animeDaily'];
/**
* Turns sparse per-title daily values into running totals: one point per
* title for every calendar day from its first active day to the last active
* day of any title, so the stacked area charts carry each total forward.
*/
export function buildCumulativePerAnime(points: PerAnimeDataPoint[]): PerAnimeDataPoint[] {
const byAnime = new Map<string, Map<number, number>>();
let maxDay = Number.NEGATIVE_INFINITY;
for (const point of points) {
const dayMap = byAnime.get(point.animeTitle) ?? new Map<number, number>();
dayMap.set(point.epochDay, (dayMap.get(point.epochDay) ?? 0) + point.value);
byAnime.set(point.animeTitle, dayMap);
maxDay = Math.max(maxDay, point.epochDay);
}
const result: PerAnimeDataPoint[] = [];
for (const [animeTitle, dayMap] of byAnime) {
const firstDay = Math.min(...dayMap.keys());
let cumulative = 0;
for (let epochDay = firstDay; epochDay <= maxDay; epochDay += 1) {
cumulative += dayMap.get(epochDay) ?? 0;
result.push({ epochDay, animeTitle, value: cumulative });
}
}
return result;
}
export function buildAnimeCumulative(daily: AnimeSeries): AnimeSeries {
return {
watchTime: buildCumulativePerAnime(daily.watchTime),
episodes: buildCumulativePerAnime(daily.episodes),
cards: buildCumulativePerAnime(daily.cards),
words: buildCumulativePerAnime(daily.words),
};
}
+1 -1
View File
@@ -105,7 +105,7 @@ test('getTrendsDashboard accepts 365d range and builds correct URL', async () =>
},
ratios: { lookupsPerHundred: [] },
librarySummary: [],
animeCumulative: { watchTime: [], episodes: [], cards: [], words: [] },
animeDaily: { watchTime: [], episodes: [], cards: [], words: [] },
patterns: { watchTimeByDayOfWeek: [], watchTimeByHour: [] },
}),
async (requests) => {