Files
SubMiner/src/core/services/immersion-tracker/query-library.ts
T
sudacode 430373f010 feat(tokenizer): use Yomitan word classes for subtitle POS filtering (#57)
* feat(tokenizer): use Yomitan word classes for subtitle POS filtering

- Carry matched headword wordClasses from termsFind into YomitanScanToken
- Map recognized Yomitan wordClasses to SubMiner coarse POS before annotation
- MeCab enrichment now fills only missing POS fields, preserving existing coarse pos1
- Exclude standalone grammar particles, して helper fragments, and single-kana surfaces from annotations
- Respect source-text punctuation gaps when counting N+1 sentence words
- Preserve known-word highlight on excluded kanji-containing tokens
- Add backlog tasks 304 (N+1 boundary bug) and 305 (wordClasses POS, done)

* fix(tokenizer): preserve annotation and enrichment behavior

* fix: restore jlpt subtitle underlines

* fix: exclude kana-only n+1 targets

* fix: refresh overlay on Hyprland fullscreen

* fix: address fullscreen and n-plus-one review notes

* fix: address CodeRabbit review comments

* fix: accept modified digits for multi-line sentence mining

* Cancel pending Linux MPV fullscreen overlay refresh bursts

- return a cancel handle from the Linux refresh burst scheduler
- clear pending refresh bursts when overlays hide or windows close
- tighten the burst test polling to wait for the async refresh

* fix: suppress N+1 for kana-only candidates and fix minSentenceWords coun

- Treat kana-only tokens with surrounding subtitle punctuation (…, ―, etc.) as kana-only so they are not promoted to N+1 targets
- Exclude unknown tokens filtered from N+1 targeting from the minSentenceWords count so filtered kana-only unknowns cannot satisfy sentence length threshold
- Add regression tests for kana-only candidate suppression and filtered-unknown padding cases

* Suppress subtitle annotations for grammar fragments

- Hide annotation metadata for auxiliary inflection and ja-nai endings
- Preserve lexical `くれる` forms and add regression coverage

* Fix kana-only N+1 tokenizer regression test

- Use a pure-kana fixture for the subtitle token N+1 case
- Update task notes for the latest CodeRabbit follow-up

* Fix managed playback exit and tokenizer grammar splits

- Ignore background stats daemons during regular app startup
- Split standalone grammar endings before applying annotations
- Clear helper-span annotations for auxiliary-only tokens

* fix: refresh current subtitle after known-word mining

* fix: suppress sigh interjection annotations

* fix: preserve jlpt underline color after lookup

* Replace grammar-ending permutations with shared matcher; preserve word a

- Extract `grammar-ending.ts` with `isStandaloneGrammarEndingText` / `isSubtitleGrammarEndingText` pattern matchers
- Replace `STANDALONE_GRAMMAR_ENDINGS` set in parser-selection-stage with shared matcher
- Replace generated phrase sets in subtitle-annotation-filter with shared matcher
- Remove stale duplicate subtitle-exclusion constants and helpers from annotation-stage
- Manual clipboard card updates now write only to the sentence audio field, leaving word/expression audio untouched

* fix: CI changelog, annotation options threading, and Jellyfin quit

- Add `type: fixed` / `area:` frontmatter to `changes/319` to pass `changelog:lint`
- Thread `TokenizerAnnotationOptions` through `stripSubtitleAnnotationMetadata` so `sourceText` is honored
- Include `jellyfinPlay` in `shouldQuitOnDisconnectWhenOverlayRuntimeInitialized` predicate
- Make mouse test `elementFromPoint` stubs coordinate-sensitive
- Make Lua test `.tmp` mkdir portable on Windows

* Preserve overlay across macOS flaps and mpv playlist changes

- keep visible overlays alive during transient macOS tracker loss
- reuse the running mpv overlay path on playlist navigation
- update regression coverage and changelog fragments

* fix: restore stats daemon deferral

* fix: keep subtitle prefetch alive after cache hits

* Fix JLPT underline color drift and AniList skipped-threshold sync

- Replace JLPT `text-decoration` underlines with `border-bottom` so Chromium selection/hover cannot repaint them to another annotation's color
- Lock JLPT underline color for combined annotation selectors (known, n+1, frequency) and character hover/selection states
- Trigger AniList post-watch check on every mpv time-position update to catch skipped completion thresholds
- Fall back to filename-parser season/episode when guessit omits them

* fix: address coderabbit feedback

* fix: sync AniList after seeked completion

* fix: preserve ordinal frequency annotations

* fix: preserve known highlighting for filtered tokens

* fix: address PR #57 CodeRabbit feedback

- Acquire AniList post-watch in-flight lock before async gating to prevent duplicate writes
- Isolate manual watched mark result from AniList post-watch callback failures
- Report known-word cache clears as mutations during immediate append when state existed
- Add regression tests for each fix

* fix: stop AniList setup reopening on Linux when keyring token exists

- Gate setup success on token persistence: `saveToken` now returns `boolean`; on failure, keeps the setup window open instead of reporting success
- Config reload passes `allowSetupPrompt: false` so playback reloads don't re-open the setup window
- Add regression test for persistence-failure path

* fix: suppress known highlights for subtitle particles

* fix: retry transient AniList safeStorage failures

* fix: hide overlay focus ring

* fix: align Hyprland fullscreen overlays

* fix: restore subtitle playback keybindings

* fix: align Hyprland overlay windows to mpv and stop pinning them

- Force-apply exact Hyprland move/resize/setprop dispatches when bounds are provided
- Stop pinning overlay windows; toggle pin off when Hyprland reports pinned=true
- Compensate stats overlay outer placement for Electron/Wayland content insets
- Make stats overlay window and page opaque so mpv cannot show through transparent insets
- Constrain stats app to h-screen with internal scroll so content covers mpv from y=0
- Lock overlay/stats window titles against page-title-updated events
- Add regression coverage for placement dispatches, inset compensation, and CSS overlay mode

* fix: retain frequency rank for honorific prefix-noun tokens

- Add `shouldAllowHonorificPrefixNounFrequency` to exempt お/ご/御 + noun merged tokens from frequency exclusion
- Add regression test for `ご機嫌` asserting rank 5484 is preserved after MeCab enrichment and annotation
- Close TASK-341

* fix: map openCharacterDictionary session action to --open-character-dict

- Add missing Lua CLI dispatch entry for openCharacterDictionary
- Add regression test for Alt+Meta+A binding and CLI flag forwarding

* fix: keep macOS overlay interactive while mpv remains active

- Overlay no longer hides or becomes click-through during tracker refreshes when mpv is the focused window
- Preserve already-visible overlay when tracker is temporarily not ready but mpv target signal is active
- Add regression tests for active-mpv tracker refresh and transient tracker-not-ready paths

* fix: address coderabbit subtitle follow-ups

* fix: resolve media detail from sessions when lifetime summary is absent

- Change `getMediaDetail` JOIN to LEFT JOIN on `imm_lifetime_media` and fall back to aggregated session metrics when no lifetime row exists
- Add filter `AND (lm.video_id IS NOT NULL OR s.session_id IS NOT NULL)` to keep results valid
- Add regression test covering the session-visible / media-detail-missing mismatch

* fix: address PR-57 CodeRabbit findings and CI failures

- use filtered word counts in media detail session token aggregation
- cancel fullscreen refresh burst on exit via updateLinuxMpvFullscreenOverlayRefreshBurst
- guard Hyprland JSON.parse in try/catch; exclude windowtitle from geometry events
- narrow focus suppression from :focus to :focus-visible
- apply JLPT lock selectors to word-name-match tokens (N1–N5)

* fix: macOS overlay z-order and Yomitan compound token known highlighting

- Release always-on-top when tracked mpv loses foreground on macOS
- Skip visible overlay blur restacking on macOS to avoid covering unrelated windows
- Prefer Yomitan internal parse tokens over fragmented scanner output for known-word decisions
- Add regression tests for both behaviors

* fix: macOS visible-overlay blur no longer invokes Windows-only blur call

- Split win32/darwin branches in handleOverlayWindowBlurred so darwin visible blur returns early without calling onWindowsVisibleOverlayBlur
- Add regression test asserting Windows callback stays inactive on macOS visible overlay blur
- Close TASK-347
2026-05-12 12:08:09 -07:00

729 lines
24 KiB
TypeScript

import type { DatabaseSync } from './sqlite';
import type {
AnimeAnilistEntryRow,
AnimeDetailRow,
AnimeEpisodeRow,
AnimeLibraryRow,
AnimeWordRow,
EpisodeCardEventRow,
EpisodesPerDayRow,
ImmersionSessionRollupRow,
MediaArtRow,
MediaDetailRow,
MediaLibraryRow,
NewAnimePerDayRow,
SessionSummaryQueryRow,
StreakCalendarRow,
WatchTimePerAnimeRow,
} from './types';
import {
ACTIVE_SESSION_METRICS_CTE,
SESSION_WORD_COUNTS_CTE,
SESSION_WORD_COUNTS_SELECT,
fromDbTimestamp,
resolvedCoverBlobExpr,
sessionDisplayWordsExpr,
visibleWordSql,
} from './query-shared';
export function getAnimeLibrary(db: DatabaseSync): AnimeLibraryRow[] {
const rows = db
.prepare(
`
SELECT
a.anime_id AS animeId,
a.canonical_title AS canonicalTitle,
a.anilist_id AS anilistId,
COALESCE(lm.total_sessions, 0) AS totalSessions,
COALESCE(lm.total_active_ms, 0) AS totalActiveMs,
COALESCE(lm.total_cards, 0) AS totalCards,
COALESCE(lm.total_tokens_seen, 0) AS totalTokensSeen,
COUNT(DISTINCT v.video_id) AS episodeCount,
a.episodes_total AS episodesTotal,
COALESCE(lm.last_watched_ms, 0) AS lastWatchedMs
FROM imm_anime a
JOIN imm_lifetime_anime lm ON lm.anime_id = a.anime_id
JOIN imm_videos v ON v.anime_id = a.anime_id
GROUP BY a.anime_id
ORDER BY totalActiveMs DESC, lm.last_watched_ms DESC, canonicalTitle ASC
`,
)
.all() as Array<AnimeLibraryRow & { lastWatchedMs: number | string }>;
return rows.map((row) => ({
...row,
lastWatchedMs: fromDbTimestamp(row.lastWatchedMs) ?? 0,
}));
}
export function getAnimeDetail(db: DatabaseSync, animeId: number): AnimeDetailRow | null {
const row = db
.prepare(
`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
a.anime_id AS animeId,
a.canonical_title AS canonicalTitle,
a.anilist_id AS anilistId,
a.title_romaji AS titleRomaji,
a.title_english AS titleEnglish,
a.title_native AS titleNative,
a.description AS description,
COALESCE(lm.total_sessions, 0) AS totalSessions,
COALESCE(lm.total_active_ms, 0) AS totalActiveMs,
COALESCE(lm.total_cards, 0) AS totalCards,
COALESCE(lm.total_tokens_seen, 0) AS totalTokensSeen,
COALESCE(lm.total_lines_seen, 0) AS totalLinesSeen,
COALESCE(SUM(COALESCE(asm.lookupCount, s.lookup_count, 0)), 0) AS totalLookupCount,
COALESCE(SUM(COALESCE(asm.lookupHits, s.lookup_hits, 0)), 0) AS totalLookupHits,
COALESCE(SUM(COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0)), 0) AS totalYomitanLookupCount,
COUNT(DISTINCT v.video_id) AS episodeCount,
COALESCE(lm.last_watched_ms, 0) AS lastWatchedMs
FROM imm_anime a
JOIN imm_lifetime_anime lm ON lm.anime_id = a.anime_id
JOIN imm_videos v ON v.anime_id = a.anime_id
LEFT JOIN imm_sessions s ON s.video_id = v.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
WHERE a.anime_id = ?
GROUP BY a.anime_id
`,
)
.get(animeId) as (AnimeDetailRow & { lastWatchedMs: number | string }) | null;
return row
? {
...row,
lastWatchedMs: fromDbTimestamp(row.lastWatchedMs) ?? 0,
}
: null;
}
export function getAnimeAnilistEntries(db: DatabaseSync, animeId: number): AnimeAnilistEntryRow[] {
return db
.prepare(
`
SELECT DISTINCT
m.anilist_id AS anilistId,
m.title_romaji AS titleRomaji,
m.title_english AS titleEnglish,
v.parsed_season AS season
FROM imm_videos v
JOIN imm_media_art m ON m.video_id = v.video_id
WHERE v.anime_id = ?
AND m.anilist_id IS NOT NULL
ORDER BY v.parsed_season ASC
`,
)
.all(animeId) as unknown as AnimeAnilistEntryRow[];
}
export function getAnimeEpisodes(db: DatabaseSync, animeId: number): AnimeEpisodeRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
v.anime_id AS animeId,
v.video_id AS videoId,
v.canonical_title AS canonicalTitle,
v.parsed_title AS parsedTitle,
v.parsed_season AS season,
v.parsed_episode AS episode,
v.duration_ms AS durationMs,
(
SELECT COALESCE(
NULLIF(s_recent.ended_media_ms, 0),
(
SELECT MAX(line.segment_end_ms)
FROM imm_subtitle_lines line
WHERE line.session_id = s_recent.session_id
AND line.segment_end_ms IS NOT NULL
),
(
SELECT MAX(event.segment_end_ms)
FROM imm_session_events event
WHERE event.session_id = s_recent.session_id
AND event.segment_end_ms IS NOT NULL
)
)
FROM imm_sessions s_recent
WHERE s_recent.video_id = v.video_id
AND (
s_recent.ended_media_ms IS NOT NULL
OR EXISTS (
SELECT 1
FROM imm_subtitle_lines line
WHERE line.session_id = s_recent.session_id
AND line.segment_end_ms IS NOT NULL
)
OR EXISTS (
SELECT 1
FROM imm_session_events event
WHERE event.session_id = s_recent.session_id
AND event.segment_end_ms IS NOT NULL
)
)
ORDER BY
COALESCE(s_recent.ended_at_ms, s_recent.LAST_UPDATE_DATE, s_recent.started_at_ms) DESC,
s_recent.session_id DESC
LIMIT 1
) AS endedMediaMs,
v.watched AS watched,
COUNT(DISTINCT s.session_id) AS totalSessions,
COALESCE(SUM(COALESCE(asm.activeWatchedMs, s.active_watched_ms, 0)), 0) AS totalActiveMs,
COALESCE(SUM(COALESCE(asm.cardsMined, s.cards_mined, 0)), 0) AS totalCards,
COALESCE(SUM(${wordsExpr}), 0) AS totalTokensSeen,
COALESCE(SUM(COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0)), 0) AS totalYomitanLookupCount,
MAX(s.started_at_ms) AS lastWatchedMs
FROM imm_videos v
LEFT JOIN imm_sessions s ON s.video_id = v.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE v.anime_id = ?
GROUP BY v.video_id
ORDER BY
CASE WHEN v.parsed_season IS NULL THEN 1 ELSE 0 END,
v.parsed_season ASC,
CASE WHEN v.parsed_episode IS NULL THEN 1 ELSE 0 END,
v.parsed_episode ASC,
v.video_id ASC
`,
)
.all(animeId) as Array<
AnimeEpisodeRow & {
endedMediaMs: number | string | null;
lastWatchedMs: number | string;
}
>;
return rows.map((row) => ({
...row,
endedMediaMs: fromDbTimestamp(row.endedMediaMs),
lastWatchedMs: fromDbTimestamp(row.lastWatchedMs) ?? 0,
}));
}
export function getMediaLibrary(db: DatabaseSync): MediaLibraryRow[] {
const rows = db
.prepare(
`
SELECT
v.video_id AS videoId,
v.canonical_title AS canonicalTitle,
COALESCE(lm.total_sessions, 0) AS totalSessions,
COALESCE(lm.total_active_ms, 0) AS totalActiveMs,
COALESCE(lm.total_cards, 0) AS totalCards,
COALESCE(lm.total_tokens_seen, 0) AS totalTokensSeen,
COALESCE(lm.last_watched_ms, 0) AS lastWatchedMs,
yv.youtube_video_id AS youtubeVideoId,
yv.video_url AS videoUrl,
yv.video_title AS videoTitle,
yv.video_thumbnail_url AS videoThumbnailUrl,
yv.channel_id AS channelId,
yv.channel_name AS channelName,
yv.channel_url AS channelUrl,
yv.channel_thumbnail_url AS channelThumbnailUrl,
yv.uploader_id AS uploaderId,
yv.uploader_url AS uploaderUrl,
yv.description AS description,
CASE
WHEN ma.cover_blob_hash IS NOT NULL OR ma.cover_blob IS NOT NULL THEN 1
ELSE 0
END AS hasCoverArt
FROM imm_videos v
JOIN imm_lifetime_media lm ON lm.video_id = v.video_id
LEFT JOIN imm_media_art ma ON ma.video_id = v.video_id
LEFT JOIN imm_youtube_videos yv ON yv.video_id = v.video_id
ORDER BY lm.last_watched_ms DESC
`,
)
.all() as Array<MediaLibraryRow & { lastWatchedMs: number | string }>;
return rows.map((row) => ({
...row,
lastWatchedMs: fromDbTimestamp(row.lastWatchedMs) ?? 0,
}));
}
export function getMediaDetail(db: DatabaseSync, videoId: number): MediaDetailRow | null {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
return db
.prepare(
`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
v.video_id AS videoId,
v.canonical_title AS canonicalTitle,
v.anime_id AS animeId,
CASE
WHEN lm.video_id IS NOT NULL THEN COALESCE(lm.total_sessions, 0)
ELSE COUNT(DISTINCT s.session_id)
END AS totalSessions,
CASE
WHEN lm.video_id IS NOT NULL THEN COALESCE(lm.total_active_ms, 0)
ELSE COALESCE(SUM(COALESCE(asm.activeWatchedMs, s.active_watched_ms, 0)), 0)
END AS totalActiveMs,
CASE
WHEN lm.video_id IS NOT NULL THEN COALESCE(lm.total_cards, 0)
ELSE COALESCE(SUM(COALESCE(asm.cardsMined, s.cards_mined, 0)), 0)
END AS totalCards,
CASE
WHEN lm.video_id IS NOT NULL THEN COALESCE(lm.total_tokens_seen, 0)
ELSE COALESCE(SUM(${wordsExpr}), 0)
END AS totalTokensSeen,
CASE
WHEN lm.video_id IS NOT NULL THEN COALESCE(lm.total_lines_seen, 0)
ELSE COALESCE(SUM(COALESCE(asm.linesSeen, s.lines_seen, 0)), 0)
END AS totalLinesSeen,
COALESCE(SUM(COALESCE(asm.lookupCount, s.lookup_count, 0)), 0) AS totalLookupCount,
COALESCE(SUM(COALESCE(asm.lookupHits, s.lookup_hits, 0)), 0) AS totalLookupHits,
COALESCE(SUM(COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0)), 0) AS totalYomitanLookupCount,
yv.youtube_video_id AS youtubeVideoId,
yv.video_url AS videoUrl,
yv.video_title AS videoTitle,
yv.video_thumbnail_url AS videoThumbnailUrl,
yv.channel_id AS channelId,
yv.channel_name AS channelName,
yv.channel_url AS channelUrl,
yv.channel_thumbnail_url AS channelThumbnailUrl,
yv.uploader_id AS uploaderId,
yv.uploader_url AS uploaderUrl,
yv.description AS description
FROM imm_videos v
LEFT JOIN imm_lifetime_media lm ON lm.video_id = v.video_id
LEFT JOIN imm_youtube_videos yv ON yv.video_id = v.video_id
LEFT JOIN imm_sessions s ON s.video_id = v.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE v.video_id = ?
AND (lm.video_id IS NOT NULL OR s.session_id IS NOT NULL)
GROUP BY v.video_id
`,
)
.get(videoId) as unknown as MediaDetailRow | null;
}
export function getMediaSessions(
db: DatabaseSync,
videoId: number,
limit = 100,
): SessionSummaryQueryRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
s.session_id AS sessionId,
s.video_id AS videoId,
v.canonical_title AS canonicalTitle,
s.started_at_ms AS startedAtMs,
s.ended_at_ms AS endedAtMs,
COALESCE(asm.totalWatchedMs, s.total_watched_ms, 0) AS totalWatchedMs,
COALESCE(asm.activeWatchedMs, s.active_watched_ms, 0) AS activeWatchedMs,
COALESCE(asm.linesSeen, s.lines_seen, 0) AS linesSeen,
${wordsExpr} AS tokensSeen,
COALESCE(asm.cardsMined, s.cards_mined, 0) AS cardsMined,
COALESCE(asm.lookupCount, s.lookup_count, 0) AS lookupCount,
COALESCE(asm.lookupHits, s.lookup_hits, 0) AS lookupHits,
COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0) AS yomitanLookupCount
FROM imm_sessions s
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
LEFT JOIN imm_videos v ON v.video_id = s.video_id
WHERE s.video_id = ?
ORDER BY s.started_at_ms DESC
LIMIT ?
`,
)
.all(videoId, limit) as Array<
SessionSummaryQueryRow & {
startedAtMs: number | string;
endedAtMs: number | string | null;
}
>;
return rows.map((row) => ({
...row,
startedAtMs: fromDbTimestamp(row.startedAtMs) ?? 0,
endedAtMs: fromDbTimestamp(row.endedAtMs),
}));
}
export function getMediaDailyRollups(
db: DatabaseSync,
videoId: number,
limit = 90,
): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
return db
.prepare(
`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
),
daily_word_counts AS (
SELECT
CAST(
julianday(CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
AS INTEGER
) AS rollupDay,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
GROUP BY rollupDay, s.video_id
),
recent_days AS (
SELECT DISTINCT rollup_day
FROM imm_daily_rollups
WHERE video_id = ?
ORDER BY rollup_day DESC
LIMIT ?
)
SELECT
rollup_day AS rollupDayOrMonth,
video_id AS videoId,
total_sessions AS totalSessions,
total_active_min AS totalActiveMin,
total_lines_seen AS totalLinesSeen,
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > total_tokens_seen THEN dwc.totalTokensSeen
ELSE total_tokens_seen
END AS totalTokensSeen,
total_cards AS totalCards,
cards_per_hour AS cardsPerHour,
CASE
WHEN total_active_min > 0 THEN (
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > total_tokens_seen THEN dwc.totalTokensSeen
ELSE total_tokens_seen
END
) * 1.0 / total_active_min
ELSE NULL
END AS tokensPerMin,
lookup_hit_rate AS lookupHitRate
FROM imm_daily_rollups
LEFT JOIN daily_word_counts dwc
ON dwc.rollupDay = rollup_day
AND dwc.videoId = video_id
WHERE video_id = ?
AND rollup_day IN (SELECT rollup_day FROM recent_days)
ORDER BY rollup_day DESC, video_id DESC
`,
)
.all(videoId, limit, videoId) as unknown as ImmersionSessionRollupRow[];
}
export function getAnimeDailyRollups(
db: DatabaseSync,
animeId: number,
limit = 90,
): ImmersionSessionRollupRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc');
return db
.prepare(
`
WITH session_word_counts AS (
${SESSION_WORD_COUNTS_SELECT}
),
daily_word_counts AS (
SELECT
CAST(
julianday(CAST(s.started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
AS INTEGER
) AS rollupDay,
s.video_id AS videoId,
SUM(${wordsExpr}) AS totalTokensSeen
FROM imm_sessions s
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.ended_at_ms IS NOT NULL
GROUP BY rollupDay, s.video_id
),
recent_days AS (
SELECT DISTINCT r.rollup_day
FROM imm_daily_rollups r
JOIN imm_videos v ON v.video_id = r.video_id
WHERE v.anime_id = ?
ORDER BY r.rollup_day DESC
LIMIT ?
)
SELECT r.rollup_day AS rollupDayOrMonth, r.video_id AS videoId,
r.total_sessions AS totalSessions, r.total_active_min AS totalActiveMin,
r.total_lines_seen AS totalLinesSeen,
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > r.total_tokens_seen THEN dwc.totalTokensSeen
ELSE r.total_tokens_seen
END AS totalTokensSeen,
r.total_cards AS totalCards,
r.cards_per_hour AS cardsPerHour,
CASE
WHEN r.total_active_min > 0 THEN (
CASE
WHEN dwc.totalTokensSeen IS NOT NULL AND dwc.totalTokensSeen > r.total_tokens_seen THEN dwc.totalTokensSeen
ELSE r.total_tokens_seen
END
) * 1.0 / r.total_active_min
ELSE NULL
END AS tokensPerMin,
r.lookup_hit_rate AS lookupHitRate
FROM imm_daily_rollups r
JOIN imm_videos v ON v.video_id = r.video_id
LEFT JOIN daily_word_counts dwc
ON dwc.rollupDay = r.rollup_day
AND dwc.videoId = r.video_id
WHERE v.anime_id = ?
AND r.rollup_day IN (SELECT rollup_day FROM recent_days)
ORDER BY r.rollup_day DESC, r.video_id DESC
`,
)
.all(animeId, limit, animeId) as unknown as ImmersionSessionRollupRow[];
}
export function getAnimeCoverArt(db: DatabaseSync, animeId: number): MediaArtRow | null {
const resolvedCoverBlob = resolvedCoverBlobExpr('a', 'cab');
const row = db
.prepare(
`
SELECT
a.video_id AS videoId,
a.anilist_id AS anilistId,
a.cover_url AS coverUrl,
${resolvedCoverBlob} AS coverBlob,
a.title_romaji AS titleRomaji,
a.title_english AS titleEnglish,
a.episodes_total AS episodesTotal,
a.fetched_at_ms AS fetchedAtMs
FROM imm_media_art a
JOIN imm_videos v ON v.video_id = a.video_id
LEFT JOIN imm_cover_art_blobs cab ON cab.blob_hash = a.cover_blob_hash
WHERE v.anime_id = ?
AND ${resolvedCoverBlob} IS NOT NULL
ORDER BY a.fetched_at_ms DESC, a.video_id DESC
LIMIT 1
`,
)
.get(animeId) as (MediaArtRow & { fetchedAtMs: number | string }) | null;
return row
? {
...row,
fetchedAtMs: fromDbTimestamp(row.fetchedAtMs) ?? 0,
}
: null;
}
export function getCoverArt(db: DatabaseSync, videoId: number): MediaArtRow | null {
const resolvedCoverBlob = resolvedCoverBlobExpr('a', 'cab');
const row = db
.prepare(
`
SELECT
a.video_id AS videoId,
a.anilist_id AS anilistId,
a.cover_url AS coverUrl,
${resolvedCoverBlob} AS coverBlob,
a.title_romaji AS titleRomaji,
a.title_english AS titleEnglish,
a.episodes_total AS episodesTotal,
a.fetched_at_ms AS fetchedAtMs
FROM imm_media_art a
LEFT JOIN imm_cover_art_blobs cab ON cab.blob_hash = a.cover_blob_hash
WHERE a.video_id = ?
`,
)
.get(videoId) as (MediaArtRow & { fetchedAtMs: number | string }) | null;
return row
? {
...row,
fetchedAtMs: fromDbTimestamp(row.fetchedAtMs) ?? 0,
}
: null;
}
export function getStreakCalendar(db: DatabaseSync, days = 90): StreakCalendarRow[] {
const now = new Date();
const localMidnight = new Date(now.getFullYear(), now.getMonth(), now.getDate()).getTime();
const todayLocalDay = Math.floor(localMidnight / 86_400_000);
const cutoffDay = todayLocalDay - days;
return db
.prepare(
`
SELECT rollup_day AS epochDay, SUM(total_active_min) AS totalActiveMin
FROM imm_daily_rollups
WHERE rollup_day >= ?
GROUP BY rollup_day
ORDER BY rollup_day ASC
`,
)
.all(cutoffDay) as StreakCalendarRow[];
}
export function getAnimeWords(db: DatabaseSync, animeId: number, limit = 50): AnimeWordRow[] {
return db
.prepare(
`
SELECT w.id AS wordId, w.headword, w.word, w.reading, w.part_of_speech AS partOfSpeech,
SUM(o.occurrence_count) AS frequency
FROM imm_word_line_occurrences o
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
JOIN imm_words w ON w.id = o.word_id
WHERE sl.anime_id = ? AND ${visibleWordSql('w')}
GROUP BY w.id
ORDER BY frequency DESC
LIMIT ?
`,
)
.all(animeId, limit) as unknown as AnimeWordRow[];
}
export function getEpisodesPerDay(db: DatabaseSync, limit = 90): EpisodesPerDayRow[] {
return db
.prepare(
`
SELECT CAST(julianday(s.started_at_ms / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER) AS epochDay,
COUNT(DISTINCT s.video_id) AS episodeCount
FROM imm_sessions s
GROUP BY epochDay
ORDER BY epochDay DESC
LIMIT ?
`,
)
.all(limit) as EpisodesPerDayRow[];
}
export function getNewAnimePerDay(db: DatabaseSync, limit = 90): NewAnimePerDayRow[] {
return db
.prepare(
`
SELECT first_day AS epochDay, COUNT(*) AS newAnimeCount
FROM (
SELECT CAST(julianday(MIN(s.started_at_ms) / 1000, 'unixepoch', 'localtime') - 2440587.5 AS INTEGER) AS first_day
FROM imm_sessions s
JOIN imm_videos v ON v.video_id = s.video_id
WHERE v.anime_id IS NOT NULL
GROUP BY v.anime_id
)
GROUP BY first_day
ORDER BY first_day DESC
LIMIT ?
`,
)
.all(limit) as NewAnimePerDayRow[];
}
export function getWatchTimePerAnime(db: DatabaseSync, limit = 90): WatchTimePerAnimeRow[] {
const nowD = new Date();
const cutoffDay =
Math.floor(
new Date(nowD.getFullYear(), nowD.getMonth(), nowD.getDate()).getTime() / 86_400_000,
) - limit;
return db
.prepare(
`
SELECT r.rollup_day AS epochDay, a.anime_id AS animeId,
a.canonical_title AS animeTitle,
SUM(r.total_active_min) AS totalActiveMin
FROM imm_daily_rollups r
JOIN imm_videos v ON v.video_id = r.video_id
JOIN imm_anime a ON a.anime_id = v.anime_id
WHERE r.rollup_day >= ?
GROUP BY r.rollup_day, a.anime_id
ORDER BY r.rollup_day ASC
`,
)
.all(cutoffDay) as WatchTimePerAnimeRow[];
}
export function getEpisodeWords(db: DatabaseSync, videoId: number, limit = 50): AnimeWordRow[] {
return db
.prepare(
`
SELECT w.id AS wordId, w.headword, w.word, w.reading, w.part_of_speech AS partOfSpeech,
SUM(o.occurrence_count) AS frequency
FROM imm_word_line_occurrences o
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
JOIN imm_words w ON w.id = o.word_id
WHERE sl.video_id = ?
GROUP BY w.id
ORDER BY frequency DESC
LIMIT ?
`,
)
.all(videoId, limit) as unknown as AnimeWordRow[];
}
export function getEpisodeSessions(db: DatabaseSync, videoId: number): SessionSummaryQueryRow[] {
const wordsExpr = sessionDisplayWordsExpr('s', 'swc', 'COALESCE(asm.tokensSeen, s.tokens_seen)');
const rows = db
.prepare(
`
${ACTIVE_SESSION_METRICS_CTE}
SELECT
s.session_id AS sessionId, s.video_id AS videoId,
v.canonical_title AS canonicalTitle,
s.started_at_ms AS startedAtMs, s.ended_at_ms AS endedAtMs,
COALESCE(asm.totalWatchedMs, s.total_watched_ms, 0) AS totalWatchedMs,
COALESCE(asm.activeWatchedMs, s.active_watched_ms, 0) AS activeWatchedMs,
COALESCE(asm.linesSeen, s.lines_seen, 0) AS linesSeen,
${wordsExpr} AS tokensSeen,
COALESCE(asm.cardsMined, s.cards_mined, 0) AS cardsMined,
COALESCE(asm.lookupCount, s.lookup_count, 0) AS lookupCount,
COALESCE(asm.lookupHits, s.lookup_hits, 0) AS lookupHits,
COALESCE(asm.yomitanLookupCount, s.yomitan_lookup_count, 0) AS yomitanLookupCount
FROM imm_sessions s
JOIN imm_videos v ON v.video_id = s.video_id
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
LEFT JOIN session_word_counts swc ON swc.sessionId = s.session_id
WHERE s.video_id = ?
ORDER BY s.started_at_ms DESC
`,
)
.all(videoId) as Array<
SessionSummaryQueryRow & {
startedAtMs: number | string;
endedAtMs: number | string | null;
}
>;
return rows.map((row) => ({
...row,
startedAtMs: fromDbTimestamp(row.startedAtMs) ?? 0,
endedAtMs: fromDbTimestamp(row.endedAtMs),
}));
}
export function getEpisodeCardEvents(db: DatabaseSync, videoId: number): EpisodeCardEventRow[] {
const rows = db
.prepare(
`
SELECT e.event_id AS eventId, e.session_id AS sessionId,
e.ts_ms AS tsMs, e.cards_delta AS cardsDelta,
e.payload_json AS payloadJson
FROM imm_session_events e
JOIN imm_sessions s ON s.session_id = e.session_id
WHERE s.video_id = ? AND e.event_type = 4
ORDER BY CAST(e.ts_ms AS REAL) DESC
`,
)
.all(videoId) as Array<{
eventId: number;
sessionId: number;
tsMs: number | string;
cardsDelta: number;
payloadJson: string | null;
}>;
return rows.map((row) => {
let noteIds: number[] = [];
if (row.payloadJson) {
try {
const parsed = JSON.parse(row.payloadJson);
if (Array.isArray(parsed.noteIds)) noteIds = parsed.noteIds;
} catch {}
}
return {
eventId: row.eventId,
sessionId: row.sessionId,
tsMs: fromDbTimestamp(row.tsMs) ?? 0,
cardsDelta: row.cardsDelta,
noteIds,
};
});
}