mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-08-14 01:55:58 -07:00
Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sudacode
|
d2f9a91d39
|
||
|
sudacode
|
7cc9f72fc9
|
||
|
sudacode
|
de5b1b5d10
|
||
|
sudacode
|
b98d4d65c7 | ||
|
sudacode
|
046e74ea91
|
||
|
sudacode
|
8bf847503d |
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"name": "subminer-local",
|
||||
"interface": {
|
||||
"displayName": "SubMiner Local"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "subminer-workflow",
|
||||
"source": {
|
||||
"source": "local",
|
||||
"path": "./plugins/subminer-workflow"
|
||||
},
|
||||
"policy": {
|
||||
"installation": "AVAILABLE",
|
||||
"authentication": "ON_INSTALL"
|
||||
},
|
||||
"category": "Productivity"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,22 +1,45 @@
|
||||
---
|
||||
name: 'subminer-change-verification'
|
||||
description: 'Compatibility shim. Canonical SubMiner change verification workflow now lives in the repo-local subminer-workflow plugin.'
|
||||
name: subminer-change-verification
|
||||
description: Verify SubMiner changes with repo-native cheap-first test lanes. Use after code, config, launcher, plugin, runtime, stats, documentation, or workflow changes; do not use for read-only questions.
|
||||
---
|
||||
|
||||
# Compatibility Shim
|
||||
# SubMiner Change Verification
|
||||
|
||||
Canonical source:
|
||||
Verify the behavior claimed by a change without running unrelated expensive checks by default.
|
||||
|
||||
- `plugins/subminer-workflow/skills/subminer-change-verification/SKILL.md`
|
||||
## Workflow
|
||||
|
||||
Canonical helper scripts:
|
||||
1. Inspect the requested scope and changed paths with `git status --short` and `git diff`.
|
||||
2. Read `docs/workflow/verification.md` as the source of truth for maintained lanes.
|
||||
3. Run the cheapest lane or lanes that cover the changed behavior.
|
||||
4. Escalate to the full handoff gate only for substantial or cross-boundary changes.
|
||||
5. Report exact commands, results, skipped checks, blockers, and remaining risk.
|
||||
|
||||
- `plugins/subminer-workflow/skills/subminer-change-verification/scripts/classify_subminer_diff.sh`
|
||||
- `plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh`
|
||||
Do not use hidden wrapper commands. Verification commands are owned by `package.json` and the workflow documentation.
|
||||
|
||||
When this shim is invoked:
|
||||
## Lane Selection
|
||||
|
||||
1. Read the canonical plugin-owned skill.
|
||||
2. Follow the plugin-owned skill as the source of truth.
|
||||
3. Use the wrapper scripts in this shim directory only for compatibility with existing commands and docs.
|
||||
4. Do not duplicate workflow changes here; update the plugin-owned skill and scripts instead.
|
||||
- Internal docs, `AGENTS.md`, or `.agents/skills/**`: `bun run test:docs:kb`
|
||||
- User-facing `docs-site/**`: `bun run docs:test`, then `bun run docs:build`
|
||||
- Config/schema/defaults: `bun run test:config`
|
||||
- If defaults or templates changed, also run `bun run generate:config-example` and `bun run verify:config-example`.
|
||||
- General TypeScript source: `bun run typecheck`, then `bun run test:fast`
|
||||
- Launcher or mpv plugin: `bun run test:launcher` or `bun run test:env`, based on the behavior changed
|
||||
- Runtime compatibility or dist-sensitive wiring: `bun run test:runtime:compat`
|
||||
- Stats dashboard: `bun run test:stats`
|
||||
- Build/release scripts: `bun run test:scripts`
|
||||
|
||||
For substantial changes, use the full gate documented in `AGENTS.md` and `docs/workflow/verification.md`.
|
||||
|
||||
## Runtime Escalation
|
||||
|
||||
Real runtime checks are required when the claim depends on actual Electron, mpv, overlay, focus, window tracking, launch, or socket behavior. Run the relevant application flow when the environment supports it. Otherwise, report the missing runtime dependency and do not present cheaper checks as authoritative runtime validation.
|
||||
|
||||
## Pre-Handoff Checks
|
||||
|
||||
Before handoff, reconcile both questions:
|
||||
|
||||
1. Do behavior, defaults, flags, shortcuts, ports, APIs, architecture, or workflow changes require documentation updates?
|
||||
2. Does the change require a current-outcome fragment under `changes/` according to `changes/README.md`?
|
||||
|
||||
Complete required updates before handoff or report the blocker.
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_ROOT=$(cd "$SCRIPT_DIR/../../../.." && pwd)
|
||||
TARGET="$REPO_ROOT/plugins/subminer-workflow/skills/subminer-change-verification/scripts/classify_subminer_diff.sh"
|
||||
|
||||
if [[ ! -x "$TARGET" ]]; then
|
||||
echo "Missing canonical script: $TARGET" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exec "$TARGET" "$@"
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
REPO_ROOT=$(cd "$SCRIPT_DIR/../../../.." && pwd)
|
||||
TARGET="$REPO_ROOT/plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh"
|
||||
|
||||
if [[ ! -x "$TARGET" ]]; then
|
||||
echo "Missing canonical script: $TARGET" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exec "$TARGET" "$@"
|
||||
@@ -1,18 +0,0 @@
|
||||
---
|
||||
name: 'subminer-scrum-master'
|
||||
description: 'Compatibility shim. Canonical SubMiner scrum-master workflow now lives in the repo-local subminer-workflow plugin.'
|
||||
---
|
||||
|
||||
# Compatibility Shim
|
||||
|
||||
Canonical source:
|
||||
|
||||
- `plugins/subminer-workflow/skills/subminer-scrum-master/SKILL.md`
|
||||
|
||||
When this shim is invoked:
|
||||
|
||||
1. Read the canonical plugin-owned skill.
|
||||
2. Follow the plugin-owned skill as the source of truth.
|
||||
3. Do not duplicate workflow changes here; update the plugin-owned skill instead.
|
||||
|
||||
This shim exists so existing repo references and prompts keep resolving during the migration to the repo-local plugin workflow.
|
||||
@@ -1,4 +1,4 @@
|
||||
# AGENTS.MD
|
||||
# AGENTS.md
|
||||
|
||||
## Internal Docs
|
||||
|
||||
@@ -13,7 +13,7 @@ Start here, then leave this file.
|
||||
|
||||
`docs-site/` is user-facing. Do not treat it as the canonical internal source of truth.
|
||||
|
||||
`CLAUDE.md` is a symlink to this file — there is one project instruction file, not two.
|
||||
`CLAUDE.md` is a symlink to this file; there is one project instruction file, not two.
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -25,8 +25,9 @@ Start here, then leave this file.
|
||||
|
||||
## Build / Test
|
||||
|
||||
- Runtime/package manager: Bun (`packageManager: bun@1.3.5`)
|
||||
- Default handoff gate:
|
||||
- Runtime/package manager: Bun; use the version pinned by `package.json`.
|
||||
- Follow [`docs/workflow/verification.md`](./docs/workflow/verification.md) and start with the cheapest sufficient lane.
|
||||
- Full handoff gate for substantial changes:
|
||||
`bun run typecheck`
|
||||
`bun run test:fast`
|
||||
`bun run test:env`
|
||||
@@ -44,13 +45,15 @@ Start here, then leave this file.
|
||||
- Runtime-compat / dist-sensitive: `bun run test:runtime:compat`
|
||||
- Stats dashboard UI (`stats/`): `bun run test:stats`
|
||||
- Build/release scripts (`scripts/**`): `bun run test:scripts`
|
||||
- Docs-only: `bun run docs:test`, then `bun run docs:build`
|
||||
- Internal docs, `AGENTS.md`, or repo skills: `bun run test:docs:kb`
|
||||
- User-facing `docs-site/`: `bun run docs:test`, then `bun run docs:build`
|
||||
- macOS mpv window helper: `bun test scripts/get-mpv-window-macos.test.ts`
|
||||
- Test lanes are directory-discovered via `scripts/test-lanes.ts`; never hand-list test files in `package.json`
|
||||
|
||||
## Docs Upkeep
|
||||
|
||||
- Docs ship with the change, not after. If a change alters behavior, defaults, flags, shortcuts, ports, or APIs, update the matching docs in the same PR. Touching code without reconciling its docs is an incomplete change.
|
||||
- Source of truth for config defaults is the generated `config.example.jsonc`. Never write a default value into prose you didn't read from it — and don't restate the same default across multiple docs; cite/link to one place so there's a single thing to update.
|
||||
- Source of truth for config defaults is the generated `config.example.jsonc`. Never write a default value into prose you didn't read from it, and don't restate the same default across multiple docs; cite/link to one place so there's a single thing to update.
|
||||
- Trigger map (touch left → update right):
|
||||
- `src/config/definitions/**` (schema/defaults/template) → `bun run generate:config-example`, then reconcile `docs-site/configuration.md` + any feature doc that cites that default
|
||||
- shortcuts/keybindings (`shortcuts.*`, `keybindings`, `stats.*Key`, `subtitleSidebar.toggleKey`, controller bindings) → `docs-site/shortcuts.md`
|
||||
@@ -71,16 +74,10 @@ Start here, then leave this file.
|
||||
|
||||
## Release / PR Notes
|
||||
|
||||
- User-visible PRs need reconciled current-outcome fragment(s) in `changes/*.md` — format and rules in [`changes/README.md`](./changes/README.md) (`type` + `area` keys required; inspect existing same-PR fragments, then update/remove stale bullets or add only genuinely separate outcomes; apply the `skip-changelog` label to opt out)
|
||||
- User-visible PRs need reconciled current-outcome fragment(s) in `changes/*.md`. Format and rules live in [`changes/README.md`](./changes/README.md) (`type` + `area` keys required; inspect existing same-PR fragments, then update/remove stale bullets or add only genuinely separate outcomes; apply the `skip-changelog` label to opt out).
|
||||
- User-visible docs changes get a `type: docs` fragment
|
||||
- CI enforces `bun run changelog:lint` and `bun run changelog:pr-check`
|
||||
- PR review helpers:
|
||||
- `gh pr view --json number,title,url --jq '"PR #\\(.number): \\(.title)\\n\\(.url)"'`
|
||||
- `gh pr view --json number,title --jq '"PR #\\(.number): \\(.title)"'`
|
||||
- `gh api repos/:owner/:repo/pulls/<num>/comments --paginate`
|
||||
|
||||
## Runtime Notes
|
||||
|
||||
- Use Codex background for long jobs; tmux only when persistence/interaction is required
|
||||
- CI red: `gh run list/view`, rerun, fix, repeat until green
|
||||
- TypeScript: keep files small; follow existing patterns
|
||||
- Only Swift is the `scripts/get-mpv-window-macos.swift` helper (macOS mpv window detection); validate via `bun test scripts/get-mpv-window-macos.test.ts`
|
||||
- For CI debugging, inspect runs with `gh run list/view`; rerun or fix only within the requested scope.
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
type: fixed
|
||||
area: Anki media
|
||||
|
||||
- Fixed sentence-audio generation timing out on slow network-mounted MKV files with many subtitle and font-attachment streams. Selected audio tracks now use bounded FFmpeg probing and a two-minute extraction budget, and missing output reports a clear FFmpeg error instead of raw `ENOENT`.
|
||||
@@ -0,0 +1,5 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Typeset subtitles no longer flood the stats. Karaoke openings and animated signs are authored as one subtitle event per animation frame, and immersion tracking counted every frame, which was enough to put an OP lyric at the top of "Top Repeated Words" for good. Lines are now collapsed on the way in using the same rules the subtitle sidebar already applies: matching parsed timings record exactly the cues the sidebar shows, while shifted, changing, or unparsed sources use a strict fallback where identical, contiguous, sub-0.1s lines stop counting after a few frames. Ordinary repeated dialogue and rewatches are unaffected.
|
||||
- Added a cleanup for stats already affected. The Vocabulary tab has a **Duplicates** button that scans a chosen window (7 days through all time), shows the bursts it found and the word and kanji counts they added, and collapses each run to one line once confirmed. `subminer stats cleanup --duplicate-lines` does the same from the terminal, with `--dry-run` and `--lookback-days <n>`. Only subtitle lines and the vocabulary counts they feed are touched; watch time and lines-seen totals are left as recorded.
|
||||
@@ -1,6 +0,0 @@
|
||||
type: added
|
||||
area: stats
|
||||
|
||||
- Library: duplicate cards for the same show can now be combined. Press "Select" above the library grid, tick the cards, and use "Merge Selected"; the dialog picks which entry to keep and moves every episode onto it. Sessions, mined cards, and watch time are preserved, the emptied entries disappear, and remembered title aliases keep future episodes on the merged card.
|
||||
- Library: episodes can be reassigned to another library entry from the "→" button on an episode row, which is the fix when one file lands under a stray title (e.g. an episode name parsed as the series). Manual assignments now survive later filename parsing, Jellyfin refreshes, and season repair. Compatible local episodes in the same directory reuse a uniquely corrected destination, while conflicting seasons or manual destinations are not forced together. Emptying an entry this way removes it and returns to the grid.
|
||||
- Library: exact AniList title matches with compatible seasons fold duplicate cards automatically. Fuzzy same-AniList matches appear as dismissible "Possible duplicate" reviews instead of changing the library without confirmation; conflicting explicit seasons are left alone.
|
||||
@@ -0,0 +1,8 @@
|
||||
type: fixed
|
||||
area: stats
|
||||
|
||||
- Stats deletes no longer freeze the stats dashboard: the delete worker module now resolves when running from source, so deletes actually run off the serving thread instead of silently falling back to it.
|
||||
- Deletes now subtract their exact contribution from lifetime summaries instead of rebuilding them from retained sessions, making delete cost proportional to what is deleted and preserving lifetime totals older than the session retention window.
|
||||
- If the delete worker crashes, the delete now retries on the current thread instead of failing.
|
||||
- Library merges, video moves, AniList reassignments, and `subminer stats cleanup -l` also stopped rebuilding lifetime summaries from retained sessions; they now recompute from per-episode history, so those operations are faster and no longer erase lifetime totals older than the session retention window.
|
||||
- Deleting content that contains very common words no longer rescans every occurrence of those words across the whole library; first/last-seen dates are refreshed with index seeks instead.
|
||||
@@ -34,7 +34,7 @@ The same immersion data powers the stats dashboard.
|
||||
- In-app overlay: focus the visible overlay, then press the key from `stats.toggleKey` (default: `` ` `` / `Backquote`).
|
||||
- Launcher command: run `subminer stats` to start the local stats server on demand (it also opens the dashboard in your browser when `stats.autoOpenBrowser` is enabled; the default is `false`).
|
||||
- Background server: run `subminer stats -b` to start or reuse a dedicated background stats daemon without keeping the launcher attached, and `subminer stats -s` to stop that daemon.
|
||||
- Maintenance commands: run `subminer stats cleanup` or `subminer stats cleanup -v` to backfill/repair vocabulary metadata (`headword`, `reading`, POS) and purge stale or excluded rows from `imm_words` on demand; `subminer stats cleanup -l` repairs lifetime summary tables. `subminer stats rebuild` and `subminer stats backfill` rebuild or backfill rollup data.
|
||||
- Maintenance commands: run `subminer stats cleanup` or `subminer stats cleanup -v` to backfill/repair vocabulary metadata (`headword`, `reading`, POS) and purge stale or excluded rows from `imm_words` on demand; `subminer stats cleanup -l` repairs lifetime summary tables non-destructively (recomputed from per-episode history, so lifetime totals older than the session retention window are kept); `subminer stats cleanup --duplicate-lines` collapses repeated lines left behind by typeset subtitles (see [Repeated Line Cleanup](#repeated-line-cleanup)). `subminer stats rebuild` and `subminer stats backfill` rebuild or backfill rollup data.
|
||||
- Browser page: open `http://127.0.0.1:6969` directly if the local stats server is already running.
|
||||
|
||||
### Dashboard Tabs
|
||||
@@ -57,13 +57,6 @@ Jellyfin stream URLs are normalized to stable item links before stats titles are
|
||||
|
||||
When YouTube channel metadata is available, the Library tab groups videos by creator/channel and treats each tracked video as an episode-like entry inside that channel section.
|
||||
|
||||
A library entry is identified by its parsed title plus any detected season, so the same show can end up on several cards when releases disagree about the title or omit the season tag. Two fixes are available:
|
||||
|
||||
- **Merge duplicates.** Hit **Select** above the grid, tick the cards that are the same show, and choose **Merge Selected**. Pick which entry to keep in the dialog; every episode moves onto it and the other cards are removed. Nothing is deleted, so sessions, mined cards and watch time all carry over. SubMiner remembers the merged title variants, so future episodes parsed with one of those names join the kept entry instead of recreating a duplicate card.
|
||||
- **Move a single episode.** Hover an episode row in a title's episode list and use the **→** button to reassign it to another library entry. The correction is remembered, so later filename parsing or Jellyfin metadata cannot move that episode back. For local files, later episodes in the same directory inherit the correction when their detected seasons are compatible and every manual correction there points to the same entry. Conflicting seasons or manual destinations are left for review. If the move empties the old entry, that card is removed and you are returned to the grid.
|
||||
|
||||
Once cover art resolves a series to an AniList entry, cards with compatible seasons are folded together automatically only when the searched title exactly matches an AniList title or synonym. A fuzzy result that points at an AniList entry already used by another card appears as a **Possible duplicate** review above the Library grid instead. Choose **Review merge** to compare the cards and pick which one to keep, or **Not duplicates** to dismiss that suggestion permanently. Entries with conflicting explicit season numbers are left alone rather than merged or suggested.
|
||||
|
||||
Open a title and use **Delete Entry** in its header to remove a mistakenly tracked show outright. This deletes every episode of that title along with their sessions, subtitle lines, rollups and cover art, drops the words and kanji that were only seen there, and removes the card from the Library grid. Individual episodes and sessions can still be deleted on their own from the episode list and session rows. Entry deletion is refused while that title is the one currently playing.
|
||||
|
||||
|
||||
@@ -132,6 +125,34 @@ Secondary subtitle text (typically English translations) is stored alongside pri
|
||||
|
||||
The Vocabulary tab toolbar includes an **Exclusions** button for hiding words from all vocabulary views. Excluded words are stored in the immersion database, with older browser localStorage exclusions imported on first load after upgrade. They can be managed (restored or cleared) from the exclusion modal. Exclusions affect stat cards, charts, the frequency rank table, and the word list.
|
||||
|
||||
### Repeated Line Cleanup
|
||||
|
||||
Karaoke openings and animated signs are authored as one subtitle event per animation frame, all carrying the same text. Playback reports every one of those frames, so a single OP lyric could be recorded hundreds of times and dominate "Top Repeated Words".
|
||||
|
||||
Recording now collapses those runs as they happen, matching what the subtitle sidebar shows:
|
||||
|
||||
- When the active subtitle source has been parsed, its cue list has already had duplicate events and animation bursts merged. A line landing inside a surviving cue but after that cue's start is a frame the sidebar merged away, and is not recorded.
|
||||
- When no parsed cue covers the live timing, including while a subtitle source is changing or shifted, the strict metadata-free rule applies: a run of identical, contiguous lines each shorter than 0.1s stops being recorded after a few frames. Runs are tracked per line of text, so dual-line karaoke (a kanji and a romaji line frame-flipped together) collapses both lines. Ordinary repeated dialogue, and lines held for a normal beat, always record.
|
||||
|
||||
For stats recorded before this, the Vocabulary tab toolbar has a **Duplicates** button:
|
||||
|
||||
- Pick how far back to look (7 days, 30 days, 90 days, 1 year, or all time). A narrower window does less work and keeps older history untouched.
|
||||
- **Scan** reports the bursts found, the lines they added, and the word and kanji counts they inflated, without writing anything.
|
||||
- **Clean Up** applies exactly what the scan reported: each run collapses to its first line (extended to cover the run), and the removed lines' word and kanji occurrences are subtracted from the vocabulary aggregates.
|
||||
|
||||
The same thing runs from the terminal:
|
||||
|
||||
```bash
|
||||
subminer stats cleanup --duplicate-lines --dry-run --lookback-days 30
|
||||
subminer stats cleanup --duplicate-lines --lookback-days 30
|
||||
```
|
||||
|
||||
`--duplicate-lines` (short: `-d`) picks the cleanup mode, so it cannot be combined with `--vocab` or `--lifetime`, and `--dry-run` and `--lookback-days <days>` only apply to it. Omitting `--lookback-days` scans all history; the value must be at least one day.
|
||||
|
||||
The cleanup chains runs per line of text, so interleaved dual-line karaoke collapses each of its lines. It also removes the short residue the live rule stores before a run is long enough to recognize: a run one frame short of the usual minimum qualifies when every event is under the strict 0.1s bound.
|
||||
|
||||
Runs never cross a session boundary, so rewatching an episode keeps both watches. Session telemetry (watch time, lines seen, tokens seen) and the rollups derived from it are left as recorded: they are cumulative samples taken during playback, and cannot be recomputed for sessions whose raw rows have since been pruned.
|
||||
|
||||
## Retention Defaults
|
||||
|
||||
By default, SubMiner keeps all retention tables and raw data (`0` means keep all) while continuing daily/monthly rollup maintenance:
|
||||
|
||||
@@ -151,6 +151,7 @@ subminer stats -b # start background stats daemon
|
||||
| `subminer stats` | Start the stats server (opens the dashboard when `stats.autoOpenBrowser` is on) |
|
||||
| `subminer stats -b` / `-s` | Start/reuse or stop the background stats daemon |
|
||||
| `subminer stats cleanup` | Backfill vocabulary metadata and prune stale rows (`-v` vocab, `-l` lifetime summaries) |
|
||||
| `subminer stats cleanup -d` | Collapse repeated lines from typeset subs (`--dry-run`, `--lookback-days <n>`) |
|
||||
| `subminer stats rebuild` / `backfill` | Rebuild or backfill rollup data |
|
||||
| `subminer doctor` | Dependency + config + socket diagnostics (`--refresh-known-words` refreshes the known-word cache) |
|
||||
| `subminer settings` | Open the SubMiner settings window |
|
||||
|
||||
@@ -171,7 +171,9 @@ Without FFmpeg, card creation still works but audio and image fields will be emp
|
||||
|
||||
**Audio or screenshot generation hangs**
|
||||
|
||||
Media generation has a 30-second timeout (60 seconds for animated AVIF). If your video file is on a slow network mount or the codec requires software decoding, generation may time out. Try:
|
||||
Audio extraction has a 2-minute timeout. SubMiner also limits FFmpeg probing when mpv provides the selected audio stream, which avoids scanning unrelated subtitle and font-attachment streams in large MKV files. Screenshots retain a 30-second timeout, and animated AVIF uses 60 seconds.
|
||||
|
||||
If your video file is on a slow or unresponsive network mount, generation may still time out. Try:
|
||||
|
||||
- Using a local copy of the video file.
|
||||
- Reducing `ankiConnect.media.imageQuality` or switching from `avif` to `static` image type.
|
||||
|
||||
+5
-1
@@ -95,6 +95,8 @@ subminer texthooker # Texthooker-only mode (-o also opens the brow
|
||||
subminer stats -b # Start/reuse the background stats daemon
|
||||
subminer stats -s # Stop the background stats daemon
|
||||
subminer stats cleanup # Backfill vocabulary metadata, prune stale rows
|
||||
subminer stats cleanup -d --dry-run # Preview cleanup of repeated typeset subtitle lines
|
||||
subminer stats cleanup -d --lookback-days 30 # Clean only lines recorded in the last 30 days
|
||||
subminer stats rebuild # Rebuild rollup data
|
||||
subminer doctor --refresh-known-words # Refresh the known-word cache
|
||||
subminer logs -e # Export a sanitized log ZIP and print its path
|
||||
@@ -107,6 +109,8 @@ subminer app --stop # Stop the background app
|
||||
subminer --version # Print the launcher's version
|
||||
```
|
||||
|
||||
`stats cleanup` runs one mode per invocation: `-v`/`--vocab` (the default), `-l`/`--lifetime`, or `-d`/`--duplicate-lines`; explicitly selected modes cannot be combined. `--dry-run` and `--lookback-days <days>` apply to `--duplicate-lines` only and are rejected without it; `--lookback-days` must be at least one day, and leaving it off scans all history.
|
||||
|
||||
Jellyfin, cross-machine sync, and character-dictionary commands have their own sections: [Jellyfin](/jellyfin-integration), [Sync Between Machines](/launcher-script#sync-between-machines), and [Character Dictionary](/character-dictionary).
|
||||
|
||||
</details>
|
||||
@@ -137,7 +141,7 @@ SubMiner.AppImage --start --log-level debug # Verbose logging without dev mode
|
||||
SubMiner.AppImage --help # Show all options
|
||||
```
|
||||
|
||||
The remaining flags are internal or scripting-only surfaces: the `--jellyfin-*` family (login, library listing, item playback, cast announce), `--sync-cli` (the app's headless sync entrypoint that `subminer sync` proxies to), `--dictionary-candidates` / `--dictionary-select`, and `--playback-feedback <text>`. Run `SubMiner.AppImage --help` for the complete list. The previous `--open-animetosho` flag is still accepted as a deprecated alias for `--open-tsukihime`.
|
||||
The remaining flags are internal or scripting-only surfaces: the `--jellyfin-*` family (login, library listing, item playback, cast announce), `--sync-cli` (the app's headless sync entrypoint that `subminer sync` proxies to), the `--stats-cleanup-*` family that `subminer stats cleanup` forwards (`--stats-cleanup-vocab`, `--stats-cleanup-lifetime`, `--stats-cleanup-duplicate-lines`, and its `--stats-cleanup-dry-run` / `--stats-cleanup-lookback-days <days>` modifiers), `--dictionary-candidates` / `--dictionary-select`, and `--playback-feedback <text>`. Run `SubMiner.AppImage --help` for the complete list. The previous `--open-animetosho` flag is still accepted as a deprecated alias for `--open-tsukihime`.
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -25,8 +25,7 @@ Read when: you need to find the owner module for a behavior or test surface
|
||||
- Anki workflow: `src/anki-integration/`, `src/core/services/anki-jimaku*.ts`
|
||||
- Immersion tracking: `src/core/services/immersion-tracker/`
|
||||
Includes stats storage/query schema such as `imm_videos`, `imm_media_art`, and `imm_youtube_videos` for per-video and YouTube-specific library metadata.
|
||||
Library-entry identity aliases and merge recommendations are persisted alongside this schema; the stats HTTP and SPA layers only expose and present those domain decisions.
|
||||
`delete-maintenance-scheduler.ts` coalesces and serializes stats deletes; expensive deletion and summary rebuilds run in `delete-maintenance-worker-thread.ts` while the tracker queues playback writes. Each batch uses one transaction, lexical update, rollup refresh, and lifetime rebuild.
|
||||
`delete-maintenance-scheduler.ts` coalesces and serializes stats deletes; the expensive work runs in `delete-maintenance-worker-thread.ts` while the tracker queues playback writes. Each batch uses one transaction, lexical update, rollup refresh, and incremental lifetime subtraction (`planLifetimeRemovals`/`applyLifetimeRemovals` in `lifetime.ts`). Merges, moves, AniList reassignments, and `stats cleanup -l` use `repairLifetimeSummariesFromMedia` (recompute from the per-video media ledger). The full lifetime rebuild survives only as the empty-table bootstrap — anywhere else it would collapse lifetime totals to the session retention window.
|
||||
- AniList tracking + character dictionary: `src/core/services/anilist/`, `src/main/runtime/composers/anilist-*`, `src/main/character-dictionary-runtime.ts`, `src/main/character-dictionary-runtime/`
|
||||
- Jellyfin integration: `src/core/services/jellyfin*.ts`, `src/main/runtime/composers/jellyfin-*`
|
||||
- Window trackers: `src/window-trackers/`
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# Documentation Catalog
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-05-23
|
||||
Last verified: 2026-08-13
|
||||
Owner: Kyle Yasuda
|
||||
Read when: finding internal docs or checking verification status
|
||||
|
||||
@@ -17,10 +17,10 @@ Read when: finding internal docs or checking verification status
|
||||
| KB rules | `docs/knowledge-base/README.md` | active | 2026-05-23 | maintenance policy |
|
||||
| Core beliefs | `docs/knowledge-base/core-beliefs.md` | active | 2026-03-13 | agent-first principles |
|
||||
| Quality scorecard | `docs/knowledge-base/quality.md` | active | 2026-03-13 | quality grades and gaps |
|
||||
| Workflow index | `docs/workflow/README.md` | active | 2026-05-23 | execution map |
|
||||
| Workflow index | `docs/workflow/README.md` | active | 2026-08-13 | execution map |
|
||||
| Planning guide | `docs/workflow/planning.md` | active | 2026-05-23 | lightweight vs execution plans |
|
||||
| Agent plugins | `docs/workflow/agent-plugins.md` | active | 2026-05-23 | repo-local agent workflow plugin ownership |
|
||||
| Verification guide | `docs/workflow/verification.md` | active | 2026-05-23 | maintained verification lanes |
|
||||
| Agent skills | `docs/workflow/agent-skills.md` | active | 2026-08-13 | repo-local workflow skill ownership |
|
||||
| Verification guide | `docs/workflow/verification.md` | active | 2026-08-13 | maintained verification lanes |
|
||||
| Release guide | `docs/RELEASING.md` | active | 2026-05-23 | release checklist |
|
||||
|
||||
## Update Rules
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,184 +0,0 @@
|
||||
# Library Summary Replaces Per-Day Trends — Design
|
||||
|
||||
**Status:** Draft
|
||||
**Date:** 2026-04-09
|
||||
**Scope:** `stats/` frontend, `src/core/services/immersion-tracker/query-trends.ts` backend
|
||||
|
||||
## Problem
|
||||
|
||||
The "Library — Per Day" section on the stats Trends tab (`stats/src/components/trends/TrendsTab.tsx:224-254`) renders six stacked-area charts — Videos, Watch Time, Cards, Words, Lookups, and Lookups/100w, each broken down per title per day.
|
||||
|
||||
In practice these charts are not useful:
|
||||
|
||||
- Most titles only have activity on one or two days in a window, so they render as isolated bumps on a noisy baseline.
|
||||
- Stacking 7+ titles with mostly-zero days makes individual lines hard to follow.
|
||||
- The top "Activity" and "Period Trends" sections already answer "what am I doing per day" globally.
|
||||
- The "Library — Cumulative" section directly below already answers "which titles am I progressing through" with less noise.
|
||||
|
||||
The per-day section occupies significant vertical space without carrying its weight, and the user has confirmed it should be replaced.
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the six per-day stacked charts with a single "Library — Summary" section that surfaces per-title aggregate statistics over the selected date range. The new view should make it trivially easy to answer: "For the selected window, which titles am I spending time on, how much mining output have they produced, and how efficient is my lookup rate on each?"
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changing the "Library — Cumulative" section (stays as-is).
|
||||
- Changing the "Activity", "Period Trends", or "Patterns" sections.
|
||||
- Adding a new API endpoint — the existing dashboard endpoint is extended in place.
|
||||
- Renaming internal `anime*` data-model identifiers (`animeId`, `imm_anime`, etc.). Those stay per the convention established in `c5e778d7`; only new fields/types/user-visible strings use generic "title"/"library" wording.
|
||||
- Supporting a true all-time library view on the Trends tab. If that's ever wanted, it belongs on a different tab.
|
||||
|
||||
## Solution Overview
|
||||
|
||||
Delete the "Library — Per Day" section. In its place, add "Library — Summary", composed of:
|
||||
|
||||
1. A horizontal-bar leaderboard chart of watch time per title (top 10, descending).
|
||||
2. A sortable table of every title with activity in the selected window, with columns: Title, Watch Time, Videos, Sessions, Cards, Words, Lookups, Lookups/100w, Date Range.
|
||||
|
||||
Both controls are scoped to the top-of-page date range selector. The existing shared Anime Visibility filter continues to work — it now gates Summary + Cumulative instead of Per-Day + Cumulative.
|
||||
|
||||
## Backend
|
||||
|
||||
### New type
|
||||
|
||||
Add to `stats/src/types/stats.ts` and the backend query module:
|
||||
|
||||
```ts
|
||||
type LibrarySummaryRow = {
|
||||
title: string; // display title — anime series, YouTube video title, etc.
|
||||
watchTimeMin: number; // sum(total_active_min) across the window
|
||||
videos: number; // distinct video_id count
|
||||
sessions: number; // session count from imm_sessions
|
||||
cards: number; // sum(total_cards)
|
||||
words: number; // sum(total_tokens_seen)
|
||||
lookups: number; // sum(lookup_count) from imm_sessions
|
||||
lookupsPerHundred: number | null; // lookups / words * 100, null when words == 0
|
||||
firstWatched: number; // min(rollup_day) as epoch day, within the window
|
||||
lastWatched: number; // max(rollup_day) as epoch day, within the window
|
||||
};
|
||||
```
|
||||
|
||||
### Query changes in `src/core/services/immersion-tracker/query-trends.ts`
|
||||
|
||||
- Add `librarySummary: LibrarySummaryRow[]` to `TrendsDashboardQueryResult`.
|
||||
- Populate it from a single aggregating query over `imm_daily_rollups` joined to `imm_videos` → `imm_anime`, filtered by `rollup_day` within the selected window. Session count and lookup count come from `imm_sessions` aggregated by `video_id` and then grouped by the parent library entry. Use a single query (or at most two joined/unioned) — no N+1.
|
||||
- `imm_anime` is the generic library-grouping table; anime series, YouTube videos, and yt-dlp imports all land there. The internal table name stays `imm_anime`; only the new field uses generic naming.
|
||||
- Return rows pre-sorted by `watchTimeMin` descending so the leaderboard is zero-cost and the table default sort matches.
|
||||
- Emit `lookupsPerHundred: null` when `words == 0`.
|
||||
|
||||
### Removed from API response
|
||||
|
||||
Drop the entire `animePerDay` field from `TrendsDashboardQueryResult` (both backend in `src/core/services/immersion-tracker/query-trends.ts` and frontend in `stats/src/types/stats.ts`).
|
||||
|
||||
Internally, the existing helpers (`buildPerAnimeFromDailyRollups`, `buildEpisodesPerAnimeFromDailyRollups`) are still used as intermediates to build `animeCumulative.*` via `buildCumulativePerAnime`. Keep those helpers — just scope their output to local variables inside `getTrendsDashboard` instead of exposing them on the response. The `buildPerAnimeFromSessions` call for lookups and the `buildLookupsPerHundredPerAnime` helper become unused and can be deleted.
|
||||
|
||||
Before removing `animePerDay` from the frontend type, verify no other file under `stats/src/` references it. Based on current inspection, only `TrendsTab.tsx` and `stats/src/types/stats.ts` touch it.
|
||||
|
||||
## Frontend
|
||||
|
||||
### New component: `stats/src/components/trends/LibrarySummarySection.tsx`
|
||||
|
||||
Owns the header, leaderboard chart, visibility-filtered data, and the table. Keeps `TrendsTab.tsx` from growing. Component props: `{ rows: LibrarySummaryRow[]; hiddenTitles: ReadonlySet<string>; windowStart: Date; windowEnd: Date }`.
|
||||
|
||||
Internal state: `useState<{ column: ColumnId; direction: 'asc' | 'desc' }>` for sort, defaulting to `{ column: 'watchTimeMin', direction: 'desc' }`.
|
||||
|
||||
### Layout
|
||||
|
||||
Replaces `TrendsTab.tsx:224-254`:
|
||||
|
||||
```
|
||||
[SectionHeader: "Library — Summary"]
|
||||
[AnimeVisibilityFilter — unchanged, shared with Cumulative below]
|
||||
[Card, col-span-full: Leaderboard — horizontal bar chart, ~260px tall]
|
||||
[Card, col-span-full: Sortable table, auto height up to ~480px with internal scroll]
|
||||
```
|
||||
|
||||
Both cards use the existing chart/card wrapper styling.
|
||||
|
||||
### Leaderboard chart
|
||||
|
||||
- Recharts horizontal bar chart (matches the rest of the page — existing charts use `recharts`, not ECharts).
|
||||
- Top 10 titles by watch time. If fewer titles have activity, render what's there.
|
||||
- Y-axis: title (category), truncated with ellipsis at container width; full title visible in the Recharts tooltip.
|
||||
- X-axis: minutes (number).
|
||||
- Use `layout="vertical"` with `YAxis dataKey="title" type="category"` and `XAxis type="number"`.
|
||||
- Single series color: `#8aadf4` (matching the existing Watch Time color).
|
||||
- Reuse `CHART_DEFAULTS`, `CHART_THEME`, `TOOLTIP_CONTENT_STYLE` from `stats/src/lib/chart-theme.ts` so theming matches the rest of the dashboard.
|
||||
- Chart order is fixed at watch-time desc regardless of table sort — the leaderboard's meaning is fixed.
|
||||
|
||||
### Table
|
||||
|
||||
- Plain HTML `<table>` with Tailwind classes. No new deps.
|
||||
- Columns, in order:
|
||||
1. **Title** — left-aligned, sticky, truncated with ellipsis, full title on hover.
|
||||
2. **Watch Time** — formatted `Xh Ym` when ≥60 min, else `Xm`.
|
||||
3. **Videos** — integer.
|
||||
4. **Sessions** — integer.
|
||||
5. **Cards** — integer.
|
||||
6. **Words** — integer.
|
||||
7. **Lookups** — integer.
|
||||
8. **Lookups/100w** — one decimal place, `—` when null.
|
||||
9. **Date Range** — `Mon D → Mon D` using the title's `firstWatched` / `lastWatched` within the window.
|
||||
- Click a column header to sort; click again to reverse. Visual arrow on the active column.
|
||||
- Numeric columns right-aligned.
|
||||
- Null `lookupsPerHundred` sorts as the lowest value in both directions (consistent with "no data").
|
||||
- Row hover highlight; no row click action (read-only view).
|
||||
- Empty state: "No library activity in the selected window."
|
||||
|
||||
### Visibility filter integration
|
||||
|
||||
Hiding a title via `AnimeVisibilityFilter` removes it from both the leaderboard and the table. The filter's set of available titles is built from the union of titles that appear in `librarySummary` and the existing `animeCumulative.*` arrays (matches current behavior in `buildAnimeVisibilityOptions`).
|
||||
|
||||
### `TrendsTab.tsx` changes
|
||||
|
||||
- Remove the `filteredEpisodesPerAnime`, `filteredWatchTimePerAnime`, `filteredCardsPerAnime`, `filteredWordsPerAnime`, `filteredLookupsPerAnime`, `filteredLookupsPerHundredPerAnime` locals.
|
||||
- Remove the six `<StackedTrendChart>` calls in the "Library — Per Day" section.
|
||||
- Remove the `<SectionHeader>Library — Per Day</SectionHeader>` and the `<AnimeVisibilityFilter>` from that position.
|
||||
- Insert `<SectionHeader>Library — Summary</SectionHeader>` + `<AnimeVisibilityFilter>` + `<LibrarySummarySection>` in the same place.
|
||||
- Update `buildAnimeVisibilityOptions` input to use `librarySummary` titles instead of the six dropped `animePerDay.*` arrays.
|
||||
|
||||
## Data flow
|
||||
|
||||
1. `useTrends(range, groupBy)` calls `/api/stats/trends/dashboard`.
|
||||
2. Response now includes `librarySummary` (sorted by watch time desc).
|
||||
3. `TrendsTab` holds the shared `hiddenAnime` set (unchanged).
|
||||
4. `LibrarySummarySection` receives `librarySummary` + `hiddenAnime`, filters out hidden rows, renders the leaderboard from the top-10 slice of the filtered list, renders the table from the filtered list with local sort state applied.
|
||||
5. Date-range selector changes trigger a new fetch; `groupBy` toggle does not affect the summary section (it's always window-total).
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **No activity in window:** Section renders header + empty-state card. Leaderboard card hidden. Visibility filter hidden.
|
||||
- **One title only:** Leaderboard renders a single bar; table renders one row. No special-casing.
|
||||
- **Title with zero words but non-zero lookups:** `lookupsPerHundred` is `null`, rendered as `—`. Sort treats null as lowest.
|
||||
- **Title with zero cards/lookups/words but non-zero watch time:** Normal zero rendering, still shown.
|
||||
- **Very long titles:** Ellipsis in chart y-axis labels and table title column; full title in `title` attribute / ECharts tooltip.
|
||||
- **Mixed sources (anime + YouTube):** No special case — both land in `imm_anime` and are grouped uniformly.
|
||||
|
||||
## Testing
|
||||
|
||||
### Backend (`query-trends.ts`)
|
||||
|
||||
New unit tests, following the existing pattern:
|
||||
|
||||
1. Empty window returns `librarySummary: []`.
|
||||
2. Single title with a few rollups: all aggregates are correct; `firstWatched`/`lastWatched` match the bounding days within the window.
|
||||
3. Multiple titles: rows returned sorted by watch time desc.
|
||||
4. Mixed sources (anime-style + YouTube-style entries in `imm_anime`): both appear in the summary with their own aggregates.
|
||||
5. Title with `words == 0`: `lookupsPerHundred` is `null`.
|
||||
6. Date range excludes some rollups: excluded rollups are not counted; `firstWatched`/`lastWatched` reflect only within-window activity.
|
||||
7. `sessions` and `lookups` come from `imm_sessions`, not `imm_daily_rollups`, and are correctly attributed to the parent library entry.
|
||||
|
||||
### Frontend
|
||||
|
||||
- Existing Trends tab smoke test should continue to pass after wiring.
|
||||
- Optional: a targeted render test for `LibrarySummarySection` (empty state, single title, sort toggle, visibility filter interaction). Not required for merge if the smoke test exercises the happy path.
|
||||
|
||||
## Release / docs
|
||||
|
||||
- One fragment in `changes/*.md` summarizing the replacement.
|
||||
- No user-facing docs (`docs-site/`) changes unless the per-day section was documented there — verify during implementation.
|
||||
|
||||
## Open items
|
||||
|
||||
None.
|
||||
@@ -1,347 +0,0 @@
|
||||
# Stats Dashboard Feedback Pass — Design
|
||||
|
||||
Date: 2026-04-09
|
||||
Scope: Stats dashboard UX follow-ups from user feedback (items 1–7).
|
||||
Delivery: **Single PR**, broken into logically scoped commits.
|
||||
|
||||
## Goals
|
||||
|
||||
Address seven concrete pieces of feedback against the Statistics menu:
|
||||
|
||||
1. Library — collapse episodes behind a per-series dropdown.
|
||||
2. Sessions — roll up multiple sessions of the same episode within a day.
|
||||
3. Trends — add a 365d range option.
|
||||
4. Library — delete an episode (video) from its detail view.
|
||||
5. Vocabulary — tighten spacing between word and reading in the Top 50 table.
|
||||
6. Episode detail — hide cards whose Anki notes have been deleted.
|
||||
7. Trend/watch charts — add gridlines, fix tick legibility, unify theming.
|
||||
|
||||
Out of scope for this pass: English-token ingestion cleanup and Overview stat-card drill-downs (feedback items 8 and 9). Those require a larger design decision and a migration respectively.
|
||||
|
||||
## Files touched (inventory)
|
||||
|
||||
Dashboard (`stats/src/`):
|
||||
- `components/library/LibraryTab.tsx` — collapsible groups (item 1).
|
||||
- `components/library/MediaDetailView.tsx`, `components/library/MediaHeader.tsx` — delete-episode action (item 4).
|
||||
- `components/sessions/SessionsTab.tsx`, `components/library/MediaSessionList.tsx` — episode rollup (item 2).
|
||||
- `components/trends/DateRangeSelector.tsx`, `hooks/useTrends.ts`, `lib/api-client.ts`, `lib/api-client.test.ts` — 365d (item 3).
|
||||
- `components/vocabulary/FrequencyRankTable.tsx` — word/reading column collapse (item 5).
|
||||
- `components/anime/EpisodeDetail.tsx` — filter deleted Anki cards (item 6).
|
||||
- `components/trends/TrendChart.tsx`, `components/trends/StackedTrendChart.tsx`, `components/overview/WatchTimeChart.tsx`, `lib/chart-theme.ts` — chart clarity (item 7).
|
||||
- New file: `stats/src/lib/session-grouping.ts` + `session-grouping.test.ts`.
|
||||
|
||||
Backend (`src/core/services/`):
|
||||
- `immersion-tracker/query-trends.ts` — extend `TrendRange` and `TREND_DAY_LIMITS` (item 3).
|
||||
- `immersion-tracker/__tests__/query.test.ts` — 365d coverage (item 3).
|
||||
- `stats-server.ts` — passthrough if range validation lives here (check before editing).
|
||||
- `__tests__/stats-server.test.ts` — 365d coverage (item 3).
|
||||
|
||||
## Commit plan
|
||||
|
||||
One PR, one feature per commit. Order picks low-risk mechanical changes first so failures in later commits don't block merging of earlier ones.
|
||||
|
||||
1. `feat(stats): add 365d range to trends dashboard` (item 3)
|
||||
2. `fix(stats): tighten word/reading column in Top 50 table` (item 5)
|
||||
3. `fix(stats): hide cards deleted from Anki in episode detail` (item 6)
|
||||
4. `feat(stats): delete episode from library detail view` (item 4)
|
||||
5. `feat(stats): collapsible series groups in library` (item 1)
|
||||
6. `feat(stats): roll up same-episode sessions within a day` (item 2)
|
||||
7. `feat(stats): gridlines and unified theme for trend charts` (item 7)
|
||||
|
||||
Each commit must pass `bun run typecheck`, `bun run test:fast`, and any change-specific checks listed below.
|
||||
|
||||
---
|
||||
|
||||
## Item 1 — Library collapsible series groups
|
||||
|
||||
### Current behavior
|
||||
|
||||
`LibraryTab.tsx` groups media via `groupMediaLibraryItems` and always renders the full grid of `MediaCard`s beneath each group header.
|
||||
|
||||
### Target behavior
|
||||
|
||||
Each group header becomes clickable. Groups with `items.length > 1` default to **collapsed**; single-video groups stay expanded (collapsing them would be visual noise).
|
||||
|
||||
### Implementation
|
||||
|
||||
- State: `const [collapsedGroups, setCollapsedGroups] = useState<Set<string>>(...)`. Initialize from `grouped` where `items.length > 1`.
|
||||
- Toggle helper: `toggleGroup(key: string)` adds/removes from the set.
|
||||
- Group header: wrap in a `<button>` with `aria-expanded` and a chevron icon (`▶`/`▼`). Keep the existing cover + title + subtitle layout inside the button.
|
||||
- Children grid is conditionally rendered on `!collapsedGroups.has(group.key)`.
|
||||
- Header summary (`N videos · duration · cards`) stays visible in both states so collapsed groups remain informative.
|
||||
|
||||
### Tests
|
||||
|
||||
- New `LibraryTab.test.tsx` (if not already present — check first) covering:
|
||||
- Multi-video group renders collapsed on first mount.
|
||||
- Single-video group renders expanded on first mount.
|
||||
- Clicking the header toggles visibility.
|
||||
- Header summary is visible in both states.
|
||||
|
||||
---
|
||||
|
||||
## Item 2 — Sessions episode rollup within a day
|
||||
|
||||
### Current behavior
|
||||
|
||||
`SessionsTab.tsx:10-24` groups sessions by day label only (`formatSessionDayLabel(startedAtMs)`). Multiple sessions of the same episode on the same day show as independent rows. `MediaSessionList.tsx` has the same problem inside the library detail view.
|
||||
|
||||
### Target behavior
|
||||
|
||||
Within each day, sessions with the same `videoId` collapse into one parent row showing combined totals. A chevron reveals the individual sessions. Single-session buckets render flat (no pointless nesting).
|
||||
|
||||
### Implementation
|
||||
|
||||
- New helper in `stats/src/lib/session-grouping.ts`:
|
||||
```ts
|
||||
export interface SessionBucket {
|
||||
key: string; // videoId as string, or `s-${sessionId}` for singletons
|
||||
videoId: number | null;
|
||||
sessions: SessionSummary[];
|
||||
totalActiveMs: number;
|
||||
totalCardsMined: number;
|
||||
representativeSession: SessionSummary; // most recent, for header display
|
||||
}
|
||||
export function groupSessionsByVideo(sessions: SessionSummary[]): SessionBucket[];
|
||||
```
|
||||
Sessions missing a `videoId` become singleton buckets.
|
||||
|
||||
- `SessionsTab.tsx`: after day grouping, pipe each `daySessions` through `groupSessionsByVideo`. Render each bucket:
|
||||
- `sessions.length === 1`: existing `SessionRow` behavior, unchanged.
|
||||
- `sessions.length >= 2`: render a **bucket row** that looks like `SessionRow` but shows combined totals and session count (e.g. `3 sessions · 1h 24m · 12 cards`). Chevron state stored in a second `Set<string>` on bucket key. Expanded buckets render the child `SessionRow`s indented (`pl-8`) beneath the header.
|
||||
- `MediaSessionList.tsx`: within the media detail view, a single video's sessions are all the same `videoId` by definition — grouping here is by day only, and within a day multiple sessions render nested under a day header. Re-use the same visual pattern; factor the bucket row into a shared `SessionBucketRow` component.
|
||||
|
||||
### Delete semantics
|
||||
|
||||
- Deleting a bucket header offers "Delete all N sessions in this group" (reuse `confirmDayGroupDelete` pattern with a bucket-specific message, or add `confirmBucketDelete`).
|
||||
- Deleting an individual session from inside an expanded bucket keeps the existing single-delete flow.
|
||||
|
||||
### Tests
|
||||
|
||||
- `session-grouping.test.ts`:
|
||||
- Empty input → empty output.
|
||||
- All unique videos → N singleton buckets.
|
||||
- Two sessions same videoId → one bucket with correct totals and representative (most recent start time).
|
||||
- Missing videoId → singleton bucket keyed by sessionId.
|
||||
- `SessionsTab.test.tsx` (extend or add) verifying the rendered bucket rows expand/collapse and delete hooks fire with the right ID set.
|
||||
|
||||
---
|
||||
|
||||
## Item 3 — 365d trends range
|
||||
|
||||
### Backend
|
||||
|
||||
`src/core/services/immersion-tracker/query-trends.ts`:
|
||||
- `type TrendRange = '7d' | '30d' | '90d' | '365d' | 'all';`
|
||||
- Add `'365d': 365` to `TREND_DAY_LIMITS`.
|
||||
- `getTrendDayLimit` picks up the new key automatically because of the `Exclude<TrendRange, 'all'>` generic.
|
||||
|
||||
`src/core/services/stats-server.ts`:
|
||||
- Search for any hardcoded range validation (e.g. allow-list in the trends route handler) and extend it.
|
||||
|
||||
### Frontend
|
||||
|
||||
- `hooks/useTrends.ts`: widen the `TimeRange` union.
|
||||
- `components/trends/DateRangeSelector.tsx`: add `'365d'` to the options list. Display label stays as `365d`.
|
||||
- `lib/api-client.ts` / `api-client.test.ts`: if the client validates ranges, add `365d`.
|
||||
|
||||
### Tests
|
||||
|
||||
- `query.test.ts`: extend the existing range table to cover `365d` returning 365 days of data.
|
||||
- `stats-server.test.ts`: ensure the route accepts `range=365d`.
|
||||
- `api-client.test.ts`: ensure the client emits the new range.
|
||||
|
||||
### Change-specific checks
|
||||
|
||||
- `bun run test:config` is not required here (no schema/defaults change).
|
||||
- Run `bun run typecheck` + `bun run test:fast`.
|
||||
|
||||
---
|
||||
|
||||
## Item 4 — Delete episode from library detail
|
||||
|
||||
### Current behavior
|
||||
|
||||
`MediaDetailView.tsx` provides session-level delete only. The backend `deleteVideo` exists (`query-maintenance.ts:509`), the API is exposed at `stats-server.ts:559`, and `api-client.deleteVideo` is already wired (`stats/src/lib/api-client.ts:146`). `EpisodeList.tsx:46` already uses it from the anime tab.
|
||||
|
||||
### Target behavior
|
||||
|
||||
A "Delete Episode" action in `MediaHeader` (top-right, small, `text-ctp-red`), gated by `confirmEpisodeDelete(title)`. On success, call `onBack()` and make sure the parent `LibraryTab` refetches.
|
||||
|
||||
### Implementation
|
||||
|
||||
- Add an `onDeleteEpisode?: () => void` prop to `MediaHeader` and render the button only if provided.
|
||||
- In `MediaDetailView`:
|
||||
- New handler `handleDeleteEpisode` that calls `apiClient.deleteVideo(videoId)`, then `onBack()`.
|
||||
- Reuse `confirmEpisodeDelete` from `stats/src/lib/delete-confirm.ts`.
|
||||
- In `LibraryTab`:
|
||||
- `useMediaLibrary` returns fresh data on mount. The simplest fix: pass a `refresh` function from the hook (extend the hook if it doesn't already expose one) and call it when the detail view signals back.
|
||||
- Alternative: force a remount by incrementing a `libraryVersion` key on the library list. Prefer `refresh` for clarity.
|
||||
|
||||
### Tests
|
||||
|
||||
- Extend the existing `MediaDetailView.test.tsx`: mock `apiClient.deleteVideo`, click the new button, confirm `onBack` fires after success.
|
||||
- `useMediaLibrary.test.ts`: if we add a `refresh` method, cover it.
|
||||
|
||||
---
|
||||
|
||||
## Item 5 — Vocabulary word/reading column collapse
|
||||
|
||||
### Current behavior
|
||||
|
||||
`FrequencyRankTable.tsx:110-144` uses a 5-column table: `Rank | Word | Reading | POS | Seen`. Word and Reading are auto-sized, producing a large gap.
|
||||
|
||||
### Target behavior
|
||||
|
||||
Merge Word + Reading into a single column titled "Word". Reading sits immediately after the headword in a muted, smaller style.
|
||||
|
||||
### Implementation
|
||||
|
||||
- Drop the `<th>Reading</th>` header and cell.
|
||||
- Word cell becomes:
|
||||
```tsx
|
||||
<td className="py-1.5 pr-3">
|
||||
<span className="text-ctp-text font-medium">{w.headword}</span>
|
||||
{reading && (
|
||||
<span className="text-ctp-subtext0 text-xs ml-1.5">
|
||||
【{reading}】
|
||||
</span>
|
||||
)}
|
||||
</td>
|
||||
```
|
||||
where `reading = fullReading(w.headword, w.reading)` and differs from `headword`.
|
||||
- Keep `fullReading` import from `reading-utils`.
|
||||
|
||||
### Tests
|
||||
|
||||
- Extend `FrequencyRankTable.test.tsx` (if present — otherwise add a focused test) to assert:
|
||||
- Headword renders.
|
||||
- Reading renders when different from headword.
|
||||
- Reading does not render when equal to headword.
|
||||
|
||||
---
|
||||
|
||||
## Item 6 — Hide Anki-deleted cards in Cards Mined
|
||||
|
||||
### Current behavior
|
||||
|
||||
`EpisodeDetail.tsx:109-147` iterates `cardEvents`, fetches note info via `ankiNotesInfo(allNoteIds)`, and for each `noteId` renders a row even if no matching `info` came back — the user sees an empty word with an "Open in Anki" button that leads nowhere.
|
||||
|
||||
### Target behavior
|
||||
|
||||
After `ankiNotesInfo` resolves:
|
||||
- Drop `noteId`s that are not in the resolved map.
|
||||
- Drop `cardEvents` whose `noteIds` list was non-empty but is now empty after filtering.
|
||||
- Card events with a positive `cardsDelta` but no `noteIds` (legacy rollup path) still render as `+N cards` — we have no way to cross-reference them, so leave them alone.
|
||||
|
||||
### Implementation
|
||||
|
||||
- Compute `filteredCardEvents` as a `useMemo` depending on `data.cardEvents` and `noteInfos`.
|
||||
- Iterate `filteredCardEvents` instead of `cardEvents` in the render.
|
||||
- Surface a subtle note (optional, muted) "N cards hidden (deleted from Anki)" at the end of the list if any were filtered — helps the user understand why counts here diverge from session totals. Final decision on the note can be made at PR review; default: **show it**.
|
||||
|
||||
### Tests
|
||||
|
||||
- Add a test in `EpisodeDetail.test.tsx` (add the file if not present) that stubs `ankiNotesInfo` to return only a subset of notes and verifies the missing ones are not rendered.
|
||||
|
||||
### Other call sites
|
||||
|
||||
- Grep so far shows `ankiNotesInfo` is only used in `EpisodeDetail.tsx`. Re-verify before landing the commit; if another call site appears, apply the same filter.
|
||||
|
||||
---
|
||||
|
||||
## Item 7 — Trend/watch chart clarity pass
|
||||
|
||||
### Current behavior
|
||||
|
||||
`TrendChart.tsx`, `StackedTrendChart.tsx`, and `WatchTimeChart.tsx` render Recharts components with:
|
||||
- No `CartesianGrid` → no horizontal reference lines.
|
||||
- 9px axis ticks → borderline unreadable.
|
||||
- Height 120 → cramped.
|
||||
- Tooltip uses raw labels (`04/04` etc.).
|
||||
- No shared theme object; each chart redefines colors and tooltip styles inline.
|
||||
|
||||
`stats/src/lib/chart-theme.ts` already exists and currently exports a single `CHART_THEME` constant with tick/tooltip colors and `barFill`. It will be extended, not replaced, to preserve existing consumers.
|
||||
|
||||
### Target behavior
|
||||
|
||||
All three charts share a theme, have horizontal gridlines, readable ticks, and sensible tooltips.
|
||||
|
||||
### Implementation
|
||||
|
||||
Extend `stats/src/lib/chart-theme.ts` with the additional shared defaults (keeping the existing `CHART_THEME` export intact so current consumers don't break):
|
||||
```ts
|
||||
export const CHART_THEME = {
|
||||
tick: '#a5adcb',
|
||||
tooltipBg: '#363a4f',
|
||||
tooltipBorder: '#494d64',
|
||||
tooltipText: '#cad3f5',
|
||||
tooltipLabel: '#b8c0e0',
|
||||
barFill: '#8aadf4',
|
||||
grid: '#494d64',
|
||||
axisLine: '#494d64',
|
||||
} as const;
|
||||
|
||||
export const CHART_DEFAULTS = {
|
||||
height: 160,
|
||||
tickFontSize: 11,
|
||||
margin: { top: 8, right: 8, bottom: 0, left: 0 },
|
||||
grid: { strokeDasharray: '3 3', vertical: false },
|
||||
} as const;
|
||||
|
||||
export const TOOLTIP_CONTENT_STYLE = {
|
||||
background: CHART_THEME.tooltipBg,
|
||||
border: `1px solid ${CHART_THEME.tooltipBorder}`,
|
||||
borderRadius: 6,
|
||||
color: CHART_THEME.tooltipText,
|
||||
fontSize: 12,
|
||||
};
|
||||
```
|
||||
|
||||
Apply to each chart:
|
||||
- Import `CartesianGrid` from recharts.
|
||||
- Insert `<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />` inside each chart container.
|
||||
- `<XAxis tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }} />` and equivalent `YAxis`.
|
||||
- `YAxis` gains `axisLine={{ stroke: CHART_THEME.axisLine }}`.
|
||||
- `ResponsiveContainer` height changes from 120 → `CHART_DEFAULTS.height`.
|
||||
- `Tooltip` `contentStyle` uses `TOOLTIP_CONTENT_STYLE`, and charts pass a `labelFormatter` when the label is a date key (e.g. show `Fri Apr 4`).
|
||||
|
||||
### Unit formatters
|
||||
|
||||
- `TrendChart` already accepts a `formatter` prop — extend usage sites to pass unit-aware formatters where they aren't already (`formatDuration`, `formatNumber`, etc.).
|
||||
|
||||
### Tests
|
||||
|
||||
- `chart-theme.test.ts` (if present — otherwise add a trivial snapshot to keep the shape stable).
|
||||
- `TrendChart` snapshot/render tests: no regression, gridline element present.
|
||||
|
||||
---
|
||||
|
||||
## Verification gate
|
||||
|
||||
Before requesting code review, run:
|
||||
|
||||
```
|
||||
bun run typecheck
|
||||
bun run test:fast
|
||||
bun run test:env
|
||||
bun run test:runtime:compat # dist-sensitive check for the charts
|
||||
bun run build
|
||||
bun run test:smoke:dist
|
||||
```
|
||||
|
||||
No docs-site changes are planned in this spec; if `docs-site/` ends up touched (e.g. screenshots), also run `bun run docs:test` and `bun run docs:build`.
|
||||
|
||||
No config schema changes → `bun run test:config` and `bun run generate:config-example` are not required.
|
||||
|
||||
## Risks and open questions
|
||||
|
||||
- **MediaDetailView refresh**: `useMediaLibrary` may not expose a `refresh` function. If it doesn't, the simplest path is adding one; the alternative (keying a remount) works but is harder to test. Decide during implementation.
|
||||
- **Session bucket delete UX**: "Delete all N sessions in this group" is powerful. The copy must make it clear the underlying sessions are being removed, not just the grouping. Reuse `confirmBucketDelete` wording from existing confirm helpers if possible.
|
||||
- **Anki-deleted-cards hidden notice**: Showing a subtle "N cards hidden" footer is a call that can be made at PR review.
|
||||
- **Bucket delete helper**: `confirmBucketDelete` does not currently exist in `delete-confirm.ts`. Implementation either adds it or reuses `confirmDayGroupDelete` with bucket-specific wording — decide during the session-rollup commit.
|
||||
|
||||
## Changelog entry
|
||||
|
||||
User-visible PR → needs a fragment under `changes/*.md`. Suggested title:
|
||||
`Stats dashboard: collapsible series, session rollups, 365d trends, chart polish, episode delete.`
|
||||
@@ -3,7 +3,7 @@
|
||||
# Workflow
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-05-23
|
||||
Last verified: 2026-08-13
|
||||
Owner: Kyle Yasuda
|
||||
Read when: planning or executing nontrivial work in this repo
|
||||
|
||||
@@ -13,7 +13,7 @@ This section is the internal workflow map for contributors and agents.
|
||||
|
||||
- [Planning](./planning.md) - when to write a lightweight plan vs a full execution plan
|
||||
- [Verification](./verification.md) - maintained test/build lanes and handoff gate
|
||||
- [Agent Plugins](./agent-plugins.md) - repo-local plugin ownership for agent workflow skills
|
||||
- [Agent Skills](./agent-skills.md) - repo-local workflow skill ownership
|
||||
- [Release Guide](../RELEASING.md) - tagged release workflow
|
||||
|
||||
## Default Flow
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
<!-- read_when: using or modifying repo-local agent plugins -->
|
||||
|
||||
# Agent Plugins
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-05-23
|
||||
Owner: Kyle Yasuda
|
||||
Read when: packaging or migrating repo-local agent workflow skills into plugins
|
||||
|
||||
## SubMiner Workflow Plugin
|
||||
|
||||
- Canonical plugin path: `plugins/subminer-workflow/`
|
||||
- Marketplace catalog: `.agents/plugins/marketplace.json`
|
||||
- Canonical skill sources:
|
||||
- `plugins/subminer-workflow/skills/subminer-scrum-master/`
|
||||
- `plugins/subminer-workflow/skills/subminer-change-verification/`
|
||||
|
||||
## Migration Rule
|
||||
|
||||
- Plugin-owned skills are the source of truth.
|
||||
- `.agents/skills/subminer-*` remain only as compatibility shims.
|
||||
- Existing script entrypoints under `.agents/skills/subminer-change-verification/scripts/` stay as wrappers so historical commands do not break.
|
||||
|
||||
## Verification
|
||||
|
||||
- For plugin/docs-only changes, start with `bun run test:docs:kb`.
|
||||
- Use the plugin-owned verifier when the change crosses from docs into scripts or workflow logic.
|
||||
@@ -0,0 +1,31 @@
|
||||
<!-- read_when: using or modifying repo-local agent skills -->
|
||||
|
||||
# Agent Skills
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-08-13
|
||||
Owner: Kyle Yasuda
|
||||
Read when: using, adding, or changing a repo-local agent workflow skill
|
||||
|
||||
## Canonical Skills
|
||||
|
||||
- `.agents/skills/subminer-change-verification/`
|
||||
- Selects the cheapest sufficient repo-native verification lane.
|
||||
- Defers command ownership to `package.json` and `docs/workflow/verification.md`.
|
||||
|
||||
Repo-local workflows stay as standalone skills. Do not add plugin packaging, marketplace metadata, or compatibility shims unless the workflow is intentionally being distributed beyond this repository.
|
||||
|
||||
## Rules
|
||||
|
||||
- Keep each skill focused on one repeatable repository task.
|
||||
- Prefer instructions over helper scripts unless deterministic tooling provides clear value beyond existing package commands.
|
||||
- Keep trigger descriptions narrow enough to avoid invoking skills for unrelated requests.
|
||||
- Update this page and the documentation catalog when skill ownership changes.
|
||||
|
||||
## Verification
|
||||
|
||||
For skill or internal workflow documentation changes, run:
|
||||
|
||||
```bash
|
||||
bun run test:docs:kb
|
||||
```
|
||||
@@ -3,14 +3,14 @@
|
||||
# Verification
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-07-06
|
||||
Last verified: 2026-08-13
|
||||
Owner: Kyle Yasuda
|
||||
Read when: selecting the right verification lane for a change
|
||||
|
||||
## Lane Infrastructure
|
||||
|
||||
- Lane membership is defined once in `scripts/test-lanes.ts` and discovered by
|
||||
directory — new test files join their lane automatically; never hand-list test
|
||||
directory, so new test files join their lane automatically; never hand-list test
|
||||
files in `package.json`.
|
||||
- `scripts/run-test-lane.mjs` runs each test file in its own `bun test` process
|
||||
(per-file isolation with a wall timeout) so a hanging test or leaked global in
|
||||
@@ -43,8 +43,8 @@ bun run docs:build
|
||||
|
||||
## Cheap-First Lane Selection
|
||||
|
||||
- Docs-only boundary/content changes: `bun run docs:test`, `bun run docs:build`
|
||||
- Internal KB / `AGENTS.md` changes: `bun run test:docs:kb`
|
||||
- User-facing `docs-site/` changes: `bun run docs:test`, `bun run docs:build`
|
||||
- Internal KB, `AGENTS.md`, or `.agents/skills/**` changes: `bun run test:docs:kb`
|
||||
- Config/schema/defaults: `bun run test:config`, then `bun run generate:config-example` if template/defaults changed
|
||||
- Launcher/plugin: `bun run test:launcher` or `bun run test:env`
|
||||
- Runtime-compat / compiled behavior: `bun run test:runtime:compat`
|
||||
|
||||
@@ -157,6 +157,15 @@ export async function runStatsCommand(
|
||||
if (args.statsCleanupLifetime) {
|
||||
forwarded.push('--stats-cleanup-lifetime');
|
||||
}
|
||||
if (args.statsCleanupDuplicateLines) {
|
||||
forwarded.push('--stats-cleanup-duplicate-lines');
|
||||
}
|
||||
if (args.statsCleanupDryRun) {
|
||||
forwarded.push('--stats-cleanup-dry-run');
|
||||
}
|
||||
if (args.statsCleanupLookbackDays) {
|
||||
forwarded.push('--stats-cleanup-lookback-days', String(args.statsCleanupLookbackDays));
|
||||
}
|
||||
if (shouldForwardLogLevel(args.logLevel)) {
|
||||
forwarded.push('--log-level', args.logLevel);
|
||||
}
|
||||
|
||||
@@ -134,6 +134,9 @@ test('applyInvocationsToArgs maps config and jellyfin invocation state', () => {
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
statsCleanupLookbackDays: null,
|
||||
statsLogLevel: null,
|
||||
syncTriggered: false,
|
||||
syncCliTokens: [],
|
||||
@@ -185,6 +188,9 @@ test('applyInvocationsToArgs maps settings invocation to settings window', () =>
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
statsCleanupLookbackDays: null,
|
||||
statsLogLevel: null,
|
||||
syncTriggered: false,
|
||||
syncCliTokens: [],
|
||||
@@ -229,6 +235,9 @@ test('applyInvocationsToArgs fails when config invocation has no action', () =>
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
statsCleanupLookbackDays: null,
|
||||
statsLogLevel: null,
|
||||
syncTriggered: false,
|
||||
syncCliTokens: [],
|
||||
@@ -271,6 +280,9 @@ test('applyInvocationsToArgs maps texthooker browser-open request', () => {
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
statsCleanupLookbackDays: null,
|
||||
statsLogLevel: null,
|
||||
syncTriggered: false,
|
||||
syncCliTokens: [],
|
||||
|
||||
@@ -162,6 +162,8 @@ export function createDefaultArgs(
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
doctor: false,
|
||||
doctorRefreshKnownWords: false,
|
||||
logsExport: false,
|
||||
@@ -258,6 +260,11 @@ export function applyInvocationsToArgs(parsed: Args, invocations: CliInvocations
|
||||
if (invocations.statsCleanup) parsed.statsCleanup = true;
|
||||
if (invocations.statsCleanupVocab) parsed.statsCleanupVocab = true;
|
||||
if (invocations.statsCleanupLifetime) parsed.statsCleanupLifetime = true;
|
||||
if (invocations.statsCleanupDuplicateLines) parsed.statsCleanupDuplicateLines = true;
|
||||
if (invocations.statsCleanupDryRun) parsed.statsCleanupDryRun = true;
|
||||
if (invocations.statsCleanupLookbackDays !== null) {
|
||||
parsed.statsCleanupLookbackDays = invocations.statsCleanupLookbackDays;
|
||||
}
|
||||
if (invocations.dictionaryTarget) {
|
||||
parsed.dictionaryTarget = parseDictionaryTarget(invocations.dictionaryTarget);
|
||||
} else if (
|
||||
|
||||
@@ -37,6 +37,9 @@ export interface CliInvocations {
|
||||
statsCleanup: boolean;
|
||||
statsCleanupVocab: boolean;
|
||||
statsCleanupLifetime: boolean;
|
||||
statsCleanupDuplicateLines: boolean;
|
||||
statsCleanupDryRun: boolean;
|
||||
statsCleanupLookbackDays: number | null;
|
||||
statsLogLevel: string | null;
|
||||
syncTriggered: boolean;
|
||||
syncCliTokens: string[];
|
||||
@@ -53,6 +56,16 @@ export interface CliInvocations {
|
||||
texthookerOpenBrowser: boolean;
|
||||
}
|
||||
|
||||
/** `--lookback-days` narrows the duplicate-line cleanup; fractions are floored. */
|
||||
function parseStatsLookbackDays(value: unknown): number | null {
|
||||
if (typeof value !== 'string' && typeof value !== 'number') return null;
|
||||
const days = Number(value);
|
||||
if (!Number.isFinite(days) || days < 1) {
|
||||
throw new Error('Stats --lookback-days must be at least one day.');
|
||||
}
|
||||
return Math.floor(days);
|
||||
}
|
||||
|
||||
function applyRootOptions(program: Command): void {
|
||||
program
|
||||
.option(
|
||||
@@ -169,6 +182,9 @@ export function parseCliPrograms(
|
||||
let statsCleanup = false;
|
||||
let statsCleanupVocab = false;
|
||||
let statsCleanupLifetime = false;
|
||||
let statsCleanupDuplicateLines = false;
|
||||
let statsCleanupDryRun = false;
|
||||
let statsCleanupLookbackDays: number | null = null;
|
||||
let statsLogLevel: string | null = null;
|
||||
let syncTriggered = false;
|
||||
let syncCliTokens: string[] = [];
|
||||
@@ -269,6 +285,9 @@ export function parseCliPrograms(
|
||||
.option('-s, --stop', 'Stop the background stats server')
|
||||
.option('-v, --vocab', 'Clean vocabulary rows in the stats database')
|
||||
.option('-l, --lifetime', 'Rebuild lifetime summary rows from retained data')
|
||||
.option('-d, --duplicate-lines', 'Collapse repeated subtitle lines from typeset animations')
|
||||
.option('--dry-run', 'Report what a cleanup would remove without changing anything')
|
||||
.option('--lookback-days <days>', 'Only clean lines recorded in the last N days')
|
||||
.option('--log-level <level>', 'Log level')
|
||||
.action((action: string | undefined, options: Record<string, unknown>) => {
|
||||
statsTriggered = true;
|
||||
@@ -289,13 +308,35 @@ export function parseCliPrograms(
|
||||
if (normalizedAction && (statsBackground || statsStop)) {
|
||||
throw new Error('Stats background and stop flags cannot be combined with stats actions.');
|
||||
}
|
||||
if (normalizedAction !== 'cleanup' && (options.vocab === true || options.lifetime === true)) {
|
||||
throw new Error('Stats --vocab and --lifetime flags require the cleanup action.');
|
||||
if (
|
||||
normalizedAction !== 'cleanup' &&
|
||||
(options.vocab === true || options.lifetime === true || options.duplicateLines === true)
|
||||
) {
|
||||
throw new Error(
|
||||
'Stats --vocab, --lifetime and --duplicate-lines flags require the cleanup action.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
options.duplicateLines !== true &&
|
||||
(options.dryRun === true || options.lookbackDays !== undefined)
|
||||
) {
|
||||
throw new Error('Stats --dry-run and --lookback-days require --duplicate-lines.');
|
||||
}
|
||||
if (normalizedAction === 'cleanup') {
|
||||
statsCleanup = true;
|
||||
statsCleanupLifetime = options.lifetime === true;
|
||||
statsCleanupVocab = statsCleanupLifetime ? false : options.vocab !== false;
|
||||
statsCleanupDuplicateLines = options.duplicateLines === true;
|
||||
const explicitModeCount = [options.vocab, options.lifetime, options.duplicateLines].filter(
|
||||
(value) => value === true,
|
||||
).length;
|
||||
if (explicitModeCount > 1) {
|
||||
throw new Error('Stats cleanup runs one mode at a time.');
|
||||
}
|
||||
// Vocabulary cleanup stays the default so `stats cleanup` keeps its old meaning.
|
||||
statsCleanupVocab =
|
||||
statsCleanupLifetime || statsCleanupDuplicateLines ? false : options.vocab !== false;
|
||||
statsCleanupDryRun = options.dryRun === true;
|
||||
statsCleanupLookbackDays = parseStatsLookbackDays(options.lookbackDays);
|
||||
} else if (normalizedAction === 'rebuild' || normalizedAction === 'backfill') {
|
||||
statsCleanup = true;
|
||||
statsCleanupLifetime = true;
|
||||
@@ -483,6 +524,9 @@ export function parseCliPrograms(
|
||||
statsCleanup,
|
||||
statsCleanupVocab,
|
||||
statsCleanupLifetime,
|
||||
statsCleanupDuplicateLines,
|
||||
statsCleanupDryRun,
|
||||
statsCleanupLookbackDays,
|
||||
statsLogLevel,
|
||||
syncTriggered,
|
||||
syncCliTokens,
|
||||
|
||||
@@ -232,6 +232,75 @@ test('parseArgs maps lifetime stats cleanup flag', () => {
|
||||
assert.equal(parsed.statsCleanupLifetime, true);
|
||||
});
|
||||
|
||||
test('parseArgs maps duplicate-line stats cleanup flags', () => {
|
||||
const parsed = parseArgs(
|
||||
['stats', 'cleanup', '--duplicate-lines', '--dry-run', '--lookback-days', '30'],
|
||||
'subminer',
|
||||
{},
|
||||
);
|
||||
|
||||
assert.equal(parsed.statsCleanup, true);
|
||||
assert.equal(parsed.statsCleanupVocab, false);
|
||||
assert.equal(parsed.statsCleanupDuplicateLines, true);
|
||||
assert.equal(parsed.statsCleanupDryRun, true);
|
||||
assert.equal(parsed.statsCleanupLookbackDays, 30);
|
||||
|
||||
const fractional = parseArgs(
|
||||
['stats', 'cleanup', '--duplicate-lines', '--lookback-days', '1.5'],
|
||||
'subminer',
|
||||
{},
|
||||
);
|
||||
assert.equal(fractional.statsCleanupLookbackDays, 1);
|
||||
});
|
||||
|
||||
test('parseArgs rejects duplicate-line flags without the duplicate-lines mode', () => {
|
||||
const error = withProcessExitIntercept(() => {
|
||||
parseArgs(['stats', 'cleanup', '--dry-run'], 'subminer', {});
|
||||
});
|
||||
|
||||
assert.equal(error.code, 1);
|
||||
assert.match(error.stderr, /--dry-run and --lookback-days require --duplicate-lines/);
|
||||
});
|
||||
|
||||
test('parseArgs rejects an empty lookback value outside duplicate-line cleanup', () => {
|
||||
const error = withProcessExitIntercept(() => {
|
||||
parseArgs(['stats', '--lookback-days', ''], 'subminer', {});
|
||||
});
|
||||
|
||||
assert.equal(error.code, 1);
|
||||
assert.match(error.stderr, /--dry-run and --lookback-days require --duplicate-lines/);
|
||||
});
|
||||
|
||||
test('parseArgs rejects combining explicit cleanup modes', () => {
|
||||
for (const modes of [
|
||||
['--lifetime', '--duplicate-lines'],
|
||||
['--vocab', '--duplicate-lines'],
|
||||
['--vocab', '--lifetime'],
|
||||
]) {
|
||||
const error = withProcessExitIntercept(() => {
|
||||
parseArgs(['stats', 'cleanup', ...modes], 'subminer', {});
|
||||
});
|
||||
|
||||
assert.equal(error.code, 1);
|
||||
assert.match(error.stderr, /Stats cleanup runs one mode at a time/);
|
||||
}
|
||||
});
|
||||
|
||||
test('parseArgs rejects unusable lookback windows', () => {
|
||||
for (const value of ['0', '0.5', '-5', 'soon']) {
|
||||
const error = withProcessExitIntercept(() => {
|
||||
parseArgs(
|
||||
['stats', 'cleanup', '--duplicate-lines', '--lookback-days', value],
|
||||
'subminer',
|
||||
{},
|
||||
);
|
||||
});
|
||||
|
||||
assert.equal(error.code, 1);
|
||||
assert.match(error.stderr, /--lookback-days must be at least one day/);
|
||||
}
|
||||
});
|
||||
|
||||
test('parseArgs rejects cleanup-only stats flags without cleanup action', () => {
|
||||
const error = withProcessExitIntercept(() => {
|
||||
parseArgs(['stats', '--vocab'], 'subminer', {});
|
||||
@@ -239,7 +308,10 @@ test('parseArgs rejects cleanup-only stats flags without cleanup action', () =>
|
||||
|
||||
assert.equal(error.code, 1);
|
||||
assert.match(error.message, /exit:1/);
|
||||
assert.match(error.stderr, /Stats --vocab and --lifetime flags require the cleanup action/);
|
||||
assert.match(
|
||||
error.stderr,
|
||||
/Stats --vocab, --lifetime and --duplicate-lines flags require the cleanup action/,
|
||||
);
|
||||
});
|
||||
|
||||
test('parseArgs maps stats rebuild action to cleanup lifetime mode', () => {
|
||||
|
||||
@@ -80,11 +80,6 @@ test('merges remote-only sessions with catalog, lifetime, and rollups', () => {
|
||||
{ headword: '食べる', word: '食べた', reading: 'たべた', count: 1 },
|
||||
],
|
||||
});
|
||||
withWritableDb(remotePath, (db) => {
|
||||
db.prepare(
|
||||
`UPDATE imm_videos SET anime_assignment_locked = 1 WHERE video_key = 'showb-e1'`,
|
||||
).run();
|
||||
});
|
||||
|
||||
const summary = mergeSnapshotIntoDb(localPath, remotePath);
|
||||
assert.equal(summary.sessionsMerged, 1);
|
||||
@@ -131,14 +126,6 @@ test('merges remote-only sessions with catalog, lifetime, and rollups', () => {
|
||||
`SELECT video_id FROM imm_videos WHERE video_key = 'showb-e1'`,
|
||||
)?.video_id,
|
||||
);
|
||||
assert.equal(
|
||||
queryOne<{ locked: number }>(
|
||||
localPath,
|
||||
'SELECT anime_assignment_locked AS locked FROM imm_videos WHERE video_id = ?',
|
||||
[mergedVideoId],
|
||||
)?.locked,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
count(localPath, 'SELECT COUNT(*) AS n FROM imm_daily_rollups WHERE video_id = ?', [
|
||||
mergedVideoId,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Current schema shape of the tables the sync merge touches (plus the
|
||||
// Schema-version-18 shape of the tables the sync merge touches (plus the
|
||||
// app's indexes), mirroring ensureSchema / ensureLifetimeSummaryTables /
|
||||
// ensureStatsExcludedWordsTable in src/core/services/immersion-tracker/storage.ts.
|
||||
export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
@@ -39,7 +39,6 @@ export const IMMERSION_DB_FIXTURE_DDL = `
|
||||
parser_source TEXT,
|
||||
parser_confidence REAL,
|
||||
parse_metadata_json TEXT,
|
||||
anime_assignment_locked INTEGER NOT NULL DEFAULT 0 CHECK(anime_assignment_locked IN (0, 1)),
|
||||
watched INTEGER NOT NULL DEFAULT 0,
|
||||
duration_ms INTEGER NOT NULL CHECK(duration_ms>=0),
|
||||
file_size_bytes INTEGER CHECK(file_size_bytes>=0),
|
||||
|
||||
@@ -142,6 +142,9 @@ export interface Args {
|
||||
statsCleanup?: boolean;
|
||||
statsCleanupVocab?: boolean;
|
||||
statsCleanupLifetime?: boolean;
|
||||
statsCleanupDuplicateLines?: boolean;
|
||||
statsCleanupDryRun?: boolean;
|
||||
statsCleanupLookbackDays?: number;
|
||||
dictionaryTarget?: string;
|
||||
doctor: boolean;
|
||||
doctorRefreshKnownWords: boolean;
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
"name": "subminer-workflow",
|
||||
"version": "0.1.0",
|
||||
"description": "Repo-local SubMiner agent workflow plugin for orchestration and change verification.",
|
||||
"author": {
|
||||
"name": "Kyle Yasuda",
|
||||
"email": "suda@sudacode.com",
|
||||
"url": "https://github.com/sudacode"
|
||||
},
|
||||
"homepage": "https://github.com/sudacode/SubMiner/tree/main/plugins/subminer-workflow",
|
||||
"repository": "https://github.com/sudacode/SubMiner",
|
||||
"license": "GPL-3.0-or-later",
|
||||
"keywords": ["subminer", "workflow", "verification", "skills"],
|
||||
"skills": "./skills/",
|
||||
"interface": {
|
||||
"displayName": "SubMiner Workflow",
|
||||
"shortDescription": "SubMiner orchestration and verification.",
|
||||
"longDescription": "Canonical repo-local plugin for SubMiner agent workflow packaging. Owns the scrum-master and change-verification skills plus helper scripts used to plan, verify, and validate changes reproducibly inside this repo.",
|
||||
"developerName": "Kyle Yasuda",
|
||||
"category": "Productivity",
|
||||
"capabilities": ["Interactive", "Write"],
|
||||
"websiteURL": "https://github.com/sudacode/SubMiner",
|
||||
"defaultPrompt": [
|
||||
"Use SubMiner workflow to plan and ship a feature.",
|
||||
"Verify a SubMiner change with the plugin-owned verifier.",
|
||||
"Plan and ship this SubMiner task."
|
||||
],
|
||||
"brandColor": "#2F6B4F"
|
||||
}
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
<!-- read_when: migrating or using the repo-local SubMiner workflow plugin -->
|
||||
|
||||
# SubMiner Workflow Plugin
|
||||
|
||||
Status: active
|
||||
Last verified: 2026-03-26
|
||||
Owner: Kyle Yasuda
|
||||
Read when: using or updating the repo-local plugin that owns SubMiner agent workflow skills
|
||||
|
||||
This plugin is the canonical source of truth for the SubMiner agent workflow packaging.
|
||||
|
||||
## Contents
|
||||
|
||||
- `skills/subminer-scrum-master/`
|
||||
- intake, planning, dispatch, and handoff workflow
|
||||
- `skills/subminer-change-verification/`
|
||||
- cheap-first verification workflow plus helper scripts
|
||||
|
||||
## Compatibility
|
||||
|
||||
- `.agents/skills/subminer-scrum-master/` is a compatibility shim that redirects to the plugin-owned skill.
|
||||
- `.agents/skills/subminer-change-verification/` is a compatibility shim.
|
||||
- `.agents/skills/subminer-change-verification/scripts/*.sh` remain as wrapper entrypoints so existing docs and shell history keep working.
|
||||
|
||||
## Verification
|
||||
|
||||
For plugin/doc/shim changes, prefer:
|
||||
|
||||
```bash
|
||||
bun run test:docs:kb
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh --lane docs --lane core \
|
||||
plugins/subminer-workflow \
|
||||
.agents/skills/subminer-scrum-master/SKILL.md \
|
||||
.agents/skills/subminer-change-verification/SKILL.md \
|
||||
.agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh \
|
||||
.agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh \
|
||||
.agents/plugins/marketplace.json \
|
||||
docs/workflow/README.md \
|
||||
docs/workflow/agent-plugins.md
|
||||
```
|
||||
@@ -1,143 +0,0 @@
|
||||
---
|
||||
name: 'subminer-change-verification'
|
||||
description: 'Use when working in the SubMiner repo and you need to verify code changes actually work. Covers targeted regression checks during debugging and pre-handoff verification, with cheap-first lane selection for config, docs, launcher/plugin, runtime-compat, and optional real-runtime escalation.'
|
||||
---
|
||||
|
||||
# SubMiner Change Verification
|
||||
|
||||
Canonical source: this plugin path.
|
||||
|
||||
Use this skill for SubMiner code changes. Default to cheap, repo-native verification first. Escalate only when the changed behavior actually depends on Electron, mpv, overlay/window tracking, or other GUI-sensitive runtime behavior.
|
||||
|
||||
## Scripts
|
||||
|
||||
- `scripts/classify_subminer_diff.sh`
|
||||
- Emits suggested lanes and flags from explicit paths or current git changes.
|
||||
- `scripts/verify_subminer_change.sh`
|
||||
- Runs selected lanes, captures artifacts, and writes a compact summary.
|
||||
|
||||
If you need an explicit installed path, use the directory that contains this `SKILL.md`. The helper scripts live under:
|
||||
|
||||
```bash
|
||||
export SUBMINER_VERIFY_SKILL="<path-to-plugin-skill>"
|
||||
```
|
||||
|
||||
## Default workflow
|
||||
|
||||
1. Inspect the changed files or user-requested area.
|
||||
2. Run the classifier unless you already know the right lane.
|
||||
3. Run the verifier with the cheapest sufficient lane set.
|
||||
4. If the classifier emits `flag:real-runtime-candidate`, do not jump straight to runtime verification. First run the non-runtime lanes.
|
||||
5. Escalate to explicit `--lane real-runtime --allow-real-runtime` only when cheaper lanes cannot validate the behavior claim.
|
||||
6. Return:
|
||||
- verification summary
|
||||
- exact commands run
|
||||
- artifact paths
|
||||
- skipped lanes and blockers
|
||||
|
||||
## Quick start
|
||||
|
||||
Plugin-source quick start:
|
||||
|
||||
```bash
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/classify_subminer_diff.sh
|
||||
```
|
||||
|
||||
Installed-skill quick start:
|
||||
|
||||
```bash
|
||||
bash "$SUBMINER_VERIFY_SKILL/scripts/classify_subminer_diff.sh"
|
||||
```
|
||||
|
||||
Compatibility entrypoint:
|
||||
|
||||
```bash
|
||||
bash .agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh
|
||||
```
|
||||
|
||||
Classify explicit files:
|
||||
|
||||
```bash
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/classify_subminer_diff.sh \
|
||||
launcher/main.ts \
|
||||
plugin/subminer/lifecycle.lua \
|
||||
src/main/runtime/mpv-client-runtime-service.ts
|
||||
```
|
||||
|
||||
Run automatic lane selection:
|
||||
|
||||
```bash
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh
|
||||
```
|
||||
|
||||
Installed-skill form:
|
||||
|
||||
```bash
|
||||
bash "$SUBMINER_VERIFY_SKILL/scripts/verify_subminer_change.sh"
|
||||
```
|
||||
|
||||
Compatibility entrypoint:
|
||||
|
||||
```bash
|
||||
bash .agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh
|
||||
```
|
||||
|
||||
Run targeted lanes:
|
||||
|
||||
```bash
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh \
|
||||
--lane launcher-plugin \
|
||||
--lane runtime-compat
|
||||
```
|
||||
|
||||
Dry-run to inspect planned commands and artifact layout:
|
||||
|
||||
```bash
|
||||
bash plugins/subminer-workflow/skills/subminer-change-verification/scripts/verify_subminer_change.sh \
|
||||
--dry-run \
|
||||
launcher/main.ts \
|
||||
src/main.ts
|
||||
```
|
||||
|
||||
## Lane guidance
|
||||
|
||||
- `docs`
|
||||
- For `docs-site/`, `docs/`, and doc-only edits.
|
||||
- `config`
|
||||
- For `src/config/` and config-template-sensitive edits.
|
||||
- `stats`
|
||||
- For `stats/` dashboard UI edits.
|
||||
- `core`
|
||||
- For general source changes where `typecheck` + `test:fast` is the best cheap signal.
|
||||
- `launcher-plugin`
|
||||
- For `launcher/`, `plugin/subminer/`, plugin gating scripts, and wrapper/mpv routing work.
|
||||
- `runtime-compat`
|
||||
- For `src/main*`, runtime/composer wiring, mpv/overlay services, window trackers, and dist-sensitive behavior.
|
||||
- `real-runtime`
|
||||
- Only after deliberate escalation.
|
||||
|
||||
## Real Runtime Escalation
|
||||
|
||||
Escalate only when the change claim depends on actual runtime behavior, for example:
|
||||
|
||||
- overlay appears, hides, or tracks a real mpv window
|
||||
- mpv launch flags or pause-until-ready behavior
|
||||
- plugin/socket/auto-start handshake under a real player
|
||||
- macOS/window-tracker/focus-sensitive behavior
|
||||
|
||||
If the environment cannot support authoritative runtime verification, report the blocker explicitly. Do not silently downgrade a runtime-required claim to a pass.
|
||||
|
||||
## Artifact contract
|
||||
|
||||
The verifier writes under `.tmp/skill-verification/<timestamp>/`:
|
||||
|
||||
- `summary.json`
|
||||
- `summary.txt`
|
||||
- `classification.txt`
|
||||
- `env.txt`
|
||||
- `lanes.txt`
|
||||
- `steps.tsv`
|
||||
- `steps/*.stdout.log`
|
||||
- `steps/*.stderr.log`
|
||||
|
||||
On failure, quote the exact failing command and point at the artifact directory.
|
||||
-171
@@ -1,171 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: classify_subminer_diff.sh [path ...]
|
||||
|
||||
Emit suggested verification lanes for explicit paths or current local git changes.
|
||||
|
||||
Output format:
|
||||
lane:<name>
|
||||
flag:<name>
|
||||
reason:<text>
|
||||
EOF
|
||||
}
|
||||
|
||||
has_item() {
|
||||
local needle=$1
|
||||
shift || true
|
||||
local item
|
||||
for item in "$@"; do
|
||||
if [[ "$item" == "$needle" ]]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
add_lane() {
|
||||
local lane=$1
|
||||
if ! has_item "$lane" "${LANES[@]:-}"; then
|
||||
LANES+=("$lane")
|
||||
fi
|
||||
}
|
||||
|
||||
add_flag() {
|
||||
local flag=$1
|
||||
if ! has_item "$flag" "${FLAGS[@]:-}"; then
|
||||
FLAGS+=("$flag")
|
||||
fi
|
||||
}
|
||||
|
||||
add_reason() {
|
||||
REASONS+=("$1")
|
||||
}
|
||||
|
||||
collect_git_paths() {
|
||||
local top_level
|
||||
if ! top_level=$(git rev-parse --show-toplevel 2>/dev/null); then
|
||||
return 0
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$top_level"
|
||||
if git rev-parse --verify HEAD >/dev/null 2>&1; then
|
||||
git diff --name-only --relative HEAD --
|
||||
git diff --name-only --relative --cached --
|
||||
else
|
||||
git diff --name-only --relative --
|
||||
git diff --name-only --relative --cached --
|
||||
fi
|
||||
git ls-files --others --exclude-standard
|
||||
) | awk 'NF' | sort -u
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
declare -a PATHS=()
|
||||
declare -a LANES=()
|
||||
declare -a FLAGS=()
|
||||
declare -a REASONS=()
|
||||
|
||||
if [[ $# -gt 0 ]]; then
|
||||
while [[ $# -gt 0 ]]; do
|
||||
PATHS+=("$1")
|
||||
shift
|
||||
done
|
||||
else
|
||||
while IFS= read -r line; do
|
||||
[[ -n "$line" ]] && PATHS+=("$line")
|
||||
done < <(collect_git_paths)
|
||||
fi
|
||||
|
||||
if [[ ${#PATHS[@]} -eq 0 ]]; then
|
||||
add_lane "core"
|
||||
add_reason "no changed paths detected -> default to core"
|
||||
fi
|
||||
|
||||
for path in "${PATHS[@]}"; do
|
||||
specialized=0
|
||||
|
||||
case "$path" in
|
||||
docs-site/*|docs/*|changes/*|README.md)
|
||||
add_lane "docs"
|
||||
add_reason "$path -> docs"
|
||||
specialized=1
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$path" in
|
||||
src/config/*|src/generate-config-example.ts|src/verify-config-example.ts|docs-site/public/config.example.jsonc|config.example.jsonc)
|
||||
add_lane "config"
|
||||
add_reason "$path -> config"
|
||||
specialized=1
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$path" in
|
||||
stats/*)
|
||||
add_lane "stats"
|
||||
add_reason "$path -> stats"
|
||||
specialized=1
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$path" in
|
||||
launcher/*|plugin/subminer/*|plugin/subminer.conf|scripts/test-plugin-*|scripts/get-mpv-window-*|scripts/configure-plugin-binary-path.mjs)
|
||||
add_lane "launcher-plugin"
|
||||
add_reason "$path -> launcher-plugin"
|
||||
add_flag "real-runtime-candidate"
|
||||
add_reason "$path -> real-runtime-candidate"
|
||||
specialized=1
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$path" in
|
||||
src/main.ts|src/main-entry.ts|src/preload.ts|src/main/*|src/core/services/mpv*|src/core/services/overlay*|src/renderer/*|src/window-trackers/*|scripts/prepare-build-assets.mjs)
|
||||
add_lane "runtime-compat"
|
||||
add_reason "$path -> runtime-compat"
|
||||
add_flag "real-runtime-candidate"
|
||||
add_reason "$path -> real-runtime-candidate"
|
||||
specialized=1
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ "$specialized" == "0" ]]; then
|
||||
case "$path" in
|
||||
src/*|package.json|tsconfig*.json|scripts/*|Makefile)
|
||||
add_lane "core"
|
||||
add_reason "$path -> core"
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
case "$path" in
|
||||
package.json|src/main.ts|src/main-entry.ts|src/preload.ts)
|
||||
add_flag "broad-impact"
|
||||
add_reason "$path -> broad-impact"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ ${#LANES[@]} -eq 0 ]]; then
|
||||
add_lane "core"
|
||||
add_reason "no lane-specific matches -> default to core"
|
||||
fi
|
||||
|
||||
for lane in "${LANES[@]}"; do
|
||||
printf 'lane:%s\n' "$lane"
|
||||
done
|
||||
|
||||
for flag in "${FLAGS[@]}"; do
|
||||
printf 'flag:%s\n' "$flag"
|
||||
done
|
||||
|
||||
for reason in "${REASONS[@]}"; do
|
||||
printf 'reason:%s\n' "$reason"
|
||||
done
|
||||
-537
@@ -1,537 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: verify_subminer_change.sh [options] [path ...]
|
||||
|
||||
Options:
|
||||
--lane <name> Force a verification lane. Repeatable.
|
||||
--artifact-dir <dir> Use an explicit artifact directory.
|
||||
--allow-real-runtime Allow explicit real-runtime execution.
|
||||
--allow-real-gui Deprecated alias for --allow-real-runtime.
|
||||
--dry-run Record planned steps without executing commands.
|
||||
--help Show this help text.
|
||||
|
||||
If no lanes are supplied, the script classifies the provided paths. If no paths are
|
||||
provided, it classifies the current local git changes.
|
||||
|
||||
Authoritative real-runtime verification should be requested with explicit path
|
||||
arguments instead of relying on inferred local git changes.
|
||||
EOF
|
||||
}
|
||||
|
||||
timestamp() {
|
||||
date +%Y%m%d-%H%M%S
|
||||
}
|
||||
|
||||
timestamp_iso() {
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ
|
||||
}
|
||||
|
||||
generate_session_id() {
|
||||
local tmp_dir
|
||||
tmp_dir=$(mktemp -d "${TMPDIR:-/tmp}/subminer-verify-$(timestamp)-XXXXXX")
|
||||
basename "$tmp_dir"
|
||||
rmdir "$tmp_dir"
|
||||
}
|
||||
|
||||
has_item() {
|
||||
local needle=$1
|
||||
shift || true
|
||||
local item
|
||||
for item in "$@"; do
|
||||
if [[ "$item" == "$needle" ]]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
normalize_lane_name() {
|
||||
case "$1" in
|
||||
real-gui)
|
||||
printf '%s' "real-runtime"
|
||||
;;
|
||||
*)
|
||||
printf '%s' "$1"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
add_lane() {
|
||||
local lane
|
||||
lane=$(normalize_lane_name "$1")
|
||||
if ! has_item "$lane" "${SELECTED_LANES[@]:-}"; then
|
||||
SELECTED_LANES+=("$lane")
|
||||
fi
|
||||
}
|
||||
|
||||
add_blocker() {
|
||||
BLOCKERS+=("$1")
|
||||
BLOCKED=1
|
||||
}
|
||||
|
||||
validate_artifact_dir() {
|
||||
local candidate=$1
|
||||
if [[ ! "$candidate" =~ ^[A-Za-z0-9._/@:+-]+$ ]]; then
|
||||
echo "Invalid characters in --artifact-dir path" >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
append_step_record() {
|
||||
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
|
||||
"$1" "$2" "$3" "$4" "$5" "$6" "$7" "$8" >>"$STEPS_TSV"
|
||||
}
|
||||
|
||||
record_env() {
|
||||
{
|
||||
printf 'repo_root=%s\n' "$REPO_ROOT"
|
||||
printf 'session_id=%s\n' "$SESSION_ID"
|
||||
printf 'artifact_dir=%s\n' "$ARTIFACT_DIR"
|
||||
printf 'path_selection_mode=%s\n' "$PATH_SELECTION_MODE"
|
||||
printf 'dry_run=%s\n' "$DRY_RUN"
|
||||
printf 'allow_real_runtime=%s\n' "$ALLOW_REAL_RUNTIME"
|
||||
printf 'session_home=%s\n' "$SESSION_HOME"
|
||||
printf 'session_xdg_config_home=%s\n' "$SESSION_XDG_CONFIG_HOME"
|
||||
printf 'session_mpv_dir=%s\n' "$SESSION_MPV_DIR"
|
||||
printf 'session_logs_dir=%s\n' "$SESSION_LOGS_DIR"
|
||||
printf 'session_mpv_log=%s\n' "$SESSION_MPV_LOG"
|
||||
printf 'pwd=%s\n' "$(pwd)"
|
||||
git rev-parse --short HEAD 2>/dev/null | sed 's/^/git_head=/' || true
|
||||
git status --short 2>/dev/null || true
|
||||
if [[ ${#PATH_ARGS[@]} -gt 0 ]]; then
|
||||
printf 'requested_paths=\n'
|
||||
printf ' %s\n' "${PATH_ARGS[@]}"
|
||||
fi
|
||||
} >"$ARTIFACT_DIR/env.txt"
|
||||
}
|
||||
|
||||
run_step() {
|
||||
local lane=$1
|
||||
local name=$2
|
||||
local command=$3
|
||||
local note=${4:-}
|
||||
local lane_slug=${lane//[^a-zA-Z0-9_-]/-}
|
||||
local slug=${name//[^a-zA-Z0-9_-]/-}
|
||||
local step_slug="${lane_slug}--${slug}"
|
||||
local stdout_rel="steps/${step_slug}.stdout.log"
|
||||
local stderr_rel="steps/${step_slug}.stderr.log"
|
||||
local stdout_path="$ARTIFACT_DIR/$stdout_rel"
|
||||
local stderr_path="$ARTIFACT_DIR/$stderr_rel"
|
||||
local status exit_code
|
||||
|
||||
COMMANDS_RUN+=("$command")
|
||||
printf '%s\n' "$command" >"$ARTIFACT_DIR/steps/${step_slug}.command.txt"
|
||||
|
||||
if [[ "$DRY_RUN" == "1" ]]; then
|
||||
printf '[dry-run] %s\n' "$command" >"$stdout_path"
|
||||
: >"$stderr_path"
|
||||
status="dry-run"
|
||||
exit_code=0
|
||||
else
|
||||
if HOME="$SESSION_HOME" \
|
||||
XDG_CONFIG_HOME="$SESSION_XDG_CONFIG_HOME" \
|
||||
SUBMINER_SESSION_LOGS_DIR="$SESSION_LOGS_DIR" \
|
||||
SUBMINER_SESSION_MPV_LOG="$SESSION_MPV_LOG" \
|
||||
bash -c "cd \"$REPO_ROOT\" && $command" >"$stdout_path" 2>"$stderr_path"; then
|
||||
status="passed"
|
||||
exit_code=0
|
||||
EXECUTED_REAL_STEPS=1
|
||||
else
|
||||
exit_code=$?
|
||||
status="failed"
|
||||
FAILED=1
|
||||
fi
|
||||
fi
|
||||
|
||||
append_step_record "$lane" "$name" "$status" "$exit_code" "$command" "$stdout_rel" "$stderr_rel" "$note"
|
||||
printf '%s\t%s\t%s\n' "$lane" "$name" "$status"
|
||||
|
||||
if [[ "$status" == "failed" ]]; then
|
||||
FAILURE_STEP="$name"
|
||||
FAILURE_COMMAND="$command"
|
||||
FAILURE_STDOUT="$stdout_rel"
|
||||
FAILURE_STDERR="$stderr_rel"
|
||||
return "$exit_code"
|
||||
fi
|
||||
}
|
||||
|
||||
record_nonpassing_step() {
|
||||
local lane=$1
|
||||
local name=$2
|
||||
local status=$3
|
||||
local note=$4
|
||||
local lane_slug=${lane//[^a-zA-Z0-9_-]/-}
|
||||
local slug=${name//[^a-zA-Z0-9_-]/-}
|
||||
local step_slug="${lane_slug}--${slug}"
|
||||
local stdout_rel="steps/${step_slug}.stdout.log"
|
||||
local stderr_rel="steps/${step_slug}.stderr.log"
|
||||
printf '%s\n' "$note" >"$ARTIFACT_DIR/$stdout_rel"
|
||||
: >"$ARTIFACT_DIR/$stderr_rel"
|
||||
append_step_record "$lane" "$name" "$status" "0" "" "$stdout_rel" "$stderr_rel" "$note"
|
||||
printf '%s\t%s\t%s\n' "$lane" "$name" "$status"
|
||||
}
|
||||
|
||||
record_skipped_step() {
|
||||
record_nonpassing_step "$1" "$2" "skipped" "$3"
|
||||
}
|
||||
|
||||
record_blocked_step() {
|
||||
add_blocker "$3"
|
||||
record_nonpassing_step "$1" "$2" "blocked" "$3"
|
||||
}
|
||||
|
||||
record_failed_step() {
|
||||
FAILED=1
|
||||
FAILURE_STEP=$2
|
||||
FAILURE_COMMAND=${FAILURE_COMMAND:-"(validation)"}
|
||||
local lane_slug=${1//[^a-zA-Z0-9_-]/-}
|
||||
local step_slug=${2//[^a-zA-Z0-9_-]/-}
|
||||
FAILURE_STDOUT="steps/${lane_slug}--${step_slug}.stdout.log"
|
||||
FAILURE_STDERR="steps/${lane_slug}--${step_slug}.stderr.log"
|
||||
add_blocker "$3"
|
||||
record_nonpassing_step "$1" "$2" "failed" "$3"
|
||||
}
|
||||
|
||||
find_real_runtime_helper() {
|
||||
local candidate
|
||||
for candidate in \
|
||||
"$SCRIPT_DIR/run_real_runtime_smoke.sh" \
|
||||
"$SCRIPT_DIR/run_real_mpv_smoke.sh"; do
|
||||
if [[ -x "$candidate" ]]; then
|
||||
printf '%s' "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
acquire_real_runtime_lease() {
|
||||
local lease_root="$REPO_ROOT/.tmp/skill-verification/locks"
|
||||
local lease_dir="$lease_root/exclusive-real-runtime"
|
||||
mkdir -p "$lease_root"
|
||||
if mkdir "$lease_dir" 2>/dev/null; then
|
||||
REAL_RUNTIME_LEASE_DIR="$lease_dir"
|
||||
printf '%s\n' "$SESSION_ID" >"$lease_dir/session_id"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local owner=""
|
||||
if [[ -f "$lease_dir/session_id" ]]; then
|
||||
owner=$(cat "$lease_dir/session_id")
|
||||
fi
|
||||
REAL_RUNTIME_LEASE_ERROR="real-runtime lease already held${owner:+ by $owner}"
|
||||
return 1
|
||||
}
|
||||
|
||||
release_real_runtime_lease() {
|
||||
if [[ -n "$REAL_RUNTIME_LEASE_DIR" && -d "$REAL_RUNTIME_LEASE_DIR" ]]; then
|
||||
if [[ -f "$REAL_RUNTIME_LEASE_DIR/session_id" ]]; then
|
||||
local owner
|
||||
owner=$(cat "$REAL_RUNTIME_LEASE_DIR/session_id")
|
||||
if [[ "$owner" != "$SESSION_ID" ]]; then
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
rm -rf "$REAL_RUNTIME_LEASE_DIR"
|
||||
fi
|
||||
}
|
||||
|
||||
compute_final_status() {
|
||||
if [[ "$FAILED" == "1" ]]; then
|
||||
FINAL_STATUS="failed"
|
||||
elif [[ "$BLOCKED" == "1" ]]; then
|
||||
FINAL_STATUS="blocked"
|
||||
elif [[ "$EXECUTED_REAL_STEPS" == "1" ]]; then
|
||||
FINAL_STATUS="passed"
|
||||
else
|
||||
FINAL_STATUS="skipped"
|
||||
fi
|
||||
}
|
||||
|
||||
write_summary_files() {
|
||||
local lane_lines
|
||||
lane_lines=$(printf '%s\n' "${SELECTED_LANES[@]}")
|
||||
printf '%s\n' "$lane_lines" >"$ARTIFACT_DIR/lanes.txt"
|
||||
# bash 3.2 raises "unbound variable" under set -u when expanding an empty
|
||||
# array, so guard on length (matching the idiom used elsewhere here).
|
||||
if [[ ${#BLOCKERS[@]} -gt 0 ]]; then
|
||||
printf '%s\n' "${BLOCKERS[@]}" >"$ARTIFACT_DIR/blockers.txt"
|
||||
else
|
||||
: >"$ARTIFACT_DIR/blockers.txt"
|
||||
fi
|
||||
if [[ ${#PATH_ARGS[@]} -gt 0 ]]; then
|
||||
printf '%s\n' "${PATH_ARGS[@]}" >"$ARTIFACT_DIR/requested-paths.txt"
|
||||
else
|
||||
: >"$ARTIFACT_DIR/requested-paths.txt"
|
||||
fi
|
||||
|
||||
ARTIFACT_DIR_ENV="$ARTIFACT_DIR" \
|
||||
SESSION_ID_ENV="$SESSION_ID" \
|
||||
FINAL_STATUS_ENV="$FINAL_STATUS" \
|
||||
PATH_SELECTION_MODE_ENV="$PATH_SELECTION_MODE" \
|
||||
ALLOW_REAL_RUNTIME_ENV="$ALLOW_REAL_RUNTIME" \
|
||||
SESSION_HOME_ENV="$SESSION_HOME" \
|
||||
SESSION_XDG_CONFIG_HOME_ENV="$SESSION_XDG_CONFIG_HOME" \
|
||||
SESSION_MPV_DIR_ENV="$SESSION_MPV_DIR" \
|
||||
SESSION_LOGS_DIR_ENV="$SESSION_LOGS_DIR" \
|
||||
SESSION_MPV_LOG_ENV="$SESSION_MPV_LOG" \
|
||||
STARTED_AT_ENV="$STARTED_AT" \
|
||||
FINISHED_AT_ENV="$FINISHED_AT" \
|
||||
FAILED_ENV="$FAILED" \
|
||||
FAILURE_COMMAND_ENV="${FAILURE_COMMAND:-}" \
|
||||
FAILURE_STDOUT_ENV="${FAILURE_STDOUT:-}" \
|
||||
FAILURE_STDERR_ENV="${FAILURE_STDERR:-}" \
|
||||
bun -e '
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
|
||||
const lines = fs
|
||||
.readFileSync(path.join(process.env.ARTIFACT_DIR_ENV, "steps.tsv"), "utf8")
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.slice(1)
|
||||
.map((line) => {
|
||||
const [lane, name, status, exitCode, command, stdout, stderr, note] = line.split("\t");
|
||||
return { lane, name, status, exitCode: Number(exitCode), command, stdout, stderr, note };
|
||||
});
|
||||
|
||||
const payload = {
|
||||
sessionId: process.env.SESSION_ID_ENV,
|
||||
startedAt: process.env.STARTED_AT_ENV,
|
||||
finishedAt: process.env.FINISHED_AT_ENV,
|
||||
status: process.env.FINAL_STATUS_ENV,
|
||||
pathSelectionMode: process.env.PATH_SELECTION_MODE_ENV,
|
||||
allowRealRuntime: process.env.ALLOW_REAL_RUNTIME_ENV === "1",
|
||||
sessionHome: process.env.SESSION_HOME_ENV,
|
||||
sessionXdgConfigHome: process.env.SESSION_XDG_CONFIG_HOME_ENV,
|
||||
sessionMpvDir: process.env.SESSION_MPV_DIR_ENV,
|
||||
sessionLogsDir: process.env.SESSION_LOGS_DIR_ENV,
|
||||
sessionMpvLog: process.env.SESSION_MPV_LOG_ENV,
|
||||
failed: process.env.FAILED_ENV === "1",
|
||||
failure: process.env.FAILURE_COMMAND_ENV
|
||||
? {
|
||||
command: process.env.FAILURE_COMMAND_ENV,
|
||||
stdout: process.env.FAILURE_STDOUT_ENV,
|
||||
stderr: process.env.FAILURE_STDERR_ENV,
|
||||
}
|
||||
: null,
|
||||
blockers: fs
|
||||
.readFileSync(path.join(process.env.ARTIFACT_DIR_ENV, "blockers.txt"), "utf8")
|
||||
.split("\n")
|
||||
.filter(Boolean),
|
||||
lanes: fs
|
||||
.readFileSync(path.join(process.env.ARTIFACT_DIR_ENV, "lanes.txt"), "utf8")
|
||||
.split("\n")
|
||||
.filter(Boolean),
|
||||
requestedPaths: fs
|
||||
.readFileSync(path.join(process.env.ARTIFACT_DIR_ENV, "requested-paths.txt"), "utf8")
|
||||
.split("\n")
|
||||
.filter(Boolean),
|
||||
steps: lines,
|
||||
};
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(process.env.ARTIFACT_DIR_ENV, "summary.json"),
|
||||
JSON.stringify(payload, null, 2) + "\n",
|
||||
);
|
||||
|
||||
const summaryLines = [
|
||||
`status: ${payload.status}`,
|
||||
`session: ${payload.sessionId}`,
|
||||
`artifacts: ${process.env.ARTIFACT_DIR_ENV}`,
|
||||
`lanes: ${payload.lanes.join(", ") || "(none)"}`,
|
||||
];
|
||||
|
||||
if (payload.requestedPaths.length > 0) {
|
||||
summaryLines.push("requested paths:");
|
||||
for (const entry of payload.requestedPaths) {
|
||||
summaryLines.push(`- ${entry}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (payload.failure) {
|
||||
summaryLines.push(`failure command: ${payload.failure.command}`);
|
||||
summaryLines.push(`failure stdout: ${payload.failure.stdout}`);
|
||||
summaryLines.push(`failure stderr: ${payload.failure.stderr}`);
|
||||
}
|
||||
|
||||
if (payload.blockers.length > 0) {
|
||||
summaryLines.push("blockers:");
|
||||
for (const blocker of payload.blockers) {
|
||||
summaryLines.push(`- ${blocker}`);
|
||||
}
|
||||
}
|
||||
|
||||
summaryLines.push("steps:");
|
||||
for (const step of payload.steps) {
|
||||
summaryLines.push(`- ${step.lane}/${step.name}: ${step.status}`);
|
||||
}
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(process.env.ARTIFACT_DIR_ENV, "summary.txt"),
|
||||
summaryLines.join("\n") + "\n",
|
||||
);
|
||||
'
|
||||
}
|
||||
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
SKILL_DIR=$(cd "$SCRIPT_DIR/.." && pwd)
|
||||
REPO_ROOT=$(git rev-parse --show-toplevel 2>/dev/null || pwd)
|
||||
|
||||
declare -a PATH_ARGS=()
|
||||
declare -a SELECTED_LANES=()
|
||||
declare -a COMMANDS_RUN=()
|
||||
declare -a BLOCKERS=()
|
||||
|
||||
ALLOW_REAL_RUNTIME=0
|
||||
DRY_RUN=0
|
||||
FAILED=0
|
||||
BLOCKED=0
|
||||
EXECUTED_REAL_STEPS=0
|
||||
FAILURE_STEP=""
|
||||
FAILURE_COMMAND=""
|
||||
FAILURE_STDOUT=""
|
||||
FAILURE_STDERR=""
|
||||
REAL_RUNTIME_LEASE_DIR=""
|
||||
REAL_RUNTIME_LEASE_ERROR=""
|
||||
PATH_SELECTION_MODE="auto"
|
||||
|
||||
trap 'release_real_runtime_lease' EXIT
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--lane)
|
||||
shift
|
||||
[[ $# -gt 0 ]] || {
|
||||
echo "Missing value for --lane" >&2
|
||||
exit 2
|
||||
}
|
||||
add_lane "$1"
|
||||
PATH_SELECTION_MODE="explicit-lanes"
|
||||
;;
|
||||
--artifact-dir)
|
||||
shift
|
||||
[[ $# -gt 0 ]] || {
|
||||
echo "Missing value for --artifact-dir" >&2
|
||||
exit 2
|
||||
}
|
||||
ARTIFACT_DIR=$1
|
||||
;;
|
||||
--allow-real-runtime|--allow-real-gui)
|
||||
ALLOW_REAL_RUNTIME=1
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=1
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
PATH_ARGS+=("$1")
|
||||
;;
|
||||
esac
|
||||
shift || true
|
||||
done
|
||||
|
||||
if [[ -z "${ARTIFACT_DIR:-}" ]]; then
|
||||
SESSION_ID=$(generate_session_id)
|
||||
ARTIFACT_DIR="$REPO_ROOT/.tmp/skill-verification/$SESSION_ID"
|
||||
else
|
||||
validate_artifact_dir "$ARTIFACT_DIR"
|
||||
SESSION_ID=$(basename "$ARTIFACT_DIR")
|
||||
fi
|
||||
|
||||
mkdir -p "$ARTIFACT_DIR/steps"
|
||||
STEPS_TSV="$ARTIFACT_DIR/steps.tsv"
|
||||
printf 'lane\tstep\tstatus\texit_code\tcommand\tstdout\tstderr\tnote\n' >"$STEPS_TSV"
|
||||
|
||||
STARTED_AT=$(timestamp_iso)
|
||||
SESSION_HOME="$REPO_ROOT/.tmp/skill-verification/runtime/$SESSION_ID/home"
|
||||
SESSION_XDG_CONFIG_HOME="$REPO_ROOT/.tmp/skill-verification/runtime/$SESSION_ID/xdg-config"
|
||||
SESSION_MPV_DIR="$SESSION_XDG_CONFIG_HOME/mpv"
|
||||
SESSION_LOGS_DIR="$REPO_ROOT/.tmp/skill-verification/runtime/$SESSION_ID/logs"
|
||||
SESSION_MPV_LOG="$SESSION_LOGS_DIR/mpv.log"
|
||||
mkdir -p "$SESSION_HOME" "$SESSION_MPV_DIR" "$SESSION_LOGS_DIR"
|
||||
|
||||
CLASSIFIER_OUTPUT="$ARTIFACT_DIR/classification.txt"
|
||||
if [[ ${#SELECTED_LANES[@]} -eq 0 ]]; then
|
||||
if [[ ${#PATH_ARGS[@]} -gt 0 ]]; then
|
||||
PATH_SELECTION_MODE="explicit-paths"
|
||||
fi
|
||||
if "$SCRIPT_DIR/classify_subminer_diff.sh" "${PATH_ARGS[@]}" >"$CLASSIFIER_OUTPUT"; then
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
lane:*)
|
||||
add_lane "${line#lane:}"
|
||||
;;
|
||||
esac
|
||||
done <"$CLASSIFIER_OUTPUT"
|
||||
else
|
||||
record_failed_step "meta" "classify" "classification failed"
|
||||
fi
|
||||
else
|
||||
: >"$CLASSIFIER_OUTPUT"
|
||||
fi
|
||||
|
||||
record_env
|
||||
|
||||
if [[ ${#SELECTED_LANES[@]} -eq 0 ]]; then
|
||||
add_lane "core"
|
||||
fi
|
||||
|
||||
for lane in "${SELECTED_LANES[@]}"; do
|
||||
case "$lane" in
|
||||
docs)
|
||||
run_step "$lane" "docs-kb" "bun run test:docs:kb" || break
|
||||
;;
|
||||
config)
|
||||
run_step "$lane" "config" "bun run test:config" || break
|
||||
;;
|
||||
stats)
|
||||
run_step "$lane" "stats" "bun run test:stats" || break
|
||||
;;
|
||||
core)
|
||||
run_step "$lane" "typecheck" "bun run typecheck" || break
|
||||
run_step "$lane" "fast-tests" "bun run test:fast" || break
|
||||
;;
|
||||
launcher-plugin)
|
||||
run_step "$lane" "launcher" "bun run test:launcher" || break
|
||||
run_step "$lane" "plugin-src" "bun run test:plugin:src" || break
|
||||
;;
|
||||
runtime-compat)
|
||||
run_step "$lane" "runtime-compat" "bun run test:runtime:compat" || break
|
||||
;;
|
||||
real-runtime)
|
||||
if [[ "$ALLOW_REAL_RUNTIME" != "1" ]]; then
|
||||
record_blocked_step "$lane" "real-runtime" "real-runtime requested without --allow-real-runtime"
|
||||
continue
|
||||
fi
|
||||
if ! acquire_real_runtime_lease; then
|
||||
record_blocked_step "$lane" "real-runtime-lease" "$REAL_RUNTIME_LEASE_ERROR"
|
||||
continue
|
||||
fi
|
||||
helper=$(find_real_runtime_helper || true)
|
||||
if [[ -z "${helper:-}" ]]; then
|
||||
record_blocked_step "$lane" "real-runtime-helper" "no real-runtime helper script available in $SCRIPT_DIR"
|
||||
continue
|
||||
fi
|
||||
run_step "$lane" "real-runtime" "\"$helper\" \"$SESSION_ID\" \"$ARTIFACT_DIR\"" || break
|
||||
;;
|
||||
*)
|
||||
record_blocked_step "$lane" "unknown-lane" "unknown lane: $lane"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
release_real_runtime_lease
|
||||
FINISHED_AT=$(timestamp_iso)
|
||||
compute_final_status
|
||||
write_summary_files
|
||||
|
||||
printf 'summary:%s\n' "$ARTIFACT_DIR/summary.txt"
|
||||
cat "$ARTIFACT_DIR/summary.txt"
|
||||
@@ -1,118 +0,0 @@
|
||||
---
|
||||
name: 'subminer-scrum-master'
|
||||
description: 'Use in the SubMiner repo when a request should be turned into planned work and driven through execution. Records a plan, dispatches one or more subagents when useful, and requires verification before handoff.'
|
||||
---
|
||||
|
||||
# SubMiner Scrum Master
|
||||
|
||||
Canonical source: this plugin path.
|
||||
|
||||
Own workflow, not code by default.
|
||||
|
||||
Use this skill when the user gives a feature request, bug report, issue, refactor, or implementation ask and the agent should manage intake, planning, worker dispatch, and verification through completion.
|
||||
|
||||
## Core Rules
|
||||
|
||||
1. Keep the process light for questions, obvious mechanical edits, and tiny isolated changes.
|
||||
2. Record a plan before dispatching coding work.
|
||||
3. Split multi-part work into clear phases and ownership areas.
|
||||
4. Dispatch conservatively. Parallelize only disjoint write scopes.
|
||||
5. Require verification before handoff, typically via `subminer-change-verification`.
|
||||
6. Report dispatched workers, verification, blockers, and remaining risks.
|
||||
|
||||
## Intake Workflow
|
||||
|
||||
1. Parse the request.
|
||||
Classify it as question, mechanical edit, bugfix, feature, refactor, investigation, or follow-up.
|
||||
2. Write a short working plan in-thread when the work is nontrivial.
|
||||
3. Choose execution mode:
|
||||
- no subagents for trivial work
|
||||
- one worker for focused work
|
||||
- parallel workers only for disjoint scopes
|
||||
4. Run verification before handoff.
|
||||
|
||||
## Dispatch Rules
|
||||
|
||||
The scrum master orchestrates. Workers implement.
|
||||
|
||||
- Do not become the default implementer unless delegation is unnecessary.
|
||||
- Do not parallelize overlapping files or tightly coupled runtime work.
|
||||
- Give every worker explicit ownership of files/modules.
|
||||
- Tell every worker other agents may be active and they must not revert unrelated edits.
|
||||
- Require each worker to report:
|
||||
- changed files
|
||||
- tests run
|
||||
- blockers
|
||||
|
||||
Use worker agents for implementation and explorer agents only for bounded codebase questions.
|
||||
|
||||
## Verification
|
||||
|
||||
Every nontrivial code task gets verification.
|
||||
|
||||
Preferred flow:
|
||||
|
||||
1. use `subminer-change-verification`
|
||||
2. start with the cheapest sufficient lane
|
||||
3. escalate only when needed
|
||||
4. if worker verification is sufficient, accept it or run one final consolidating pass
|
||||
|
||||
Never hand off nontrivial work without stating what was verified and what was skipped.
|
||||
|
||||
## Pre-Handoff Policy Checks
|
||||
|
||||
Before handoff, always ask and answer both questions explicitly:
|
||||
|
||||
1. Docs update required?
|
||||
2. Changelog fragment required?
|
||||
|
||||
Rules:
|
||||
|
||||
- Do not assume silence implies "no."
|
||||
- If the answer is yes, complete the update or report the blocker.
|
||||
- Include final yes/no answers in the handoff summary even when both answers are "no."
|
||||
|
||||
## Failure / Scope Handling
|
||||
|
||||
- If a worker hits ambiguity, pause and ask the user.
|
||||
- If verification fails, either:
|
||||
- send the worker back with exact failure context, or
|
||||
- fix it directly if it is tiny and clearly in scope
|
||||
- If new scope appears, pause and re-plan before silently expanding work.
|
||||
|
||||
## Representative Flows
|
||||
|
||||
### Trivial work
|
||||
|
||||
- keep a short plan
|
||||
- implement directly or with one worker if helpful
|
||||
- run targeted verification
|
||||
- report outcome concisely
|
||||
|
||||
### Focused implementation
|
||||
|
||||
- record plan
|
||||
- dispatch one worker
|
||||
- integrate
|
||||
- verify
|
||||
- report outcome
|
||||
|
||||
### Multi-part execution
|
||||
|
||||
- define distinct deliverables/phases
|
||||
- record sequencing in the plan
|
||||
- dispatch workers only where scopes are disjoint
|
||||
- integrate
|
||||
- run consolidated verification
|
||||
- report outcome
|
||||
|
||||
## Output Expectations
|
||||
|
||||
At the end, report:
|
||||
|
||||
- which workers were dispatched and what they owned
|
||||
- what verification ran
|
||||
- explicit answers to:
|
||||
- docs update required?
|
||||
- changelog fragment required?
|
||||
- blockers, skips, and risks
|
||||
@@ -1,48 +0,0 @@
|
||||
## Highlights
|
||||
### Added
|
||||
- **In-App Changelog**
|
||||
- View release notes without leaving the app, from the tray menu ("View Changelog") or the "What's New" button on update notifications.
|
||||
- Shows notes for the latest published release even when it's newer than your installed build, and falls back to the notes bundled with your install if the download fails.
|
||||
- Older versions fold automatically, your installed version is badged, and newer ones are tagged "New"; navigate with `J`/`K` or the arrow keys, `Enter` to expand/collapse, `R` to refresh, and `Esc` to close.
|
||||
|
||||
### Changed
|
||||
- **Faster Subtitle Tokenization**
|
||||
- Subtitle lines are parsed and looked up roughly twice as efficiently, with results cached across lines so repeated words and grammar no longer re-query the dictionary.
|
||||
- Enabling a character dictionary no longer slows subtitle scanning as much, since name lookups now only check positions where a known name can actually start.
|
||||
- Fixed related accuracy issues along the way: readings that could go missing on certain word endings, subtitle text that stayed unannotated after mining a card, character names that could drop out of disambiguation rules, and halfwidth-katakana character names that weren't recognized or read correctly.
|
||||
|
||||
### Fixed
|
||||
- **Large Character Dictionary Generation**
|
||||
- Big character dictionaries (long-running series like One Piece) no longer fail to install with a timeout error; the import time budget now scales with dictionary size instead of using a fixed 7-second limit.
|
||||
- The "Generating character dictionary" notification now shows real progress (character/page counts, image download progress with an ETA, name-processing progress) and an elapsed-time clock, so a long-running import no longer looks frozen.
|
||||
- **Stats Deletion Responsiveness**
|
||||
- Deleting sessions, episodes, or library entries on the stats page no longer freezes the page or an active video player; deletes are now batched into a single transaction.
|
||||
- **Subtitle Sidebar Clutter from Styled Subtitles**
|
||||
- Heavily typeset subtitles (karaoke openings/endings, stylized signs) no longer flood the subtitle sidebar with garbled vector-drawing text or duplicate "shadow" copies of the same line.
|
||||
- Subtitle text is now decoded consistently in one place, matching what mpv actually renders on screen, so it can no longer diverge or get cached inconsistently.
|
||||
- **X11/XWayland Playback and Overlay Fixes**
|
||||
- Fixed a crash on the first fullscreen toggle when using an mpv `gpu-next` shader (e.g. ArtCNN) in X11/XWayland mode; SubMiner no longer forces mpv onto its older OpenGL renderer.
|
||||
- Fixed the overlay appearing oversized and offset from the video under fractional or mixed-monitor display scaling in X11/XWayland mode.
|
||||
|
||||
## What's Changed
|
||||
|
||||
- perf(tokenizer): single-pass Yomitan scan with cross-line caching and prefetch fixes by @ksyasuda in #185
|
||||
- fix(subtitles): collapse duplicate ASS events and decode text once by @ksyasuda in #186
|
||||
- feat(overlay): add in-app changelog modal by @ksyasuda in #187
|
||||
- fix(playback): stop forcing legacy OpenGL renderer on X11 mpv backend by @ksyasuda in #188
|
||||
- fix(dictionary): stop large character dictionaries from timing out by @ksyasuda in #189
|
||||
- fix(overlay): handle X11 display scaling across monitors by @ksyasuda in #193
|
||||
- fix(stats): batch deletes off the main thread by @ksyasuda in #194
|
||||
|
||||
## Installation
|
||||
|
||||
See the README and docs/installation guide for full setup steps.
|
||||
|
||||
## Assets
|
||||
|
||||
- Linux: `SubMiner.AppImage`
|
||||
- macOS: `SubMiner-*.dmg` and `SubMiner-*.zip`
|
||||
- Windows: `SubMiner-*.exe` and `SubMiner-*-win.zip`
|
||||
- Optional extras: `subminer-assets.tar.gz` and the `subminer` launcher
|
||||
|
||||
Note: the `subminer` wrapper script uses Bun (`#!/usr/bin/env bun`), so `bun` must be installed and on `PATH`.
|
||||
@@ -19,6 +19,7 @@ const requiredDocs = [
|
||||
'docs/knowledge-base/catalog.md',
|
||||
'docs/knowledge-base/quality.md',
|
||||
'docs/workflow/README.md',
|
||||
'docs/workflow/agent-skills.md',
|
||||
'docs/workflow/planning.md',
|
||||
'docs/workflow/verification.md',
|
||||
] as const;
|
||||
|
||||
@@ -1,202 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import test from 'node:test';
|
||||
|
||||
const repoRoot = process.cwd();
|
||||
const classifyScript = path.join(
|
||||
repoRoot,
|
||||
'.agents/skills/subminer-change-verification/scripts/classify_subminer_diff.sh',
|
||||
);
|
||||
const verifyScript = path.join(
|
||||
repoRoot,
|
||||
'.agents/skills/subminer-change-verification/scripts/verify_subminer_change.sh',
|
||||
);
|
||||
|
||||
function withTempDir<T>(fn: (dir: string) => T): T {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-change-verification-test-'));
|
||||
try {
|
||||
return fn(dir);
|
||||
} finally {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function runBash(args: string[]) {
|
||||
return spawnSync('bash', args, {
|
||||
cwd: repoRoot,
|
||||
env: process.env,
|
||||
encoding: 'utf8',
|
||||
});
|
||||
}
|
||||
|
||||
function parseArtifactDir(stdout: string): string {
|
||||
const match = stdout.match(/^artifacts: (.+)$/m);
|
||||
assert.ok(match, `expected artifact_dir in stdout, got:\n${stdout}`);
|
||||
return match[1] ?? '';
|
||||
}
|
||||
|
||||
function readSummaryJson(artifactDir: string) {
|
||||
return JSON.parse(fs.readFileSync(path.join(artifactDir, 'summary.json'), 'utf8')) as {
|
||||
sessionId: string;
|
||||
status: string;
|
||||
lanes: string[];
|
||||
blockers?: string[];
|
||||
artifactDir: string;
|
||||
pathSelectionMode?: string;
|
||||
steps: Array<{
|
||||
lane: string;
|
||||
name: string;
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
note: string;
|
||||
}>;
|
||||
};
|
||||
}
|
||||
|
||||
test('classifier marks launcher and plugin paths as real-runtime candidates', () => {
|
||||
const result = runBash([classifyScript, 'launcher/mpv.ts', 'plugin/subminer/process.lua']);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr || result.stdout);
|
||||
assert.match(result.stdout, /^lane:launcher-plugin$/m);
|
||||
assert.match(result.stdout, /^flag:real-runtime-candidate$/m);
|
||||
assert.doesNotMatch(result.stdout, /real-gui-candidate/);
|
||||
});
|
||||
|
||||
test('verifier blocks requested real-runtime lane when runtime execution is not allowed', () => {
|
||||
withTempDir((root) => {
|
||||
const artifactDir = path.join(root, 'artifacts');
|
||||
const result = runBash([
|
||||
verifyScript,
|
||||
'--dry-run',
|
||||
'--artifact-dir',
|
||||
artifactDir,
|
||||
'--lane',
|
||||
'real-runtime',
|
||||
'launcher/mpv.ts',
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stdout);
|
||||
|
||||
const summary = readSummaryJson(artifactDir);
|
||||
assert.equal(summary.status, 'blocked');
|
||||
assert.deepEqual(summary.lanes, ['real-runtime']);
|
||||
assert.ok(summary.sessionId.length > 0);
|
||||
assert.ok(summary.blockers?.some((entry) => entry.includes('--allow-real-runtime')));
|
||||
assert.equal(fs.existsSync(path.join(artifactDir, 'summary.json')), true);
|
||||
});
|
||||
});
|
||||
|
||||
test('verifier fails closed for unknown lanes', () => {
|
||||
withTempDir((root) => {
|
||||
const artifactDir = path.join(root, 'artifacts');
|
||||
const result = runBash([
|
||||
verifyScript,
|
||||
'--dry-run',
|
||||
'--artifact-dir',
|
||||
artifactDir,
|
||||
'--lane',
|
||||
'not-a-lane',
|
||||
'src/main.ts',
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stdout);
|
||||
|
||||
const summary = readSummaryJson(artifactDir);
|
||||
assert.equal(summary.status, 'blocked');
|
||||
assert.deepEqual(summary.lanes, ['not-a-lane']);
|
||||
assert.ok(summary.blockers?.some((entry) => entry.includes('unknown lane')));
|
||||
});
|
||||
});
|
||||
|
||||
test('verifier keeps non-passing step artifacts distinct across lanes', () => {
|
||||
withTempDir((root) => {
|
||||
const artifactDir = path.join(root, 'artifacts');
|
||||
const result = runBash([
|
||||
verifyScript,
|
||||
'--dry-run',
|
||||
'--artifact-dir',
|
||||
artifactDir,
|
||||
'--lane',
|
||||
'docs',
|
||||
'--lane',
|
||||
'not-a-lane',
|
||||
'src/main.ts',
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stdout);
|
||||
|
||||
const summary = readSummaryJson(artifactDir);
|
||||
const docsStep = summary.steps.find((step) => step.lane === 'docs' && step.name === 'docs-kb');
|
||||
const unknownStep = summary.steps.find(
|
||||
(step) => step.lane === 'not-a-lane' && step.name === 'unknown-lane',
|
||||
);
|
||||
|
||||
assert.ok(docsStep);
|
||||
assert.ok(unknownStep);
|
||||
assert.notEqual(docsStep?.stdout, unknownStep?.stdout);
|
||||
assert.equal(fs.existsSync(path.join(artifactDir, docsStep!.stdout)), true);
|
||||
assert.equal(fs.existsSync(path.join(artifactDir, unknownStep!.stdout)), true);
|
||||
});
|
||||
});
|
||||
|
||||
test('verifier records the real-runtime lease blocker once', () => {
|
||||
withTempDir((root) => {
|
||||
const artifactDir = path.join(root, 'artifacts');
|
||||
const leaseDir = path.join(
|
||||
repoRoot,
|
||||
'.tmp',
|
||||
'skill-verification',
|
||||
'locks',
|
||||
'exclusive-real-runtime',
|
||||
);
|
||||
fs.mkdirSync(leaseDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(leaseDir, 'session_id'), 'other-session');
|
||||
|
||||
try {
|
||||
const result = runBash([
|
||||
verifyScript,
|
||||
'--dry-run',
|
||||
'--artifact-dir',
|
||||
artifactDir,
|
||||
'--allow-real-runtime',
|
||||
'--lane',
|
||||
'real-runtime',
|
||||
'launcher/mpv.ts',
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stdout);
|
||||
|
||||
const summary = readSummaryJson(artifactDir);
|
||||
assert.deepEqual(summary.blockers, ['real-runtime lease already held by other-session']);
|
||||
} finally {
|
||||
fs.rmSync(leaseDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test('verifier allocates unique session ids and artifact roots by default', () => {
|
||||
const first = runBash([verifyScript, '--dry-run', '--lane', 'core', 'src/main.ts']);
|
||||
const second = runBash([verifyScript, '--dry-run', '--lane', 'core', 'src/main.ts']);
|
||||
|
||||
assert.equal(first.status, 0, first.stderr || first.stdout);
|
||||
assert.equal(second.status, 0, second.stderr || second.stdout);
|
||||
|
||||
const firstArtifactDir = parseArtifactDir(first.stdout);
|
||||
const secondArtifactDir = parseArtifactDir(second.stdout);
|
||||
|
||||
try {
|
||||
const firstSummary = readSummaryJson(firstArtifactDir);
|
||||
const secondSummary = readSummaryJson(secondArtifactDir);
|
||||
|
||||
assert.notEqual(firstSummary.sessionId, secondSummary.sessionId);
|
||||
assert.notEqual(firstArtifactDir, secondArtifactDir);
|
||||
assert.equal(firstSummary.pathSelectionMode, 'explicit-lanes');
|
||||
assert.equal(secondSummary.pathSelectionMode, 'explicit-lanes');
|
||||
} finally {
|
||||
fs.rmSync(firstArtifactDir, { recursive: true, force: true });
|
||||
fs.rmSync(secondArtifactDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -399,6 +399,30 @@ test('hasExplicitCommand and shouldStartApp preserve command intent', () => {
|
||||
assert.equal(statsLifetimeRebuild.statsCleanupLifetime, true);
|
||||
assert.equal(statsLifetimeRebuild.statsCleanupVocab, false);
|
||||
|
||||
assert.throws(
|
||||
() =>
|
||||
parseArgs([
|
||||
'--stats',
|
||||
'--stats-cleanup',
|
||||
'--stats-cleanup-duplicate-lines',
|
||||
'--stats-cleanup-lookback-days',
|
||||
'0.5',
|
||||
]),
|
||||
/at least one day/,
|
||||
);
|
||||
assert.equal(
|
||||
parseArgs([
|
||||
'--stats',
|
||||
'--stats-cleanup',
|
||||
'--stats-cleanup-duplicate-lines',
|
||||
'--stats-cleanup-lookback-days',
|
||||
'1.5',
|
||||
]).statsCleanupLookbackDays,
|
||||
1,
|
||||
);
|
||||
assert.equal(parseArgs(['--stats-cleanup-lookback-days=30']).statsCleanupLookbackDays, 30);
|
||||
assert.throws(() => parseArgs(['--stats-cleanup-lookback-days=30=oops']), /at least one day/);
|
||||
|
||||
const jellyfinLibraries = parseArgs(['--jellyfin-libraries']);
|
||||
assert.equal(jellyfinLibraries.jellyfinLibraries, true);
|
||||
assert.equal(hasExplicitCommand(jellyfinLibraries), true);
|
||||
|
||||
+22
-1
@@ -64,6 +64,9 @@ export interface CliArgs {
|
||||
statsCleanup?: boolean;
|
||||
statsCleanupVocab?: boolean;
|
||||
statsCleanupLifetime?: boolean;
|
||||
statsCleanupDuplicateLines?: boolean;
|
||||
statsCleanupDryRun?: boolean;
|
||||
statsCleanupLookbackDays?: number;
|
||||
statsResponsePath?: string;
|
||||
jellyfin: boolean;
|
||||
jellyfinLogin: boolean;
|
||||
@@ -109,6 +112,14 @@ export interface CliArgs {
|
||||
|
||||
export type CliCommandSource = 'initial' | 'second-instance';
|
||||
|
||||
function parseStatsCleanupLookbackDays(value: string | undefined): number {
|
||||
const days = Number(value);
|
||||
if (!Number.isFinite(days) || days < 1) {
|
||||
throw new Error('Stats --lookback-days must be at least one day.');
|
||||
}
|
||||
return Math.floor(days);
|
||||
}
|
||||
|
||||
export function parseArgs(argv: string[]): CliArgs {
|
||||
const args: CliArgs = {
|
||||
background: false,
|
||||
@@ -167,6 +178,8 @@ export function parseArgs(argv: string[]): CliArgs {
|
||||
statsCleanup: false,
|
||||
statsCleanupVocab: false,
|
||||
statsCleanupLifetime: false,
|
||||
statsCleanupDuplicateLines: false,
|
||||
statsCleanupDryRun: false,
|
||||
jellyfin: false,
|
||||
jellyfinLogin: false,
|
||||
jellyfinLogout: false,
|
||||
@@ -368,7 +381,15 @@ export function parseArgs(argv: string[]): CliArgs {
|
||||
} else if (arg === '--stats-cleanup') args.statsCleanup = true;
|
||||
else if (arg === '--stats-cleanup-vocab') args.statsCleanupVocab = true;
|
||||
else if (arg === '--stats-cleanup-lifetime') args.statsCleanupLifetime = true;
|
||||
else if (arg.startsWith('--stats-response-path=')) {
|
||||
else if (arg === '--stats-cleanup-duplicate-lines') args.statsCleanupDuplicateLines = true;
|
||||
else if (arg === '--stats-cleanup-dry-run') args.statsCleanupDryRun = true;
|
||||
else if (arg.startsWith('--stats-cleanup-lookback-days=')) {
|
||||
args.statsCleanupLookbackDays = parseStatsCleanupLookbackDays(
|
||||
arg.slice('--stats-cleanup-lookback-days='.length),
|
||||
);
|
||||
} else if (arg === '--stats-cleanup-lookback-days') {
|
||||
args.statsCleanupLookbackDays = parseStatsCleanupLookbackDays(readValue(argv[i + 1]));
|
||||
} else if (arg.startsWith('--stats-response-path=')) {
|
||||
const value = arg.split('=', 2)[1];
|
||||
if (value) args.statsResponsePath = value;
|
||||
} else if (arg === '--stats-response-path') {
|
||||
|
||||
@@ -1,322 +0,0 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import type { DatabaseSync } from '../immersion-tracker/sqlite';
|
||||
|
||||
type ImmersionTrackerService = import('../immersion-tracker-service').ImmersionTrackerService;
|
||||
type ImmersionTrackerServiceCtor =
|
||||
typeof import('../immersion-tracker-service').ImmersionTrackerService;
|
||||
|
||||
let trackerCtor: ImmersionTrackerServiceCtor | null = null;
|
||||
|
||||
async function loadTrackerCtor(): Promise<ImmersionTrackerServiceCtor> {
|
||||
if (trackerCtor) return trackerCtor;
|
||||
const mod = await import('../immersion-tracker-service');
|
||||
trackerCtor = mod.ImmersionTrackerService;
|
||||
return trackerCtor;
|
||||
}
|
||||
|
||||
function makeDbPath(): string {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-write-queue-test-'));
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) return;
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
interface TrackerInternals {
|
||||
db: DatabaseSync;
|
||||
queue: unknown[];
|
||||
recordWrite: (write: Record<string, unknown>) => void;
|
||||
deleteSession: (sessionId: number) => Promise<void>;
|
||||
mergeAnime: (targetAnimeId: number, sourceAnimeIds: number[]) => Promise<unknown>;
|
||||
moveVideoToAnime: (videoId: number, targetAnimeId: number) => Promise<unknown>;
|
||||
rebuildLifetimeSummaries: () => Promise<unknown>;
|
||||
reassignAnimeAnilist: (animeId: number, info: { anilistId: number }) => Promise<void>;
|
||||
flushNow: () => void;
|
||||
writeLock: { locked: boolean };
|
||||
}
|
||||
|
||||
test('delete maintenance fails closed when queued writes cannot drain', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
let deleteRunnerCalls = 0;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor(
|
||||
{ dbPath, policy: { batchSize: 2 } },
|
||||
{
|
||||
runDeleteMaintenanceTask: async () => {
|
||||
deleteRunnerCalls += 1;
|
||||
},
|
||||
},
|
||||
);
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 1);
|
||||
let flushCalls = 0;
|
||||
internals.flushNow = () => {
|
||||
flushCalls += 1;
|
||||
if (flushCalls > 1) throw new Error('bounded no-progress sentinel');
|
||||
};
|
||||
|
||||
await assert.rejects(internals.deleteSession(1), /queue did not drain/i);
|
||||
|
||||
assert.equal(flushCalls, 1);
|
||||
assert.equal(deleteRunnerCalls, 0);
|
||||
assert.equal(internals.writeLock.locked, false);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('reassignAnimeAnilist fails closed before resolving a conflict when writes cannot drain', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
seedTwoEntries(internals.db);
|
||||
internals.db.prepare('UPDATE imm_anime SET anilist_id = 123 WHERE anime_id = 2').run();
|
||||
queueSubtitleLines(internals, 1);
|
||||
internals.flushNow = () => {};
|
||||
|
||||
await assert.rejects(
|
||||
internals.reassignAnimeAnilist(1, { anilistId: 123 }),
|
||||
/queue did not drain/i,
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
internals.db
|
||||
.prepare(
|
||||
'SELECT anime_id AS animeId, anilist_id AS anilistId FROM imm_anime ORDER BY anime_id',
|
||||
)
|
||||
.all(),
|
||||
[
|
||||
{ animeId: 1, anilistId: null },
|
||||
{ animeId: 2, anilistId: 123 },
|
||||
],
|
||||
);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('mergeAnime fails closed when queued writes cannot drain', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 1);
|
||||
internals.flushNow = () => {};
|
||||
|
||||
await assert.rejects(internals.mergeAnime(1, [2]), /queue did not drain/i);
|
||||
|
||||
assert.deepEqual(
|
||||
internals.db
|
||||
.prepare('SELECT anime_id AS animeId FROM imm_anime ORDER BY anime_id')
|
||||
.all()
|
||||
.map((row) => (row as { animeId: number }).animeId),
|
||||
[1, 2],
|
||||
);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('moveVideoToAnime fails closed when queued writes cannot drain', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 1);
|
||||
internals.flushNow = () => {};
|
||||
|
||||
await assert.rejects(internals.moveVideoToAnime(2, 1), /queue did not drain/i);
|
||||
assert.equal(
|
||||
(
|
||||
internals.db
|
||||
.prepare('SELECT anime_id AS animeId FROM imm_videos WHERE video_id = 2')
|
||||
.get() as {
|
||||
animeId: number;
|
||||
}
|
||||
).animeId,
|
||||
2,
|
||||
);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('rebuildLifetimeSummaries fails closed when queued writes cannot drain', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 1);
|
||||
internals.flushNow = () => {};
|
||||
|
||||
await assert.rejects(internals.rebuildLifetimeSummaries(), /queue did not drain/i);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
function seedTwoEntries(db: DatabaseSync): void {
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime (anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'show', 'Show', 1000, 1000), (2, 'show season 1', 'Show Season 1', 1000, 1000);
|
||||
INSERT INTO imm_videos (video_id, video_key, canonical_title, anime_id, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'local:/tmp/a.mkv', 'A', 1, 1, 0, 1440000, 1000, 1000),
|
||||
(2, 'local:/tmp/b.mkv', 'B', 2, 1, 0, 1440000, 1000, 1000);
|
||||
INSERT INTO imm_sessions (session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, active_watched_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'drain-session', 2, '1000', '2000', 2, 1000, 1000, 2000);
|
||||
`);
|
||||
}
|
||||
|
||||
function queueSubtitleLines(tracker: TrackerInternals, count: number): void {
|
||||
for (let index = 0; index < count; index += 1) {
|
||||
tracker.recordWrite({
|
||||
kind: 'subtitleLine',
|
||||
sessionId: 1,
|
||||
videoId: 2,
|
||||
lineIndex: index,
|
||||
segmentStartMs: index * 1000,
|
||||
segmentEndMs: index * 1000 + 900,
|
||||
text: `line ${index}`,
|
||||
wordOccurrences: [],
|
||||
kanjiOccurrences: [],
|
||||
firstSeen: 1000,
|
||||
lastSeen: 2000,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Queued last so it sits past the first batch. Lifetime `total_lines_seen`
|
||||
* reads this counter, not a COUNT over imm_subtitle_lines, so the rebuilt
|
||||
* summary only reflects the session once the queue is drained all the way.
|
||||
*/
|
||||
function queueTelemetry(tracker: TrackerInternals, linesSeen: number): void {
|
||||
tracker.recordWrite({
|
||||
kind: 'telemetry',
|
||||
sessionId: 1,
|
||||
sampleMs: 3000,
|
||||
lastMediaMs: 3000,
|
||||
totalWatchedMs: 4000,
|
||||
activeWatchedMs: 3500,
|
||||
linesSeen,
|
||||
tokensSeen: linesSeen * 5,
|
||||
cardsMined: 2,
|
||||
lookupCount: 0,
|
||||
lookupHits: 0,
|
||||
yomitanLookupCount: 0,
|
||||
pauseCount: 0,
|
||||
pauseMs: 0,
|
||||
seekForwardCount: 0,
|
||||
seekBackwardCount: 0,
|
||||
mediaBufferEvents: 0,
|
||||
});
|
||||
}
|
||||
|
||||
/** The queued telemetry sample only exists in the database once the queue drained fully. */
|
||||
function latestTelemetryLinesSeen(db: DatabaseSync, sessionId: number): number | null {
|
||||
const row = db
|
||||
.prepare(
|
||||
`SELECT lines_seen AS linesSeen
|
||||
FROM imm_session_telemetry
|
||||
WHERE session_id = ?
|
||||
ORDER BY sample_ms DESC, telemetry_id DESC
|
||||
LIMIT 1`,
|
||||
)
|
||||
.get(sessionId) as { linesSeen: number } | undefined;
|
||||
return row ? Number(row.linesSeen) : null;
|
||||
}
|
||||
|
||||
function countLinesForAnime(db: DatabaseSync, animeId: number): number {
|
||||
const row = db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_subtitle_lines WHERE anime_id = ?')
|
||||
.get(animeId) as { total: number };
|
||||
return Number(row.total);
|
||||
}
|
||||
|
||||
/**
|
||||
* Both entry points must see a settled database before changing episode
|
||||
* ownership. A single flushNow() only writes one batch off the front of the
|
||||
* queue, so anything past `batchSize` would still be unwritten when the merge
|
||||
* repoints rows.
|
||||
*/
|
||||
test('mergeAnime drains a queue larger than one batch before repointing rows', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 8);
|
||||
queueTelemetry(internals, 8);
|
||||
assert.ok(internals.queue.length > 2, 'expected more queued writes than one batch');
|
||||
|
||||
await internals.mergeAnime(1, [2]);
|
||||
|
||||
assert.equal(internals.queue.length, 0);
|
||||
// Every queued line landed, attributed to the surviving entry.
|
||||
assert.equal(countLinesForAnime(internals.db, 1), 8);
|
||||
assert.equal(latestTelemetryLinesSeen(internals.db, 1), 8);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('moveVideoToAnime drains a queue larger than one batch before repointing rows', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath, policy: { batchSize: 2 } });
|
||||
const internals = tracker as unknown as TrackerInternals;
|
||||
|
||||
seedTwoEntries(internals.db);
|
||||
queueSubtitleLines(internals, 8);
|
||||
queueTelemetry(internals, 8);
|
||||
|
||||
await internals.moveVideoToAnime(2, 1);
|
||||
|
||||
assert.equal(internals.queue.length, 0);
|
||||
assert.equal(countLinesForAnime(internals.db, 1), 8);
|
||||
assert.equal(latestTelemetryLinesSeen(internals.db, 1), 8);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
@@ -1032,6 +1032,180 @@ describe('stats server API routes', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it('POST /api/stats/maintenance/duplicate-lines forwards the window and dry-run flag', async () => {
|
||||
let seenOptions: unknown = null;
|
||||
const summary = {
|
||||
dryRun: true,
|
||||
lookbackDays: 30,
|
||||
scannedLines: 900,
|
||||
burstGroups: 2,
|
||||
removedLines: 180,
|
||||
removedWordOccurrences: 540,
|
||||
removedKanjiOccurrences: 120,
|
||||
samples: [],
|
||||
};
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async (options: unknown) => {
|
||||
seenOptions = options;
|
||||
return summary;
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ dryRun: true, lookbackDays: 30 }),
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(await res.json(), summary);
|
||||
assert.deepEqual(seenOptions, { dryRun: true, lookbackDays: 30 });
|
||||
});
|
||||
|
||||
it('POST /api/stats/maintenance/duplicate-lines rejects cross-origin simple requests', async () => {
|
||||
let cleanupCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async () => {
|
||||
cleanupCalls += 1;
|
||||
throw new Error('cleanup must not run');
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'text/plain',
|
||||
Origin: 'https://attacker.example',
|
||||
},
|
||||
body: JSON.stringify({ dryRun: false, lookbackDays: null }),
|
||||
});
|
||||
|
||||
assert.equal(res.status, 415);
|
||||
assert.equal(cleanupCalls, 0);
|
||||
});
|
||||
|
||||
it('POST /api/stats/maintenance/duplicate-lines rejects a window shorter than a day', async () => {
|
||||
let cleanupCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async () => {
|
||||
cleanupCalls += 1;
|
||||
return {
|
||||
dryRun: true,
|
||||
lookbackDays: null,
|
||||
scannedLines: 0,
|
||||
burstGroups: 0,
|
||||
removedLines: 0,
|
||||
removedWordOccurrences: 0,
|
||||
removedKanjiOccurrences: 0,
|
||||
samples: [],
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ dryRun: true, lookbackDays: 0.5 }),
|
||||
});
|
||||
|
||||
assert.equal(res.status, 400);
|
||||
assert.equal(cleanupCalls, 0);
|
||||
});
|
||||
|
||||
it('POST /api/stats/maintenance/duplicate-lines floors a fractional multi-day window', async () => {
|
||||
let seenOptions: unknown = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async (options: unknown) => {
|
||||
seenOptions = options;
|
||||
return {
|
||||
dryRun: true,
|
||||
lookbackDays: 1,
|
||||
scannedLines: 0,
|
||||
burstGroups: 0,
|
||||
removedLines: 0,
|
||||
removedWordOccurrences: 0,
|
||||
removedKanjiOccurrences: 0,
|
||||
samples: [],
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ dryRun: true, lookbackDays: 1.5 }),
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(seenOptions, { dryRun: true, lookbackDays: 1 });
|
||||
});
|
||||
|
||||
it('POST /api/stats/maintenance/duplicate-lines accepts an explicit empty object for all history', async () => {
|
||||
let seenOptions: unknown = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async (options: unknown) => {
|
||||
seenOptions = options;
|
||||
return {
|
||||
dryRun: false,
|
||||
lookbackDays: null,
|
||||
scannedLines: 0,
|
||||
burstGroups: 0,
|
||||
removedLines: 0,
|
||||
removedWordOccurrences: 0,
|
||||
removedKanjiOccurrences: 0,
|
||||
samples: [],
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: '{}',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(seenOptions, { dryRun: false, lookbackDays: null });
|
||||
});
|
||||
|
||||
for (const malformed of [
|
||||
{ name: 'a missing body', body: undefined },
|
||||
{ name: 'malformed JSON', body: '{' },
|
||||
{ name: 'JSON null', body: 'null' },
|
||||
{ name: 'a JSON array', body: '[]' },
|
||||
]) {
|
||||
it(`POST /api/stats/maintenance/duplicate-lines rejects ${malformed.name}`, async () => {
|
||||
let cleanupCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
cleanupDuplicateSubtitleLines: async () => {
|
||||
cleanupCalls += 1;
|
||||
throw new Error('cleanup must not run');
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/maintenance/duplicate-lines', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: malformed.body,
|
||||
});
|
||||
|
||||
assert.equal(res.status, 400);
|
||||
assert.equal(cleanupCalls, 0);
|
||||
});
|
||||
}
|
||||
|
||||
it('PUT /api/stats/excluded-words rejects malformed rows', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
|
||||
@@ -1053,55 +1227,6 @@ describe('stats server API routes', () => {
|
||||
assert.equal(body[0].canonicalTitle, 'Little Witch Academia');
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/merge-recommendations returns pending duplicate pairs', async () => {
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getAnimeMergeRecommendations: async () => [{ recommendationId: 4, animeIds: [1, 2] }],
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/merge-recommendations');
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(await res.json(), {
|
||||
recommendations: [{ recommendationId: 4, animeIds: [1, 2] }],
|
||||
});
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/merge-recommendations/:id dismisses a pending pair', async () => {
|
||||
let dismissedId: number | null = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
dismissAnimeMergeRecommendation: async (recommendationId: number) => {
|
||||
dismissedId = recommendationId;
|
||||
return true;
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/merge-recommendations/4', {
|
||||
method: 'DELETE',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(dismissedId, 4);
|
||||
assert.deepEqual(await res.json(), { ok: true });
|
||||
});
|
||||
|
||||
it('DELETE /api/stats/anime/merge-recommendations/:id reports missing recommendations', async () => {
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
dismissAnimeMergeRecommendation: async () => false,
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/merge-recommendations/99', {
|
||||
method: 'DELETE',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 404);
|
||||
});
|
||||
|
||||
it('GET /api/stats/anime/:animeId returns anime detail with episodes', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/anime/1');
|
||||
@@ -3073,148 +3198,6 @@ Aligned English subtitle
|
||||
assert.equal(deleteCalls, 0);
|
||||
});
|
||||
|
||||
it('POST /api/stats/anime/:animeId/merge folds the given entries into the target', async () => {
|
||||
let merged: { targetAnimeId: number; sourceAnimeIds: number[] } | null = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
mergeAnime: async (targetAnimeId: number, sourceAnimeIds: number[]) => {
|
||||
merged = { targetAnimeId, sourceAnimeIds };
|
||||
return {
|
||||
survivingAnimeId: targetAnimeId,
|
||||
mergedAnimeIds: sourceAnimeIds,
|
||||
movedVideos: 3,
|
||||
};
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/7/merge', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
// The target repeated in the sources must not delete the entry we keep.
|
||||
body: '{"sourceAnimeIds":[8,9,8,7]}',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(merged, { targetAnimeId: 7, sourceAnimeIds: [8, 9] });
|
||||
assert.deepEqual(await res.json(), {
|
||||
ok: true,
|
||||
animeId: 7,
|
||||
mergedAnimeIds: [8, 9],
|
||||
movedVideos: 3,
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/anime/:animeId/merge rejects an empty or malformed source list', async () => {
|
||||
let mergeCalls = 0;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
mergeAnime: async () => {
|
||||
mergeCalls += 1;
|
||||
return { survivingAnimeId: 7, mergedAnimeIds: [], movedVideos: 0 };
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
for (const body of [
|
||||
'{"sourceAnimeIds":[]}',
|
||||
'{"sourceAnimeIds":[7]}',
|
||||
'{"sourceAnimeIds":0}',
|
||||
]) {
|
||||
const res = await app.request('/api/stats/anime/7/merge', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body,
|
||||
});
|
||||
assert.equal(res.status, 400);
|
||||
}
|
||||
assert.equal(mergeCalls, 0);
|
||||
});
|
||||
|
||||
it('PATCH /api/stats/media/:videoId/anime moves the episode to another entry', async () => {
|
||||
let moved: { videoId: number; animeId: number } | null = null;
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
moveVideoToAnime: async (videoId: number, animeId: number) => {
|
||||
moved = { videoId, animeId };
|
||||
return { targetAnimeId: animeId, previousAnimeId: 4, removedPreviousAnime: true };
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/media/12/anime', {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: '{"animeId":7}',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.deepEqual(moved, { videoId: 12, animeId: 7 });
|
||||
assert.deepEqual(await res.json(), {
|
||||
ok: true,
|
||||
animeId: 7,
|
||||
previousAnimeId: 4,
|
||||
removedPreviousAnime: true,
|
||||
});
|
||||
});
|
||||
|
||||
it('POST /api/stats/anime/:animeId/merge reports a merge that folded nothing as 404', async () => {
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
mergeAnime: async (targetAnimeId: number) => ({
|
||||
survivingAnimeId: targetAnimeId,
|
||||
mergedAnimeIds: [],
|
||||
movedVideos: 0,
|
||||
}),
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/anime/7/merge', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: '{"sourceAnimeIds":[8]}',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 404);
|
||||
});
|
||||
|
||||
it('PATCH /api/stats/media/:videoId/anime reports an unknown target as 404', async () => {
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
moveVideoToAnime: async () => {
|
||||
throw new Error('Unknown episode or target library entry');
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/media/12/anime', {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: '{"animeId":99}',
|
||||
});
|
||||
|
||||
assert.equal(res.status, 404);
|
||||
});
|
||||
|
||||
it('PATCH /api/stats/media/:videoId/anime does not disguise storage failures as 404', async () => {
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
moveVideoToAnime: async () => {
|
||||
throw new Error('database is locked');
|
||||
},
|
||||
} as Partial<ImmersionTrackerService>),
|
||||
);
|
||||
|
||||
const res = await app.request('/api/stats/media/12/anime', {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: '{"animeId":7}',
|
||||
});
|
||||
|
||||
assert.notEqual(res.status, 404);
|
||||
assert.equal(res.status >= 500, true);
|
||||
});
|
||||
|
||||
it('POST /api/stats/anki/browse returns 400 for missing noteId', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/anki/browse', { method: 'POST' });
|
||||
|
||||
@@ -327,7 +327,6 @@ export function createCoverArtFetcher(
|
||||
titleEnglish: selected.title?.english ?? null,
|
||||
titleNative: selected.title?.native ?? null,
|
||||
episodesTotal: selected.episodes ?? null,
|
||||
exactTitleMatch: resolution?.exactTitleMatch ?? false,
|
||||
});
|
||||
|
||||
logger.info(
|
||||
|
||||
@@ -156,79 +156,6 @@ test('season 1 resolves to the anchor without relation lookups', async () => {
|
||||
assert.deepEqual(relationLookups, []);
|
||||
});
|
||||
|
||||
test('a sequel resolution is not certified by the anchor exact-title evidence', async () => {
|
||||
// The anchor matched the search title exactly, but the hopped-to entry is a
|
||||
// different inference (a split-cour chain can land one season short), so the
|
||||
// sequel result must report its own title evidence, not the anchor's.
|
||||
const { execute } = createExecutor(OREGAIRU_SEARCH, OREGAIRU_RELATIONS);
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'My Teen Romantic Comedy SNAFU', season: 2, episode: 1 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 20698);
|
||||
assert.equal(result?.via, 'sequel-chain');
|
||||
assert.equal(result?.exactTitleMatch, false);
|
||||
});
|
||||
|
||||
test('a sequel resolution whose own title matches the parsed title stays exact', async () => {
|
||||
const anchor: AnilistSeasonMedia = {
|
||||
id: 1,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
title: { english: 'Show' },
|
||||
};
|
||||
const sequel: AnilistSeasonMedia = {
|
||||
id: 2,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
title: { english: 'Show 2nd Season' },
|
||||
};
|
||||
const { execute } = createExecutor([anchor], {
|
||||
1: [{ relationType: 'SEQUEL', node: sequel }],
|
||||
});
|
||||
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
{ title: 'Show 2nd Season', season: 2, episode: 1 },
|
||||
{ execute },
|
||||
);
|
||||
|
||||
assert.equal(result?.id, 2);
|
||||
assert.equal(result?.via, 'sequel-chain');
|
||||
assert.equal(result?.exactTitleMatch, true);
|
||||
});
|
||||
|
||||
test('reports an exact normalized synonym match as strong evidence', async () => {
|
||||
const { execute } = createExecutor([
|
||||
{
|
||||
id: 1,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
title: { english: 'Hitori Gotoh Story' },
|
||||
synonyms: ['BOCCHI THE ROCK'],
|
||||
},
|
||||
]);
|
||||
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Bocchi the Rock!' }, { execute });
|
||||
|
||||
assert.equal(result?.exactTitleMatch, true);
|
||||
});
|
||||
|
||||
test('reports a fuzzy-only search result as weak evidence', async () => {
|
||||
const { execute } = createExecutor([
|
||||
{
|
||||
id: 1,
|
||||
episodes: 12,
|
||||
format: 'TV',
|
||||
title: { english: 'Actual Show' },
|
||||
},
|
||||
]);
|
||||
|
||||
const result = await resolveAnilistSeasonMedia({ title: 'Unrelated Release' }, { execute });
|
||||
|
||||
assert.equal(result?.exactTitleMatch, false);
|
||||
});
|
||||
|
||||
test('strips a season marker already present in the parsed title', async () => {
|
||||
const { execute, searches } = createExecutor(OREGAIRU_SEARCH, OREGAIRU_RELATIONS);
|
||||
const result = await resolveAnilistSeasonMedia(
|
||||
|
||||
@@ -9,8 +9,6 @@
|
||||
* reports `seasonResolved: false` so callers can refuse to act instead of guessing.
|
||||
*/
|
||||
|
||||
import { normalizeTitleIdentity } from '../../utils/title-normalization';
|
||||
|
||||
export interface AnilistSeasonMediaTitle {
|
||||
romaji?: string | null;
|
||||
english?: string | null;
|
||||
@@ -44,8 +42,6 @@ export interface AnilistSeasonResolution {
|
||||
seasonResolved: boolean;
|
||||
requestedSeason: number | null;
|
||||
via: AnilistSeasonResolutionVia;
|
||||
/** Exact normalized match against an AniList title or synonym. */
|
||||
exactTitleMatch: boolean;
|
||||
}
|
||||
|
||||
export interface ResolveAnilistSeasonMediaInput {
|
||||
@@ -119,6 +115,10 @@ const SEASONAL_FORMAT_PRIORITY = ['TV', 'TV_SHORT', 'ONA'];
|
||||
|
||||
const MAX_SEQUEL_HOPS = 12;
|
||||
|
||||
function normalizeTitle(value: string): string {
|
||||
return value.trim().toLowerCase().replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops season markers a release name carries but AniList titles never do,
|
||||
* so "Some Show Season 3" and "Some Show S3" both search as "Some Show".
|
||||
@@ -136,7 +136,7 @@ function mediaTitles(media: AnilistSeasonMedia): string[] {
|
||||
const synonyms = Array.isArray(media.synonyms) ? media.synonyms : [];
|
||||
return [media.title?.english, media.title?.romaji, media.title?.native, ...synonyms]
|
||||
.filter((value): value is string => typeof value === 'string' && value.trim().length > 0)
|
||||
.map((value) => normalizeTitleIdentity(value));
|
||||
.map((value) => normalizeTitle(value));
|
||||
}
|
||||
|
||||
function displayTitle(media: AnilistSeasonMedia, fallback: string): string {
|
||||
@@ -176,7 +176,6 @@ function toResolution(
|
||||
season: number | null,
|
||||
via: AnilistSeasonResolutionVia,
|
||||
seasonResolved: boolean,
|
||||
exactTitleMatch: boolean,
|
||||
): AnilistSeasonResolution {
|
||||
return {
|
||||
id: media.id,
|
||||
@@ -186,7 +185,6 @@ function toResolution(
|
||||
seasonResolved,
|
||||
requestedSeason: season,
|
||||
via,
|
||||
exactTitleMatch,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -211,10 +209,9 @@ export function pickAnchorMedia(
|
||||
: media;
|
||||
const pool = episodeFiltered.length > 0 ? episodeFiltered : media;
|
||||
|
||||
const targets = [
|
||||
normalizeTitleIdentity(title),
|
||||
normalizeTitleIdentity(stripSeasonSuffix(title)),
|
||||
].filter((value, index, all) => value.length > 0 && all.indexOf(value) === index);
|
||||
const targets = [normalizeTitle(title), normalizeTitle(stripSeasonSuffix(title))].filter(
|
||||
(value, index, all) => value.length > 0 && all.indexOf(value) === index,
|
||||
);
|
||||
|
||||
const scored = pool.map((entry, index) => {
|
||||
const candidateTitles = mediaTitles(entry);
|
||||
@@ -370,20 +367,9 @@ export async function resolveAnilistSeasonMedia(
|
||||
episode: season === null || season <= 1 ? input.episode : null,
|
||||
});
|
||||
if (!anchor) return null;
|
||||
// Certifies the media actually returned, never the anchor on its behalf: a
|
||||
// sequel-chain hop can land one season short (split-cour entries) while the
|
||||
// anchor title still matches perfectly, and that certainty must not carry
|
||||
// over to the hopped-to entry.
|
||||
const exactMatchFor = (candidate: AnilistSeasonMedia): boolean => {
|
||||
const titles = mediaTitles(candidate);
|
||||
return (
|
||||
titles.includes(normalizeTitleIdentity(searchTitle)) ||
|
||||
titles.includes(normalizeTitleIdentity(input.title))
|
||||
);
|
||||
};
|
||||
|
||||
if (season === null || season <= 1) {
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', true, exactMatchFor(anchor));
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', true);
|
||||
}
|
||||
|
||||
let chainError: unknown = null;
|
||||
@@ -397,14 +383,7 @@ export async function resolveAnilistSeasonMedia(
|
||||
deps.logInfo?.(
|
||||
`[anilist] season ${season} of "${searchTitle}" resolved via sequel chain: ${displayTitle(viaChain, searchTitle)} (${viaChain.id})`,
|
||||
);
|
||||
return toResolution(
|
||||
viaChain,
|
||||
searchTitle,
|
||||
season,
|
||||
'sequel-chain',
|
||||
true,
|
||||
exactMatchFor(viaChain),
|
||||
);
|
||||
return toResolution(viaChain, searchTitle, season, 'sequel-chain', true);
|
||||
}
|
||||
|
||||
const viaAirOrder = pickByAirOrder(anchor, season, media);
|
||||
@@ -412,14 +391,7 @@ export async function resolveAnilistSeasonMedia(
|
||||
deps.logInfo?.(
|
||||
`[anilist] season ${season} of "${searchTitle}" resolved via air order: ${displayTitle(viaAirOrder, searchTitle)} (${viaAirOrder.id})`,
|
||||
);
|
||||
return toResolution(
|
||||
viaAirOrder,
|
||||
searchTitle,
|
||||
season,
|
||||
'air-order',
|
||||
true,
|
||||
exactMatchFor(viaAirOrder),
|
||||
);
|
||||
return toResolution(viaAirOrder, searchTitle, season, 'air-order', true);
|
||||
}
|
||||
|
||||
// The chain failed for transport reasons rather than because the season is absent;
|
||||
@@ -431,5 +403,5 @@ export async function resolveAnilistSeasonMedia(
|
||||
deps.logInfo?.(
|
||||
`[anilist] could not resolve season ${season} of "${searchTitle}"; falling back to ${displayTitle(anchor, searchTitle)} (${anchor.id})`,
|
||||
);
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', false, exactMatchFor(anchor));
|
||||
return toResolution(anchor, searchTitle, season, 'anchor', false);
|
||||
}
|
||||
|
||||
@@ -2132,78 +2132,6 @@ test('handleMediaChange reuses the same provisional anime row across matching fi
|
||||
}
|
||||
});
|
||||
|
||||
test('local parsing reuses a unique compatible manual assignment from the same directory', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath });
|
||||
const anchorPath = '/tmp/grouped/Incorrect Name S01E01.mkv';
|
||||
tracker.handleMediaChange(anchorPath, 'Episode 1');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
|
||||
const privateApi = tracker as unknown as {
|
||||
db: DatabaseSync;
|
||||
sessionState: { videoId: number } | null;
|
||||
};
|
||||
const anchorVideoId = privateApi.sessionState?.videoId;
|
||||
assert.ok(anchorVideoId);
|
||||
tracker.handleMediaChange(null, null);
|
||||
const timestamp = toDbTimestamp(trackerNowMs());
|
||||
const target = privateApi.db
|
||||
.prepare(
|
||||
`
|
||||
INSERT INTO imm_anime (
|
||||
normalized_title_key,
|
||||
canonical_title,
|
||||
CREATED_DATE,
|
||||
LAST_UPDATE_DATE
|
||||
) VALUES ('correct show season 1', 'Correct Show Season 1', ?, ?)
|
||||
RETURNING anime_id AS animeId
|
||||
`,
|
||||
)
|
||||
.get(timestamp, timestamp) as { animeId: number };
|
||||
await tracker.moveVideoToAnime(anchorVideoId, target.animeId);
|
||||
|
||||
tracker.handleMediaChange(anchorPath, 'Episode 1');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
tracker.handleMediaChange('/tmp/grouped/Another Wrong Name S01E02.mkv', 'Episode 2');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
tracker.handleMediaChange('/tmp/grouped/Another Wrong Name S02E01.mkv', 'Episode 1');
|
||||
await waitForPendingAnimeMetadata(tracker);
|
||||
|
||||
const rows = privateApi.db
|
||||
.prepare(
|
||||
`
|
||||
SELECT source_path AS sourcePath, anime_id AS animeId, anime_assignment_locked AS locked
|
||||
FROM imm_videos
|
||||
WHERE source_path LIKE '/tmp/grouped/%'
|
||||
ORDER BY source_path
|
||||
`,
|
||||
)
|
||||
.all() as Array<{ sourcePath: string; animeId: number; locked: number }>;
|
||||
const assignments = new Map(rows.map((row) => [row.sourcePath, row]));
|
||||
assert.deepEqual(assignments.get(anchorPath), {
|
||||
sourcePath: anchorPath,
|
||||
animeId: target.animeId,
|
||||
locked: 1,
|
||||
});
|
||||
assert.deepEqual(assignments.get('/tmp/grouped/Another Wrong Name S01E02.mkv'), {
|
||||
sourcePath: '/tmp/grouped/Another Wrong Name S01E02.mkv',
|
||||
animeId: target.animeId,
|
||||
locked: 0,
|
||||
});
|
||||
assert.notEqual(
|
||||
assignments.get('/tmp/grouped/Another Wrong Name S02E01.mkv')?.animeId,
|
||||
target.animeId,
|
||||
);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('handleMediaChange splits matching parsed titles across distinct seasons', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
@@ -2690,67 +2618,6 @@ test('Jellyfin playback metadata links stream videos to existing series title',
|
||||
}
|
||||
});
|
||||
|
||||
test('Jellyfin metadata refresh preserves a manual episode assignment', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
|
||||
try {
|
||||
const Ctor = await loadTrackerCtor();
|
||||
tracker = new Ctor({ dbPath });
|
||||
const metadata = {
|
||||
mediaPath: 'http://jellyfin.local/Videos/item-locked/stream?api_key=token',
|
||||
displayTitle: 'Parsed Show S01E01',
|
||||
itemTitle: 'Episode 1',
|
||||
seriesTitle: 'Parsed Show',
|
||||
seasonNumber: 1,
|
||||
episodeNumber: 1,
|
||||
itemId: 'item-locked',
|
||||
};
|
||||
tracker.recordJellyfinPlaybackMetadata(metadata);
|
||||
|
||||
const privateApi = tracker as unknown as { db: DatabaseSync };
|
||||
const video = privateApi.db.prepare('SELECT video_id AS videoId FROM imm_videos').get() as {
|
||||
videoId: number;
|
||||
};
|
||||
const timestamp = toDbTimestamp(trackerNowMs());
|
||||
const target = privateApi.db
|
||||
.prepare(
|
||||
`
|
||||
INSERT INTO imm_anime (
|
||||
normalized_title_key,
|
||||
canonical_title,
|
||||
CREATED_DATE,
|
||||
LAST_UPDATE_DATE
|
||||
) VALUES ('correct show', 'Correct Show', ?, ?)
|
||||
RETURNING anime_id AS animeId
|
||||
`,
|
||||
)
|
||||
.get(timestamp, timestamp) as { animeId: number };
|
||||
|
||||
await tracker.moveVideoToAnime(video.videoId, target.animeId);
|
||||
tracker.recordJellyfinPlaybackMetadata(metadata);
|
||||
|
||||
const assignment = privateApi.db
|
||||
.prepare(
|
||||
`
|
||||
SELECT anime_id AS animeId, anime_assignment_locked AS locked
|
||||
FROM imm_videos
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
)
|
||||
.get(video.videoId) as { animeId: number; locked: number };
|
||||
assert.equal(assignment.animeId, target.animeId);
|
||||
assert.equal(assignment.locked, 1);
|
||||
const animeCount = privateApi.db.prepare('SELECT COUNT(*) AS count FROM imm_anime').get() as {
|
||||
count: number;
|
||||
};
|
||||
assert.equal(animeCount.count, 1);
|
||||
} finally {
|
||||
tracker?.destroy();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('startup repairs existing Jellyfin stream video links to metadata rows', async () => {
|
||||
const dbPath = makeDbPath();
|
||||
let tracker: ImmersionTrackerService | null = null;
|
||||
@@ -3937,24 +3804,10 @@ test('reassignAnimeAnilist redistributes conflicting legacy combined row before
|
||||
(1, 2000, 1000, 1000, 1, 10, 0, 0, 0, 0, 0, 0, 0, 0),
|
||||
(2, 4000, 2000, 2000, 2, 20, 0, 0, 0, 0, 0, 0, 0, 0),
|
||||
(3, 6000, 3000, 3000, 3, 30, 0, 0, 0, 0, 0, 0, 0, 0);
|
||||
|
||||
-- The per-video lifetime rows those finalized sessions would have left
|
||||
-- behind; redistributing videos re-derives imm_lifetime_anime from these.
|
||||
INSERT INTO imm_lifetime_media (
|
||||
video_id,
|
||||
total_sessions,
|
||||
total_active_ms,
|
||||
completed,
|
||||
first_watched_ms,
|
||||
last_watched_ms,
|
||||
CREATED_DATE,
|
||||
LAST_UPDATE_DATE
|
||||
) VALUES
|
||||
(1, 1, 1000, 0, '1000', '2000', 1000, 2000),
|
||||
(2, 1, 2000, 0, '3000', '4000', 3000, 4000),
|
||||
(3, 1, 3000, 0, '5000', '6000', 5000, 6000);
|
||||
`);
|
||||
|
||||
await tracker.rebuildLifetimeSummaries();
|
||||
|
||||
await tracker.reassignAnimeAnilist(2, {
|
||||
anilistId: 21202,
|
||||
titleRomaji: 'Kono Subarashii Sekai ni Shukufuku wo!',
|
||||
|
||||
@@ -16,8 +16,6 @@ import {
|
||||
applyPragmas,
|
||||
createTrackerPreparedStatements,
|
||||
ensureSchema,
|
||||
findManualDirectoryAnimeAssignment,
|
||||
getManualAnimeAssignment,
|
||||
executeQueuedWrite,
|
||||
getOrCreateAnimeRecord,
|
||||
getOrCreateVideoRecord,
|
||||
@@ -30,9 +28,11 @@ import {
|
||||
} from './immersion-tracker/storage';
|
||||
import {
|
||||
applySessionLifetimeSummary,
|
||||
recomputeLifetimeAnimeAggregates,
|
||||
reconcileStaleActiveSessions,
|
||||
rebuildLifetimeSummaries as rebuildLifetimeSummaryTables,
|
||||
recomputeLifetimeAnimeFromMedia,
|
||||
recomputeLifetimeGlobalFromSummaries,
|
||||
repairLifetimeSummariesFromMedia,
|
||||
shouldBackfillLifetimeSummaries,
|
||||
} from './immersion-tracker/lifetime';
|
||||
import {
|
||||
@@ -95,20 +95,16 @@ import {
|
||||
type RunDeleteMaintenanceTask,
|
||||
} from './immersion-tracker/delete-maintenance-worker-runtime';
|
||||
import { DeleteMaintenanceScheduler } from './immersion-tracker/delete-maintenance-scheduler';
|
||||
import {
|
||||
cleanupDuplicateSubtitleLines,
|
||||
type DuplicateSubtitleLineCleanupOptions,
|
||||
type DuplicateSubtitleLineCleanupSummary,
|
||||
} from './immersion-tracker/duplicate-line-cleanup';
|
||||
import { repairJellyfinStreamVideoLinks } from './immersion-tracker/jellyfin-link-repair';
|
||||
import {
|
||||
dismissAnimeMergeRecommendation,
|
||||
getAnimeMergeRecommendations,
|
||||
repairLegacySeasonlessAnimeRows,
|
||||
resolveAnimeAnilistConflict,
|
||||
type AnimeMergeRecommendation,
|
||||
} from './immersion-tracker/anime-season-repair';
|
||||
import {
|
||||
mergeAnimeRecords,
|
||||
moveVideoToAnime as moveVideoToAnimeQuery,
|
||||
type AnimeMergeSummary,
|
||||
type VideoMoveSummary,
|
||||
} from './immersion-tracker/anime-merge';
|
||||
import {
|
||||
buildVideoKey,
|
||||
deriveCanonicalTitle,
|
||||
@@ -445,7 +441,8 @@ export class ImmersionTrackerService {
|
||||
batchWindowMs: DELETE_MAINTENANCE_BATCH_WINDOW_MS,
|
||||
runTask: (task) => runDeleteMaintenanceTask(this.dbPath, task),
|
||||
onBusy: () => {
|
||||
this.requireWriteQueueDrained('delete maintenance');
|
||||
this.flushTelemetry(true);
|
||||
while (this.queue.length > 0) this.flushNow();
|
||||
this.writeLock.locked = true;
|
||||
},
|
||||
onIdle: () => {
|
||||
@@ -529,7 +526,7 @@ export class ImmersionTrackerService {
|
||||
this.logger.info(
|
||||
`Repaired season-scoped stats links on startup: scanned=${seasonRepair.scanned} movedVideos=${seasonRepair.movedVideos} deletedAnimeRows=${seasonRepair.deletedAnimeRows}`,
|
||||
);
|
||||
recomputeLifetimeAnimeAggregates(this.db);
|
||||
repairLifetimeSummariesFromMedia(this.db);
|
||||
}
|
||||
if (shouldBackfillLifetimeSummaries(this.db)) {
|
||||
const result = rebuildLifetimeSummaryTables(this.db);
|
||||
@@ -641,9 +638,32 @@ export class ImmersionTrackerService {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse animation bursts that earlier versions recorded frame by frame. The whole
|
||||
* queue is drained first so a burst still waiting to be written is scanned as stored
|
||||
* rows rather than surviving the cleanup and landing a moment after it.
|
||||
*/
|
||||
async cleanupDuplicateSubtitleLines(
|
||||
options: DuplicateSubtitleLineCleanupOptions = {},
|
||||
): Promise<DuplicateSubtitleLineCleanupSummary> {
|
||||
this.drainQueue();
|
||||
return cleanupDuplicateSubtitleLines(this.db, options);
|
||||
}
|
||||
|
||||
async rebuildLifetimeSummaries(): Promise<LifetimeRebuildSummary> {
|
||||
this.requireWriteQueueDrained('rebuilding lifetime summaries');
|
||||
return rebuildLifetimeSummaryTables(this.db);
|
||||
this.flushTelemetry(true);
|
||||
this.flushNow();
|
||||
// Non-destructive: recomputes from the media ledger (or bootstraps empty
|
||||
// lifetime tables), so history older than session retention is never reset.
|
||||
const repaired = repairLifetimeSummariesFromMedia(this.db);
|
||||
// Sessions currently tracked in the applied-sessions ledger, not sessions
|
||||
// processed by this call — the repair recomputes summaries instead of
|
||||
// re-applying sessions. Retention prunes these rows (FK cascade), so on an
|
||||
// old database this reads lower than the history the totals still include.
|
||||
const appliedRow = this.db
|
||||
.prepare('SELECT COUNT(*) AS count FROM imm_lifetime_applied_sessions')
|
||||
.get() as { count: number };
|
||||
return { appliedSessions: Number(appliedRow.count), rebuiltAtMs: repaired.repairedAtMs };
|
||||
}
|
||||
|
||||
async getKanjiStats(limit = 100): Promise<KanjiStatsRow[]> {
|
||||
@@ -709,14 +729,6 @@ export class ImmersionTrackerService {
|
||||
return getAnimeLibrary(this.db);
|
||||
}
|
||||
|
||||
async getAnimeMergeRecommendations(): Promise<AnimeMergeRecommendation[]> {
|
||||
return getAnimeMergeRecommendations(this.db);
|
||||
}
|
||||
|
||||
async dismissAnimeMergeRecommendation(recommendationId: number): Promise<boolean> {
|
||||
return dismissAnimeMergeRecommendation(this.db, recommendationId);
|
||||
}
|
||||
|
||||
async getAnimeDetail(animeId: number): Promise<AnimeDetailRow | null> {
|
||||
this.relinkYoutubeAnimeLibrary();
|
||||
return getAnimeDetail(this.db, animeId);
|
||||
@@ -824,63 +836,6 @@ export class ImmersionTrackerService {
|
||||
return this.deleteMaintenanceScheduler.enqueue(resolveTask);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fold duplicate library entries into one. Sources that hold the currently
|
||||
* playing episode are fine: the videos move, nothing is deleted out from
|
||||
* under the active session.
|
||||
*/
|
||||
async mergeAnime(targetAnimeId: number, sourceAnimeIds: number[]): Promise<AnimeMergeSummary> {
|
||||
const pendingVideoId = this.sessionState?.videoId;
|
||||
if (pendingVideoId !== undefined) {
|
||||
await this.pendingAnimeMetadataUpdates.get(pendingVideoId);
|
||||
}
|
||||
// This rebuilds the lifetime summaries, which recompute from the database:
|
||||
// queued writes have to land first or the active session is dropped from
|
||||
// the merged totals.
|
||||
this.requireWriteQueueDrained('merging library entries');
|
||||
return mergeAnimeRecords(this.db, targetAnimeId, sourceAnimeIds);
|
||||
}
|
||||
|
||||
async moveVideoToAnime(videoId: number, targetAnimeId: number): Promise<VideoMoveSummary> {
|
||||
await this.pendingAnimeMetadataUpdates.get(videoId);
|
||||
this.requireWriteQueueDrained('moving an episode');
|
||||
return moveVideoToAnimeQuery(this.db, videoId, targetAnimeId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist every queued write before a caller recomputes summaries from the
|
||||
* database.
|
||||
*
|
||||
* A single `flushNow()` is not enough: forced telemetry is appended to the
|
||||
* back of the queue while `flushNow()` writes at most `batchSize` entries off
|
||||
* the front, so a busy session leaves the newest sample unwritten. Stops as
|
||||
* soon as a pass makes no progress — a rolled-back batch is pushed back onto
|
||||
* the queue, and looping on that would spin forever.
|
||||
*
|
||||
* Returns false when the queue could not be emptied. Summary-rebuilding
|
||||
* callers fail closed in that case.
|
||||
*/
|
||||
private drainWriteQueue(context: string): boolean {
|
||||
this.flushTelemetry(true);
|
||||
while (this.queue.length > 0) {
|
||||
const pending = this.queue.length;
|
||||
this.flushNow();
|
||||
if (this.queue.length >= pending) {
|
||||
this.logger.warn(
|
||||
`Immersion tracker queue did not drain before ${context}; summaries may lag by ${this.queue.length} writes`,
|
||||
);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private requireWriteQueueDrained(context: string): void {
|
||||
if (!this.drainWriteQueue(context)) {
|
||||
throw new Error(`Immersion tracker queue did not drain before ${context}`);
|
||||
}
|
||||
}
|
||||
|
||||
async reassignAnimeAnilist(
|
||||
animeId: number,
|
||||
info: {
|
||||
@@ -893,14 +848,7 @@ export class ImmersionTrackerService {
|
||||
coverUrl?: string | null;
|
||||
},
|
||||
): Promise<void> {
|
||||
this.requireWriteQueueDrained('reassigning an AniList entry');
|
||||
// The user is acting on this entry, so it is the one that survives when
|
||||
// another row already claims the same AniList id.
|
||||
const repair = resolveAnimeAnilistConflict(this.db, animeId, info.anilistId, {
|
||||
survivor: 'target',
|
||||
matchConfidence: 'manual',
|
||||
});
|
||||
if (repair.anilistAssignmentBlocked) return;
|
||||
const conflictRepair = resolveAnimeAnilistConflict(this.db, animeId, info.anilistId);
|
||||
this.db
|
||||
.prepare(
|
||||
`
|
||||
@@ -926,8 +874,16 @@ export class ImmersionTrackerService {
|
||||
nowMs(),
|
||||
animeId,
|
||||
);
|
||||
if (repair.movedVideos > 0 || repair.deletedAnimeRows > 0) {
|
||||
recomputeLifetimeAnimeAggregates(this.db);
|
||||
// Empty lifetime tables still need the retained-session bootstrap. Once a
|
||||
// media ledger exists, only the redistributed and explicitly edited anime
|
||||
// can have changed.
|
||||
if (shouldBackfillLifetimeSummaries(this.db)) {
|
||||
repairLifetimeSummariesFromMedia(this.db);
|
||||
} else {
|
||||
const affectedAnimeIds = new Set(conflictRepair.affectedAnimeIds);
|
||||
affectedAnimeIds.add(animeId);
|
||||
recomputeLifetimeAnimeFromMedia(this.db, [...affectedAnimeIds]);
|
||||
recomputeLifetimeGlobalFromSummaries(this.db);
|
||||
}
|
||||
|
||||
// Update cover art for all videos in this anime
|
||||
@@ -1375,7 +1331,7 @@ export class ImmersionTrackerService {
|
||||
metadataJson: candidate.metadataJson,
|
||||
});
|
||||
}
|
||||
recomputeLifetimeAnimeAggregates(this.db);
|
||||
repairLifetimeSummariesFromMedia(this.db);
|
||||
}
|
||||
|
||||
recordJellyfinPlaybackMetadata(metadata: JellyfinPlaybackMetadataInput): void {
|
||||
@@ -1423,18 +1379,16 @@ export class ImmersionTrackerService {
|
||||
seasonNumber,
|
||||
episodeNumber,
|
||||
});
|
||||
const animeId =
|
||||
getManualAnimeAssignment(this.db, videoId) ??
|
||||
getOrCreateAnimeRecord(this.db, {
|
||||
parsedTitle: libraryTitle,
|
||||
canonicalTitle: libraryTitle,
|
||||
seasonScope: seasonNumber,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson,
|
||||
});
|
||||
const animeId = getOrCreateAnimeRecord(this.db, {
|
||||
parsedTitle: libraryTitle,
|
||||
canonicalTitle: libraryTitle,
|
||||
seasonScope: seasonNumber,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson,
|
||||
});
|
||||
linkVideoToAnimeRecord(this.db, videoId, {
|
||||
animeId,
|
||||
parsedBasename: null,
|
||||
@@ -1450,7 +1404,21 @@ export class ImmersionTrackerService {
|
||||
this.db.prepare('SELECT 1 FROM imm_lifetime_media WHERE video_id = ?').get(videoId),
|
||||
);
|
||||
if (hasLifetimeMedia || (previousLink && previousLink.animeId !== animeId)) {
|
||||
recomputeLifetimeAnimeAggregates(this.db);
|
||||
// Playback-time relink: only the old and new anime are affected, so
|
||||
// recompute just those from the media ledger instead of a full repair.
|
||||
const affectedAnimeIds = new Set<number>([animeId]);
|
||||
if (previousLink?.animeId) affectedAnimeIds.add(previousLink.animeId);
|
||||
let transactionStarted = false;
|
||||
try {
|
||||
this.db.exec('BEGIN IMMEDIATE');
|
||||
transactionStarted = true;
|
||||
recomputeLifetimeAnimeFromMedia(this.db, [...affectedAnimeIds]);
|
||||
recomputeLifetimeGlobalFromSummaries(this.db);
|
||||
this.db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
if (transactionStarted) this.db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1933,6 +1901,24 @@ export class ImmersionTrackerService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Write out everything queued, not just the next batch.
|
||||
*
|
||||
* `flushNow` writes at most `batchSize` entries and does nothing at all while the write
|
||||
* lock is held, so a maintenance pass that runs straight after it can still be reading
|
||||
* a database that is missing rows. Each pass has to shrink the queue to continue: a
|
||||
* failed flush puts its batch back, and looping on that would never finish.
|
||||
*/
|
||||
private drainQueue(): void {
|
||||
while (this.queue.length > 0) {
|
||||
const pendingBefore = this.queue.length;
|
||||
this.flushNow();
|
||||
if (this.queue.length >= pendingBefore) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private flushSingle(write: QueuedWrite): void {
|
||||
executeQueuedWrite(write, this.preparedStatements);
|
||||
}
|
||||
@@ -2071,21 +2057,16 @@ export class ImmersionTrackerService {
|
||||
return;
|
||||
}
|
||||
|
||||
const animeId =
|
||||
getManualAnimeAssignment(this.db, videoId) ??
|
||||
(mediaPath && !isRemoteSource(mediaPath)
|
||||
? findManualDirectoryAnimeAssignment(this.db, videoId, mediaPath, parsed.parsedSeason)
|
||||
: null) ??
|
||||
getOrCreateAnimeRecord(this.db, {
|
||||
parsedTitle: parsed.parsedTitle,
|
||||
canonicalTitle: parsed.parsedTitle,
|
||||
seasonScope: parsed.parsedSeason,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: parsed.parseMetadataJson,
|
||||
});
|
||||
const animeId = getOrCreateAnimeRecord(this.db, {
|
||||
parsedTitle: parsed.parsedTitle,
|
||||
canonicalTitle: parsed.parsedTitle,
|
||||
seasonScope: parsed.parsedSeason,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: parsed.parseMetadataJson,
|
||||
});
|
||||
linkVideoToAnimeRecord(this.db, videoId, {
|
||||
animeId,
|
||||
parsedBasename: parsed.parsedBasename,
|
||||
|
||||
@@ -1,896 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database } from '../sqlite.js';
|
||||
import type { DatabaseSync } from '../sqlite.js';
|
||||
import {
|
||||
applyPragmas,
|
||||
ensureSchema,
|
||||
findManualDirectoryAnimeAssignment,
|
||||
getManualAnimeAssignment,
|
||||
getOrCreateAnimeRecord,
|
||||
linkVideoToAnimeRecord,
|
||||
} from '../storage.js';
|
||||
import { mergeAnimeRecords, moveVideoToAnime } from '../anime-merge.js';
|
||||
import {
|
||||
dismissAnimeMergeRecommendation,
|
||||
getAnimeMergeRecommendations,
|
||||
resolveAnimeAnilistConflict,
|
||||
} from '../anime-season-repair.js';
|
||||
import { updateAnimeAnilistInfo } from '../query-maintenance.js';
|
||||
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
|
||||
function makeDbPath(): string {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-anime-merge-test-'));
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) return;
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
function withDb(work: (db: DatabaseSync) => void): void {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
applyPragmas(db);
|
||||
ensureSchema(db);
|
||||
work(db);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
}
|
||||
|
||||
interface AnimeSeed {
|
||||
animeId: number;
|
||||
key: string;
|
||||
title: string;
|
||||
anilistId?: number | null;
|
||||
titleRomaji?: string | null;
|
||||
}
|
||||
|
||||
function insertAnime(db: DatabaseSync, seed: AnimeSeed): void {
|
||||
db.prepare(
|
||||
`INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, anilist_id, title_romaji, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
||||
).run(
|
||||
seed.animeId,
|
||||
seed.key,
|
||||
seed.title,
|
||||
seed.anilistId ?? null,
|
||||
seed.titleRomaji ?? null,
|
||||
BASE_MS,
|
||||
BASE_MS,
|
||||
);
|
||||
}
|
||||
|
||||
interface EpisodeSeed {
|
||||
videoId: number;
|
||||
animeId: number;
|
||||
season?: number | null;
|
||||
episode?: number;
|
||||
activeMs?: number;
|
||||
cards?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* One episode with one ended session, plus the imm_lifetime_media row the
|
||||
* session would have left behind, so lifetime aggregates have something to sum.
|
||||
*/
|
||||
function insertEpisode(db: DatabaseSync, seed: EpisodeSeed): void {
|
||||
const activeMs = seed.activeMs ?? 1000;
|
||||
const cards = seed.cards ?? 1;
|
||||
db.prepare(
|
||||
`INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, parsed_title, parsed_season, parsed_episode, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, ?, 1, 'Show', ?, ?, 1, 1440000, ?, ?)`,
|
||||
).run(
|
||||
seed.videoId,
|
||||
`local:/tmp/show-${seed.videoId}.mkv`,
|
||||
seed.animeId,
|
||||
`Show ${seed.videoId}`,
|
||||
seed.season ?? null,
|
||||
seed.episode ?? seed.videoId,
|
||||
BASE_MS,
|
||||
BASE_MS,
|
||||
);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, active_watched_ms, cards_mined, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, ?, ?, 2, ?, ?, ?, ?)`,
|
||||
).run(
|
||||
seed.videoId,
|
||||
`session-${seed.videoId}`,
|
||||
seed.videoId,
|
||||
String(BASE_MS),
|
||||
String(BASE_MS + activeMs),
|
||||
activeMs,
|
||||
cards,
|
||||
BASE_MS,
|
||||
BASE_MS,
|
||||
);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines(session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, 1, ?, ?, ?)`,
|
||||
).run(seed.videoId, seed.videoId, seed.animeId, `line ${seed.videoId}`, BASE_MS, BASE_MS);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_lifetime_media(video_id, total_sessions, total_active_ms, total_cards, completed, first_watched_ms, last_watched_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, 1, ?, ?, 1, ?, ?, ?, ?)`,
|
||||
).run(
|
||||
seed.videoId,
|
||||
activeMs,
|
||||
cards,
|
||||
String(BASE_MS),
|
||||
String(BASE_MS + activeMs),
|
||||
BASE_MS,
|
||||
BASE_MS,
|
||||
);
|
||||
}
|
||||
|
||||
function animeIds(db: DatabaseSync): number[] {
|
||||
return (
|
||||
db.prepare('SELECT anime_id AS id FROM imm_anime ORDER BY anime_id').all() as Array<{
|
||||
id: number;
|
||||
}>
|
||||
).map((row) => row.id);
|
||||
}
|
||||
|
||||
function videoAnimeId(db: DatabaseSync, videoId: number): number | null {
|
||||
return (
|
||||
db.prepare('SELECT anime_id AS id FROM imm_videos WHERE video_id = ?').get(videoId) as {
|
||||
id: number | null;
|
||||
}
|
||||
).id;
|
||||
}
|
||||
|
||||
function assignmentLocked(db: DatabaseSync, videoId: number): number {
|
||||
return (
|
||||
db
|
||||
.prepare('SELECT anime_assignment_locked AS locked FROM imm_videos WHERE video_id = ?')
|
||||
.get(videoId) as { locked: number }
|
||||
).locked;
|
||||
}
|
||||
|
||||
function lineAnimeIds(db: DatabaseSync, animeId: number): number {
|
||||
return Number(
|
||||
(
|
||||
db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_subtitle_lines WHERE anime_id = ?')
|
||||
.get(animeId) as { total: number }
|
||||
).total,
|
||||
);
|
||||
}
|
||||
|
||||
test('mergeAnimeRecords folds episodes, lines and lifetime totals into the target', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1', anilistId: 555 });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, activeMs: 1000, cards: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1, activeMs: 2000, cards: 3 });
|
||||
|
||||
const summary = mergeAnimeRecords(db, 1, [2]);
|
||||
|
||||
assert.equal(summary.survivingAnimeId, 1);
|
||||
assert.deepEqual(summary.mergedAnimeIds, [2]);
|
||||
assert.equal(summary.movedVideos, 1);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 2), 1);
|
||||
assert.equal(lineAnimeIds(db, 1), 2);
|
||||
|
||||
const lifetime = db
|
||||
.prepare(
|
||||
'SELECT total_active_ms AS activeMs, total_cards AS cards, episodes_started AS episodes FROM imm_lifetime_anime WHERE anime_id = 1',
|
||||
)
|
||||
.get() as { activeMs: number; cards: number; episodes: number };
|
||||
assert.equal(lifetime.activeMs, 3000);
|
||||
assert.equal(lifetime.cards, 4);
|
||||
assert.equal(lifetime.episodes, 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('merge and move preserve lifetime history whose raw sessions were pruned', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertAnime(db, { animeId: 3, key: 'other show', title: 'Other Show' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, activeMs: 1000, cards: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1, activeMs: 2000, cards: 3 });
|
||||
insertEpisode(db, { videoId: 3, animeId: 3, activeMs: 4000, cards: 5 });
|
||||
// Retention pruned every raw session; only the lifetime summaries remain.
|
||||
db.exec('DELETE FROM imm_sessions');
|
||||
db.prepare(
|
||||
`UPDATE imm_lifetime_global
|
||||
SET total_sessions = 200, total_active_ms = 360000000, total_cards = 500, active_days = 90
|
||||
WHERE global_id = 1`,
|
||||
).run();
|
||||
|
||||
mergeAnimeRecords(db, 1, [2]);
|
||||
moveVideoToAnime(db, 3, 1);
|
||||
|
||||
const globalRow = db
|
||||
.prepare(
|
||||
`SELECT total_sessions AS sessions, total_active_ms AS activeMs, total_cards AS cards, active_days AS days
|
||||
FROM imm_lifetime_global WHERE global_id = 1`,
|
||||
)
|
||||
.get() as { sessions: number; activeMs: number; cards: number; days: number };
|
||||
assert.equal(globalRow.sessions, 200);
|
||||
assert.equal(globalRow.activeMs, 360000000);
|
||||
assert.equal(globalRow.cards, 500);
|
||||
assert.equal(globalRow.days, 90);
|
||||
|
||||
const survivor = db
|
||||
.prepare(
|
||||
`SELECT total_active_ms AS activeMs, total_cards AS cards, episodes_started AS episodes
|
||||
FROM imm_lifetime_anime WHERE anime_id = 1`,
|
||||
)
|
||||
.get() as { activeMs: number; cards: number; episodes: number };
|
||||
assert.equal(survivor.activeMs, 7000);
|
||||
assert.equal(survivor.cards, 9);
|
||||
assert.equal(survivor.episodes, 3);
|
||||
assert.equal(
|
||||
db.prepare('SELECT 1 FROM imm_lifetime_anime WHERE anime_id = 3').get(),
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('mergeAnimeRecords repoints subtitle lines recorded before the anime link landed', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
// Lines are written with the video's anime_id at the time, which is NULL
|
||||
// until the async title parse assigns one.
|
||||
db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines(session_id, video_id, anime_id, line_index, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (2, 2, NULL, 2, 'unlinked line', ?, ?)`,
|
||||
).run(BASE_MS, BASE_MS);
|
||||
|
||||
mergeAnimeRecords(db, 1, [2]);
|
||||
|
||||
assert.equal(lineAnimeIds(db, 1), 3);
|
||||
const orphaned = Number(
|
||||
(
|
||||
db
|
||||
.prepare('SELECT COUNT(*) AS total FROM imm_subtitle_lines WHERE anime_id IS NULL')
|
||||
.get() as { total: number }
|
||||
).total,
|
||||
);
|
||||
assert.equal(orphaned, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('mergeAnimeRecords inherits metadata the target is missing without clobbering its own', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', titleRomaji: 'Shou' });
|
||||
insertAnime(db, {
|
||||
animeId: 2,
|
||||
key: 'show season 1',
|
||||
title: 'Show Season 1',
|
||||
anilistId: 555,
|
||||
titleRomaji: 'Show Romaji',
|
||||
});
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
mergeAnimeRecords(db, 1, [2]);
|
||||
|
||||
const row = db
|
||||
.prepare(
|
||||
'SELECT canonical_title AS title, anilist_id AS anilistId, title_romaji AS romaji FROM imm_anime WHERE anime_id = 1',
|
||||
)
|
||||
.get() as { title: string; anilistId: number | null; romaji: string | null };
|
||||
assert.equal(row.title, 'Show');
|
||||
// anilist_id is UNIQUE, so inheriting it proves the source row was gone first.
|
||||
assert.equal(row.anilistId, 555);
|
||||
assert.equal(row.romaji, 'Shou');
|
||||
});
|
||||
});
|
||||
|
||||
test('mergeAnimeRecords preserves source title identities as aliases of the survivor', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO imm_anime_title_aliases(normalized_title_key, anime_id, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES ('show s01', 2, ?, ?)`,
|
||||
).run(BASE_MS, BASE_MS);
|
||||
|
||||
mergeAnimeRecords(db, 1, [2]);
|
||||
|
||||
const fromSourceTitle = getOrCreateAnimeRecord(db, {
|
||||
parsedTitle: 'Show Season 1',
|
||||
canonicalTitle: 'Show Season 1',
|
||||
seasonScope: 1,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: null,
|
||||
});
|
||||
const fromTransferredAlias = getOrCreateAnimeRecord(db, {
|
||||
parsedTitle: 'Show S01',
|
||||
canonicalTitle: 'Show S01',
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: null,
|
||||
});
|
||||
|
||||
assert.equal(fromSourceTitle, 1);
|
||||
assert.equal(fromTransferredAlias, 1);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(
|
||||
(
|
||||
db.prepare('SELECT canonical_title AS title FROM imm_anime WHERE anime_id = 1').get() as {
|
||||
title: string;
|
||||
}
|
||||
).title,
|
||||
'Show',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('mergeAnimeRecords ignores unknown targets and self-merges', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
|
||||
assert.deepEqual(mergeAnimeRecords(db, 99, [1]).mergedAnimeIds, []);
|
||||
assert.deepEqual(mergeAnimeRecords(db, 1, [1]).mergedAnimeIds, []);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('moveVideoToAnime moves one episode and prunes the emptied entry', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'stray', title: 'Stray Episode Title', anilistId: 777 });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, activeMs: 5000, cards: 2 });
|
||||
|
||||
const summary = moveVideoToAnime(db, 2, 1);
|
||||
|
||||
assert.equal(summary.targetAnimeId, 1);
|
||||
assert.equal(summary.previousAnimeId, 2);
|
||||
assert.equal(summary.removedPreviousAnime, true);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 2), 1);
|
||||
assert.equal(assignmentLocked(db, 2), 1);
|
||||
assert.equal(getManualAnimeAssignment(db, 2), 1);
|
||||
assert.equal(lineAnimeIds(db, 1), 2);
|
||||
const lifetime = db
|
||||
.prepare('SELECT total_active_ms AS activeMs FROM imm_lifetime_anime WHERE anime_id = 1')
|
||||
.get() as { activeMs: number };
|
||||
assert.equal(lifetime.activeMs, 6000);
|
||||
// The stray entry's AniList link is dropped, not inherited: a move makes no
|
||||
// claim that the two entries are the same show.
|
||||
const target = db
|
||||
.prepare('SELECT anilist_id AS anilistId FROM imm_anime WHERE anime_id = 1')
|
||||
.get() as { anilistId: number | null };
|
||||
assert.equal(target.anilistId, null);
|
||||
});
|
||||
});
|
||||
|
||||
test('moveVideoToAnime is a no-op when the episode is already in the target entry', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
|
||||
const summary = moveVideoToAnime(db, 1, 1);
|
||||
|
||||
assert.equal(summary.targetAnimeId, 1);
|
||||
assert.equal(summary.previousAnimeId, 1);
|
||||
assert.equal(summary.removedPreviousAnime, false);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
assert.equal(assignmentLocked(db, 1), 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('automatic metadata cannot overwrite a manual episode assignment', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'stray', title: 'Stray' });
|
||||
insertAnime(db, { animeId: 3, key: 'parser result', title: 'Parser Result' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 2, season: 1 });
|
||||
|
||||
moveVideoToAnime(db, 1, 1);
|
||||
linkVideoToAnimeRecord(db, 1, {
|
||||
animeId: 3,
|
||||
parsedBasename: 'Parser Result S01E01.mkv',
|
||||
parsedTitle: 'Parser Result',
|
||||
parsedSeason: 1,
|
||||
parsedEpisode: 1,
|
||||
parserSource: 'guessit',
|
||||
parserConfidence: 1,
|
||||
parseMetadataJson: null,
|
||||
});
|
||||
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
assert.equal(getManualAnimeAssignment(db, 1), 1);
|
||||
const parsedTitle = db
|
||||
.prepare('SELECT parsed_title AS parsedTitle FROM imm_videos WHERE video_id = 1')
|
||||
.get() as { parsedTitle: string | null };
|
||||
assert.equal(parsedTitle.parsedTitle, 'Parser Result');
|
||||
});
|
||||
});
|
||||
|
||||
test('directory grouping requires one season-compatible manual destination', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'stray', title: 'Stray' });
|
||||
insertAnime(db, { animeId: 3, key: 'other', title: 'Other' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 2, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 3, season: 1 });
|
||||
insertEpisode(db, { videoId: 3, animeId: 3, season: 1 });
|
||||
db.prepare('UPDATE imm_videos SET source_path = ? WHERE video_id = ?').run(
|
||||
'/library/show/Show S01E01.mkv',
|
||||
1,
|
||||
);
|
||||
db.prepare('UPDATE imm_videos SET source_path = ? WHERE video_id = ?').run(
|
||||
'/library/show/Stray S01E02.mkv',
|
||||
2,
|
||||
);
|
||||
db.prepare('UPDATE imm_videos SET source_path = ? WHERE video_id = ?').run(
|
||||
'/library/show/Other S01E03.mkv',
|
||||
3,
|
||||
);
|
||||
|
||||
moveVideoToAnime(db, 1, 1);
|
||||
|
||||
assert.equal(findManualDirectoryAnimeAssignment(db, 2, '/library/show/Stray S01E02.mkv', 1), 1);
|
||||
assert.equal(
|
||||
findManualDirectoryAnimeAssignment(db, 2, '/library/show/Stray S02E02.mkv', 2),
|
||||
null,
|
||||
);
|
||||
assert.equal(
|
||||
findManualDirectoryAnimeAssignment(db, 2, '/library/other/Stray S01E02.mkv', 1),
|
||||
null,
|
||||
);
|
||||
|
||||
moveVideoToAnime(db, 3, 3);
|
||||
assert.equal(
|
||||
findManualDirectoryAnimeAssignment(db, 2, '/library/show/Stray S01E02.mkv', 1),
|
||||
null,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('moveVideoToAnime keeps the source entry when other episodes remain', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertAnime(db, { animeId: 2, key: 'other', title: 'Other' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 2 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2 });
|
||||
|
||||
const summary = moveVideoToAnime(db, 2, 1);
|
||||
|
||||
assert.equal(summary.removedPreviousAnime, false);
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(videoAnimeId(db, 1), 2);
|
||||
assert.equal(videoAnimeId(db, 2), 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('moveVideoToAnime rejects unknown episodes and targets', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
|
||||
assert.throws(() => moveVideoToAnime(db, 99, 1));
|
||||
assert.throws(() => moveVideoToAnime(db, 1, 99));
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict folds a seasonless duplicate into the entry that owns the id', () => {
|
||||
withDb((db) => {
|
||||
// Same show, split because one release tagged S01 and the other did not.
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.equal(summary.survivingAnimeId, 1);
|
||||
assert.equal(summary.movedVideos, 1);
|
||||
assert.equal(summary.deletedAnimeRows, 1);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 2), 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict recommends a weak title collision instead of merging it', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.equal(summary.repaired, 0);
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), [{ recommendationId: 1, animeIds: [1, 2] }]);
|
||||
});
|
||||
});
|
||||
|
||||
test('automatic AniList update leaves a weak collision unassigned for user review', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
updateAnimeAnilistInfo(db, 2, {
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
episodesTotal: 12,
|
||||
exactTitleMatch: false,
|
||||
});
|
||||
|
||||
const target = db
|
||||
.prepare('SELECT anilist_id AS anilistId FROM imm_anime WHERE anime_id = 2')
|
||||
.get() as {
|
||||
anilistId: number | null;
|
||||
};
|
||||
assert.equal(target.anilistId, null);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), [{ recommendationId: 1, animeIds: [1, 2] }]);
|
||||
});
|
||||
});
|
||||
|
||||
test('dismissed weak collision stays dismissed when automatic resolution repeats', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
assert.equal(dismissAnimeMergeRecommendation(db, 1), true);
|
||||
resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('dismissed recommendation prevents a later exact automatic merge of the pair', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
resolveAnimeAnilistConflict(db, 2, 163132, { matchConfidence: 'weak' });
|
||||
assert.equal(dismissAnimeMergeRecommendation(db, 1), true);
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132, { matchConfidence: 'exact' });
|
||||
|
||||
assert.equal(summary.repaired, 0);
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('manual merge clears recommendations involving the absorbed entry', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
mergeAnimeRecords(db, 1, [2]);
|
||||
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict keeps the target entry when the user drove the change', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132, { survivor: 'target' });
|
||||
|
||||
assert.equal(summary.survivingAnimeId, 2);
|
||||
assert.deepEqual(animeIds(db), [2]);
|
||||
assert.equal(videoAnimeId(db, 1), 2);
|
||||
const row = db.prepare('SELECT anilist_id AS id FROM imm_anime WHERE anime_id = 2').get() as {
|
||||
id: number | null;
|
||||
};
|
||||
assert.equal(row.id, 163132);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict falls back to season redistribution for multi-season rows', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 1, season: 2 });
|
||||
insertEpisode(db, { videoId: 3, animeId: 2, season: 1 });
|
||||
|
||||
resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
// The mixed row is split by season instead of being poured onto one card.
|
||||
const titles = (
|
||||
db.prepare('SELECT canonical_title AS title FROM imm_anime ORDER BY title').all() as Array<{
|
||||
title: string;
|
||||
}>
|
||||
).map((row) => row.title);
|
||||
assert.deepEqual(titles, ['Show Season 1', 'Show Season 2']);
|
||||
assert.equal(videoAnimeId(db, 1), 2);
|
||||
assert.equal(videoAnimeId(db, 3), 2);
|
||||
assert.notEqual(videoAnimeId(db, 2), 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('season redistribution leaves manually assigned episodes in place', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 1, season: 2 });
|
||||
insertEpisode(db, { videoId: 3, animeId: 2, season: 1 });
|
||||
moveVideoToAnime(db, 1, 1);
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
assert.equal(assignmentLocked(db, 1), 1);
|
||||
assert.notEqual(videoAnimeId(db, 2), 1);
|
||||
assert.equal(summary.movedVideos, 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict leaves explicit incompatible seasons and assignments unchanged', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'show season 1',
|
||||
title: 'Show Season 1',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'show season 2', title: 'Show Season 2' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 2 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132, { matchConfidence: 'exact' });
|
||||
|
||||
assert.equal(summary.repaired, 0);
|
||||
assert.equal(summary.movedVideos, 0);
|
||||
assert.equal(summary.deletedAnimeRows, 0);
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
const assignments = db
|
||||
.prepare(
|
||||
'SELECT anime_id AS animeId, anilist_id AS anilistId FROM imm_anime ORDER BY anime_id',
|
||||
)
|
||||
.all() as Array<{ animeId: number; anilistId: number | null }>;
|
||||
assert.deepEqual(assignments, [
|
||||
{ animeId: 1, anilistId: 163132 },
|
||||
{ animeId: 2, anilistId: null },
|
||||
]);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('manual AniList resolution reassigns across explicit seasons without merging them', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'show season 1',
|
||||
title: 'Show Season 1',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'show season 2', title: 'Show Season 2' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 2 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132, { survivor: 'target' });
|
||||
|
||||
assert.equal(summary.anilistAssignmentBlocked, false);
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
const assignments = db
|
||||
.prepare(
|
||||
'SELECT anime_id AS animeId, anilist_id AS anilistId FROM imm_anime ORDER BY anime_id',
|
||||
)
|
||||
.all() as Array<{ animeId: number; anilistId: number | null }>;
|
||||
assert.deepEqual(assignments, [
|
||||
{ animeId: 1, anilistId: null },
|
||||
{ animeId: 2, anilistId: 163132 },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
test('automatic AniList update does not transfer an assignment across explicit seasons', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'show season 1',
|
||||
title: 'Show Season 1',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'show season 2', title: 'Show Season 2' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 2 });
|
||||
|
||||
updateAnimeAnilistInfo(db, 2, {
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show',
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
episodesTotal: 12,
|
||||
exactTitleMatch: true,
|
||||
});
|
||||
|
||||
const assignments = db
|
||||
.prepare(
|
||||
'SELECT anime_id AS animeId, anilist_id AS anilistId FROM imm_anime ORDER BY anime_id',
|
||||
)
|
||||
.all() as Array<{ animeId: number; anilistId: number | null }>;
|
||||
assert.deepEqual(assignments, [
|
||||
{ animeId: 1, anilistId: 163132 },
|
||||
{ animeId: 2, anilistId: null },
|
||||
]);
|
||||
assert.equal(videoAnimeId(db, 1), 1);
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('automatic AniList update with unknown match confidence validates stored titles', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'actual show',
|
||||
title: 'Actual Show',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'unrelated release', title: 'Unrelated Release' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
updateAnimeAnilistInfo(db, 2, {
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Actual Show',
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
episodesTotal: 12,
|
||||
});
|
||||
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), [{ recommendationId: 1, animeIds: [1, 2] }]);
|
||||
});
|
||||
});
|
||||
|
||||
test('stored AniList titles ignore season suffixes when validating an automatic merge', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, {
|
||||
animeId: 1,
|
||||
key: 'legacy show',
|
||||
title: 'Show Season 1',
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show Season 1',
|
||||
});
|
||||
insertAnime(db, { animeId: 2, key: 'show season 1', title: 'Show Season 1' });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 1 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.equal(summary.deletedAnimeRows, 1);
|
||||
assert.deepEqual(animeIds(db), [1]);
|
||||
assert.equal(videoAnimeId(db, 2), 1);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('resolveAnimeAnilistConflict leaves an entry that already links elsewhere alone', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show s2', title: 'Show Season 2', anilistId: 999 });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 2 });
|
||||
|
||||
const summary = resolveAnimeAnilistConflict(db, 2, 163132);
|
||||
|
||||
assert.equal(videoAnimeId(db, 2), 2);
|
||||
assert.ok(animeIds(db).includes(2));
|
||||
assert.equal(
|
||||
(
|
||||
db.prepare('SELECT anilist_id AS anilistId FROM imm_anime WHERE anime_id = 2').get() as {
|
||||
anilistId: number;
|
||||
}
|
||||
).anilistId,
|
||||
999,
|
||||
);
|
||||
assert.equal(summary.repaired, 0);
|
||||
assert.equal(summary.movedVideos, 0);
|
||||
assert.deepEqual(getAnimeMergeRecommendations(db), []);
|
||||
});
|
||||
});
|
||||
|
||||
test('automatic AniList update onto an entry that already links elsewhere does not throw', () => {
|
||||
withDb((db) => {
|
||||
insertAnime(db, { animeId: 1, key: 'show', title: 'Show', anilistId: 163132 });
|
||||
insertAnime(db, { animeId: 2, key: 'show s2', title: 'Show Season 2', anilistId: 999 });
|
||||
insertEpisode(db, { videoId: 1, animeId: 1, season: 1 });
|
||||
insertEpisode(db, { videoId: 2, animeId: 2, season: 2 });
|
||||
|
||||
// Entry 2 explicitly links to 999; a later video re-resolving to entry 1's
|
||||
// id must be refused, not written over the UNIQUE anilist_id column.
|
||||
updateAnimeAnilistInfo(db, 2, {
|
||||
anilistId: 163132,
|
||||
titleRomaji: 'Show',
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
episodesTotal: 12,
|
||||
exactTitleMatch: true,
|
||||
});
|
||||
|
||||
assert.deepEqual(animeIds(db), [1, 2]);
|
||||
assert.equal(
|
||||
(
|
||||
db.prepare('SELECT anilist_id AS anilistId FROM imm_anime WHERE anime_id = 2').get() as {
|
||||
anilistId: number;
|
||||
}
|
||||
).anilistId,
|
||||
999,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,467 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database } from '../sqlite.js';
|
||||
import type { DatabaseSync } from '../sqlite.js';
|
||||
import { ensureSchema } from '../storage.js';
|
||||
import { cleanupDuplicateSubtitleLines } from '../duplicate-line-cleanup.js';
|
||||
|
||||
const DAY_MS = 86_400_000;
|
||||
const BASE_MS = 1_700_000_000_000;
|
||||
const WORD_ID = 1;
|
||||
|
||||
interface SeedLine {
|
||||
session: number;
|
||||
text: string;
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
/** Recording wall-clock, i.e. what the lookback window filters on. */
|
||||
createdMs?: number;
|
||||
}
|
||||
|
||||
function makeDbPath(): string {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-duplicate-line-test-'));
|
||||
return path.join(dir, 'immersion.sqlite');
|
||||
}
|
||||
|
||||
function cleanupDbPath(dbPath: string): void {
|
||||
const dir = path.dirname(dbPath);
|
||||
if (!fs.existsSync(dir)) return;
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
/** One episode, two sessions of it, and one word occurrence per seeded line. */
|
||||
function seed(db: DatabaseSync, lines: SeedLine[]): void {
|
||||
db.exec(`
|
||||
INSERT INTO imm_anime(anime_id, normalized_title_key, canonical_title, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'show', 'Show', ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_videos(video_id, video_key, anime_id, canonical_title, source_type, watched, duration_ms, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 'v1', 1, 'Ep 1', 1, 1, 1440000, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_sessions(session_id, session_uuid, video_id, started_at_ms, ended_at_ms, status, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (1, 's1', 1, '${BASE_MS}', '${BASE_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS}),
|
||||
(2, 's2', 1, '${BASE_MS + DAY_MS}', '${BASE_MS + DAY_MS + 1000}', 2, ${BASE_MS}, ${BASE_MS});
|
||||
INSERT INTO imm_words(id, headword, word, reading, part_of_speech, pos1, first_seen, last_seen, frequency)
|
||||
VALUES (${WORD_ID}, '飛び上がる', '飛び上がる', '', 'verb', '動詞', ${Math.floor(BASE_MS / 1000)}, ${Math.floor(BASE_MS / 1000)}, 0);
|
||||
`);
|
||||
|
||||
const insertLine = db.prepare(
|
||||
`INSERT INTO imm_subtitle_lines(
|
||||
line_id, session_id, video_id, anime_id, line_index,
|
||||
segment_start_ms, segment_end_ms, text, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, 1, 1, ?, ?, ?, ?, ?, ?)`,
|
||||
);
|
||||
const insertOccurrence = db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences(line_id, word_id, occurrence_count, seen_ms)
|
||||
VALUES (?, ?, 1, ?)`,
|
||||
);
|
||||
|
||||
lines.forEach((line, index) => {
|
||||
const lineId = index + 1;
|
||||
const lineIndex = index + 1;
|
||||
const createdMs = line.createdMs ?? BASE_MS;
|
||||
insertLine.run(
|
||||
lineId,
|
||||
line.session,
|
||||
lineIndex,
|
||||
line.startMs,
|
||||
line.endMs,
|
||||
line.text,
|
||||
createdMs,
|
||||
createdMs,
|
||||
);
|
||||
insertOccurrence.run(lineId, WORD_ID, createdMs);
|
||||
});
|
||||
|
||||
db.exec(`
|
||||
UPDATE imm_words SET frequency = (
|
||||
SELECT COALESCE(SUM(o.occurrence_count), 0)
|
||||
FROM imm_word_line_occurrences o WHERE o.word_id = imm_words.id
|
||||
)
|
||||
`);
|
||||
}
|
||||
|
||||
function createDb(lines: SeedLine[]): { db: DatabaseSync; dbPath: string } {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
ensureSchema(db);
|
||||
seed(db, lines);
|
||||
return { db, dbPath };
|
||||
}
|
||||
|
||||
/** A typeset line mpv reported once per animation frame. */
|
||||
function karaokeFrames(
|
||||
session: number,
|
||||
text: string,
|
||||
startMs: number,
|
||||
frames: number,
|
||||
frameMs: number,
|
||||
): SeedLine[] {
|
||||
return Array.from({ length: frames }, (_, index) => ({
|
||||
session,
|
||||
text,
|
||||
startMs: startMs + index * frameMs,
|
||||
endMs: startMs + (index + 1) * frameMs,
|
||||
}));
|
||||
}
|
||||
|
||||
function countLines(db: DatabaseSync): number {
|
||||
return (db.prepare('SELECT COUNT(*) AS total FROM imm_subtitle_lines').get() as { total: number })
|
||||
.total;
|
||||
}
|
||||
|
||||
function wordFrequency(db: DatabaseSync): number {
|
||||
const row = db.prepare('SELECT frequency FROM imm_words WHERE id = ?').get(WORD_ID) as {
|
||||
frequency: number;
|
||||
} | null;
|
||||
return row?.frequency ?? 0;
|
||||
}
|
||||
|
||||
test('a karaoke burst collapses to one line and gives back its word counts', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 40, 40),
|
||||
{ session: 1, text: 'おはよう', startMs: 20_000, endMs: 22_000 },
|
||||
]);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 1);
|
||||
assert.equal(summary.removedLines, 39);
|
||||
assert.equal(summary.removedWordOccurrences, 39);
|
||||
assert.equal(countLines(db), 2);
|
||||
assert.equal(wordFrequency(db), 2);
|
||||
|
||||
// The surviving line covers the whole run, the way the parsed cue would.
|
||||
const kept = db
|
||||
.prepare(
|
||||
'SELECT segment_start_ms AS startMs, segment_end_ms AS endMs FROM imm_subtitle_lines WHERE line_id = 1',
|
||||
)
|
||||
.get() as { startMs: number; endMs: number };
|
||||
assert.equal(kept.startMs, 10_000);
|
||||
assert.equal(kept.endMs, 10_000 + 40 * 40);
|
||||
|
||||
assert.equal(summary.samples.length, 1);
|
||||
assert.equal(summary.samples[0]!.text, '飛び上がる');
|
||||
assert.equal(summary.samples[0]!.frames, 40);
|
||||
assert.equal(summary.samples[0]!.videoTitle, 'Ep 1');
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('ordinary repeated dialogue survives', () => {
|
||||
// Six contiguous `飛び上がる`, each held for a normal beat rather than a frame.
|
||||
const lines = Array.from({ length: 6 }, (_, index) => ({
|
||||
session: 1,
|
||||
text: '飛び上がる',
|
||||
startMs: 5_000 + index * 800,
|
||||
endMs: 5_000 + (index + 1) * 800,
|
||||
}));
|
||||
const { db, dbPath } = createDb(lines);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(summary.removedLines, 0);
|
||||
assert.equal(countLines(db), 6);
|
||||
assert.equal(wordFrequency(db), 6);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a long run of quarter-second frames is still a burst', () => {
|
||||
// Between the timing-only bound (0.1s) and the animation-frame bound (0.3s): heavier
|
||||
// typesetting lands here, and the run length is what makes it conclusive.
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 6, 250));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 1);
|
||||
assert.equal(summary.removedLines, 5);
|
||||
assert.equal(countLines(db), 1);
|
||||
assert.equal(wordFrequency(db), 1);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a qualifying short-frame burst may end with one long hold frame', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 8, 40),
|
||||
{ session: 1, text: '飛び上がる', startMs: 10_320, endMs: 12_320 },
|
||||
]);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 1);
|
||||
assert.equal(summary.removedLines, 8);
|
||||
assert.equal(countLines(db), 1);
|
||||
assert.equal(wordFrequency(db), 1);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a long event before the final frame prevents burst cleanup', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 5, 40),
|
||||
{ session: 1, text: '飛び上がる', startMs: 10_200, endMs: 12_200 },
|
||||
{ session: 1, text: '飛び上がる', startMs: 12_200, endMs: 12_240 },
|
||||
]);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 7);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a run of frames longer than the animation bound survives', () => {
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 6, 400));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 6);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('the four-frame residue the live gate stores is cleaned up', () => {
|
||||
// The streaming gate records the first four frames of a burst before the run is long
|
||||
// enough to recognise. Four contiguous identical events under the strict timing-only
|
||||
// bound are that residue, and no real dialogue.
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 1_000, 4, 40));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 1);
|
||||
assert.equal(summary.removedLines, 3);
|
||||
assert.equal(countLines(db), 1);
|
||||
assert.equal(wordFrequency(db), 1);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a four-frame run above the strict frame bound survives', () => {
|
||||
// Long enough per event to be plausible dialogue; only a five-event run may use the
|
||||
// looser animation-frame bound.
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 1_000, 4, 250));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 4);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('an explicit minRunLength raises the bar', () => {
|
||||
// Five quarter-second frames qualify under the defaults; a cautious run asking for six
|
||||
// leaves them alone. Above the strict bound, so the residue rule stays out of it.
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 5, 250));
|
||||
|
||||
try {
|
||||
const preview = cleanupDuplicateSubtitleLines(db, { dryRun: true });
|
||||
assert.equal(preview.burstGroups, 1);
|
||||
|
||||
const summary = cleanupDuplicateSubtitleLines(db, { minRunLength: 6 });
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 5);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('an explicit maxFrameSeconds tightens the frame bound', () => {
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 6, 250));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db, { maxFrameSeconds: 0.2 });
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 6);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a non-finite maxFrameSeconds falls back to the default bound', () => {
|
||||
// Six normal-beat lines: Infinity must not turn every event into a "short frame".
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 6, 800));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db, { maxFrameSeconds: Infinity });
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 6);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('sampleLimit zero removes bursts but reports no samples', () => {
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 40, 40));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db, { sampleLimit: 0 });
|
||||
|
||||
assert.equal(summary.removedLines, 39);
|
||||
assert.deepEqual(summary.samples, []);
|
||||
assert.equal(countLines(db), 1);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a short run below every threshold survives', () => {
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 1_000, 3, 40));
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 0);
|
||||
assert.equal(countLines(db), 3);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('interleaved dual-line karaoke collapses each line to one row', () => {
|
||||
// Kanji and romaji lines frame-flipped together, the way fansub OPs are typeset. The
|
||||
// rows arrive interleaved in time order; each text must still chain into its own run.
|
||||
const kanji = karaokeFrames(1, '飛び上がる', 10_000, 20, 60);
|
||||
const romaji = karaokeFrames(1, 'tobiagaru', 10_001, 20, 60);
|
||||
const interleaved = [...kanji, ...romaji].sort((a, b) => a.startMs - b.startMs);
|
||||
const { db, dbPath } = createDb(interleaved);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 2);
|
||||
assert.equal(summary.removedLines, 38);
|
||||
assert.equal(countLines(db), 2);
|
||||
assert.equal(wordFrequency(db), 2);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('the same line in a rewatch session is never merged into the first watch', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 6, 40),
|
||||
...karaokeFrames(2, '飛び上がる', 10_000, 6, 40),
|
||||
]);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 2);
|
||||
assert.equal(summary.removedLines, 10);
|
||||
// One surviving line per session, not one across both.
|
||||
assert.equal(countLines(db), 2);
|
||||
assert.equal(wordFrequency(db), 2);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a gap between runs splits them', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 6, 40),
|
||||
...karaokeFrames(1, '飛び上がる', 60_000, 6, 40),
|
||||
]);
|
||||
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db);
|
||||
|
||||
assert.equal(summary.burstGroups, 2);
|
||||
assert.equal(countLines(db), 2);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('a dry run reports what an apply would do and writes nothing', () => {
|
||||
const { db, dbPath } = createDb(karaokeFrames(1, '飛び上がる', 10_000, 40, 40));
|
||||
|
||||
try {
|
||||
const preview = cleanupDuplicateSubtitleLines(db, { dryRun: true });
|
||||
|
||||
assert.equal(preview.dryRun, true);
|
||||
assert.equal(preview.removedLines, 39);
|
||||
assert.equal(countLines(db), 40);
|
||||
assert.equal(wordFrequency(db), 40);
|
||||
|
||||
const applied = cleanupDuplicateSubtitleLines(db);
|
||||
assert.equal(applied.removedLines, preview.removedLines);
|
||||
assert.equal(applied.removedWordOccurrences, preview.removedWordOccurrences);
|
||||
assert.equal(countLines(db), 1);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('the lookback window leaves older bursts alone', () => {
|
||||
const recentMs = BASE_MS;
|
||||
const oldMs = BASE_MS - 40 * DAY_MS;
|
||||
const { db, dbPath } = createDb([
|
||||
...karaokeFrames(1, '飛び上がる', 10_000, 6, 40).map((line) => ({
|
||||
...line,
|
||||
createdMs: oldMs,
|
||||
})),
|
||||
...karaokeFrames(2, '飛び上がる', 10_000, 6, 40).map((line) => ({
|
||||
...line,
|
||||
createdMs: recentMs,
|
||||
})),
|
||||
]);
|
||||
|
||||
globalThis.__subminerTestNowMs = BASE_MS;
|
||||
try {
|
||||
const summary = cleanupDuplicateSubtitleLines(db, { lookbackDays: 30 });
|
||||
|
||||
assert.equal(summary.lookbackDays, 30);
|
||||
assert.equal(summary.scannedLines, 6);
|
||||
assert.equal(summary.burstGroups, 1);
|
||||
assert.equal(summary.removedLines, 5);
|
||||
// Six untouched old frames plus the one surviving recent line.
|
||||
assert.equal(countLines(db), 7);
|
||||
assert.equal(wordFrequency(db), 7);
|
||||
} finally {
|
||||
globalThis.__subminerTestNowMs = undefined;
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
@@ -329,3 +329,28 @@ test('upgrading an older database backfills seen_ms from the subtitle lines', ()
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('an extreme-moving delete keeps subtraction-exact frequency instead of re-summing', () => {
|
||||
const { db, dbPath } = createDb([
|
||||
{ session: 1, wordId: 7, dayOffset: 0, count: 2 },
|
||||
{ session: 2, wordId: 7, dayOffset: 3, count: 1 },
|
||||
]);
|
||||
|
||||
try {
|
||||
// Simulate drift: the stored total is higher than the occurrences justify.
|
||||
// The extremes move via index seeks while the count stays a pure
|
||||
// subtraction; drifted counts reconcile only at the zero-crossing repair
|
||||
// or via the cleanup command, never by rescanning every occurrence here.
|
||||
db.prepare('UPDATE imm_words SET frequency = 10 WHERE id = 7').run();
|
||||
|
||||
deleteSession(db, 1);
|
||||
|
||||
const word = readWord(db, 7);
|
||||
assert.equal(word?.frequency, 10 - 2);
|
||||
assert.equal(word?.firstSeen, Math.floor((BASE_MS + 3 * DAY_MS) / 1000));
|
||||
assert.equal(word?.lastSeen, Math.floor((BASE_MS + 3 * DAY_MS) / 1000));
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { startSessionRecord } from '../session.js';
|
||||
import { applySessionLifetimeSummary, rebuildLifetimeSummaries } from '../lifetime.js';
|
||||
import { deleteMaintenanceBatch } from '../query-delete-maintenance.js';
|
||||
import { toDbTimestamp } from '../query-shared.js';
|
||||
import {
|
||||
BASE_MS,
|
||||
DAY_MS,
|
||||
cleanRow,
|
||||
createDb,
|
||||
seedAnime,
|
||||
seedEndedSession,
|
||||
seedVideo,
|
||||
snapshotAnime,
|
||||
snapshotGlobal,
|
||||
snapshotMedia,
|
||||
} from './lifetime-test-fixtures.js';
|
||||
|
||||
test('fractional lifetime metrics stay normalized across apply, rebuild, and delete', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const videoId = seedVideo(db, null, 'fractional-metrics');
|
||||
const seedFractionalSession = (
|
||||
startedAtMs: number,
|
||||
metrics: { activeMs: number; cards: number; lines: number; tokens: number },
|
||||
) => {
|
||||
const { state } = startSessionRecord(db, videoId, startedAtMs);
|
||||
state.activeWatchedMs = metrics.activeMs;
|
||||
state.cardsMined = metrics.cards;
|
||||
state.linesSeen = metrics.lines;
|
||||
state.tokensSeen = metrics.tokens;
|
||||
const endedAtMs = startedAtMs + 2_000;
|
||||
db.prepare(
|
||||
`UPDATE imm_sessions SET
|
||||
ended_at_ms = ?,
|
||||
active_watched_ms = ?,
|
||||
cards_mined = ?,
|
||||
lines_seen = ?,
|
||||
tokens_seen = ?
|
||||
WHERE session_id = ?`,
|
||||
).run(
|
||||
toDbTimestamp(endedAtMs),
|
||||
metrics.activeMs,
|
||||
metrics.cards,
|
||||
metrics.lines,
|
||||
metrics.tokens,
|
||||
state.sessionId,
|
||||
);
|
||||
return { state, endedAtMs };
|
||||
};
|
||||
const readMediaMetrics = () =>
|
||||
cleanRow<{
|
||||
total_sessions: number;
|
||||
total_active_ms: number;
|
||||
total_cards: number;
|
||||
total_lines_seen: number;
|
||||
total_tokens_seen: number;
|
||||
}>(
|
||||
db
|
||||
.prepare(
|
||||
`SELECT total_sessions, total_active_ms, total_cards,
|
||||
total_lines_seen, total_tokens_seen
|
||||
FROM imm_lifetime_media WHERE video_id = ?`,
|
||||
)
|
||||
.get(videoId),
|
||||
);
|
||||
|
||||
const withoutTelemetry = seedFractionalSession(BASE_MS, {
|
||||
activeMs: 1_234.9,
|
||||
cards: 2.8,
|
||||
lines: 3.7,
|
||||
tokens: 4.6,
|
||||
});
|
||||
applySessionLifetimeSummary(db, withoutTelemetry.state, withoutTelemetry.endedAtMs);
|
||||
assert.deepEqual(readMediaMetrics(), {
|
||||
total_sessions: 1,
|
||||
total_active_ms: 1_234,
|
||||
total_cards: 2,
|
||||
total_lines_seen: 3,
|
||||
total_tokens_seen: 4,
|
||||
});
|
||||
|
||||
const withTelemetry = seedFractionalSession(BASE_MS + DAY_MS, {
|
||||
activeMs: 9_999.9,
|
||||
cards: 9.9,
|
||||
lines: 9.9,
|
||||
tokens: 9.9,
|
||||
});
|
||||
db.prepare(
|
||||
`INSERT INTO imm_session_telemetry (
|
||||
session_id, sample_ms, active_watched_ms, cards_mined, lines_seen, tokens_seen
|
||||
) VALUES (?, ?, ?, ?, ?, ?)`,
|
||||
).run(withTelemetry.state.sessionId, withTelemetry.endedAtMs, 2_345.9, 5.8, 6.7, 7.6);
|
||||
applySessionLifetimeSummary(db, withTelemetry.state, withTelemetry.endedAtMs);
|
||||
assert.deepEqual(readMediaMetrics(), {
|
||||
total_sessions: 2,
|
||||
total_active_ms: 3_579,
|
||||
total_cards: 7,
|
||||
total_lines_seen: 9,
|
||||
total_tokens_seen: 11,
|
||||
});
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId: withTelemetry.state.sessionId }]);
|
||||
const retainedMetrics = {
|
||||
total_sessions: 1,
|
||||
total_active_ms: 1_234,
|
||||
total_cards: 2,
|
||||
total_lines_seen: 3,
|
||||
total_tokens_seen: 4,
|
||||
};
|
||||
assert.deepEqual(readMediaMetrics(), retainedMetrics, 'delete subtracts floored telemetry');
|
||||
|
||||
rebuildLifetimeSummaries(db);
|
||||
assert.deepEqual(
|
||||
readMediaMetrics(),
|
||||
retainedMetrics,
|
||||
'rebuild floors session-row fallback values',
|
||||
);
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId: withoutTelemetry.state.sessionId }]);
|
||||
assert.deepEqual(snapshotMedia(db), [], 'delete subtracts the normalized metrics exactly');
|
||||
assert.deepEqual(snapshotGlobal(db), {
|
||||
total_sessions: 0,
|
||||
total_active_ms: 0,
|
||||
total_cards: 0,
|
||||
active_days: 0,
|
||||
episodes_started: 0,
|
||||
episodes_completed: 0,
|
||||
anime_completed: 0,
|
||||
});
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('incremental delete maintenance matches a full rebuild when no history is pruned', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const animeA = seedAnime(db, 'Anime A', 2);
|
||||
const animeB = seedAnime(db, 'Anime B', 1);
|
||||
const videoA1 = seedVideo(db, animeA, 'anime-a-ep1', { watched: true });
|
||||
const videoA2 = seedVideo(db, animeA, 'anime-a-ep2', { watched: true });
|
||||
const videoB1 = seedVideo(db, animeB, 'anime-b-ep1', { watched: true });
|
||||
const videoLoose = seedVideo(db, null, 'loose-video');
|
||||
|
||||
seedEndedSession(db, videoA1, BASE_MS, { activeMs: 60_000, cards: 2, lines: 30, tokens: 200 });
|
||||
const deletedSessionId = seedEndedSession(db, videoA1, BASE_MS + DAY_MS, {
|
||||
activeMs: 45_000,
|
||||
cards: 1,
|
||||
lines: 20,
|
||||
tokens: 100,
|
||||
});
|
||||
seedEndedSession(db, videoA2, BASE_MS + 2 * DAY_MS, { activeMs: 90_000, cards: 3 });
|
||||
seedEndedSession(db, videoB1, BASE_MS + 3 * DAY_MS, { activeMs: 30_000 });
|
||||
seedEndedSession(db, videoLoose, BASE_MS + 4 * DAY_MS, { activeMs: 15_000 });
|
||||
|
||||
rebuildLifetimeSummaries(db);
|
||||
|
||||
deleteMaintenanceBatch(db, [
|
||||
{ kind: 'session', sessionId: deletedSessionId },
|
||||
{ kind: 'video', videoId: videoLoose },
|
||||
{ kind: 'anime', animeId: animeB },
|
||||
]);
|
||||
|
||||
const incrementalGlobal = snapshotGlobal(db);
|
||||
const incrementalMedia = snapshotMedia(db);
|
||||
const incrementalAnime = snapshotAnime(db);
|
||||
|
||||
// With every session still retained, subtracting must land on exactly the
|
||||
// state a from-scratch rebuild computes.
|
||||
rebuildLifetimeSummaries(db);
|
||||
assert.deepEqual(incrementalGlobal, snapshotGlobal(db));
|
||||
assert.deepEqual(incrementalMedia, snapshotMedia(db));
|
||||
assert.deepEqual(incrementalAnime, snapshotAnime(db));
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting a retained session preserves lifetime history from pruned sessions', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const animeId = seedAnime(db, 'Pruned Anime', null);
|
||||
const videoId = seedVideo(db, animeId, 'pruned-ep1');
|
||||
const prunedSessionId = seedEndedSession(db, videoId, BASE_MS, {
|
||||
activeMs: 120_000,
|
||||
cards: 4,
|
||||
lines: 50,
|
||||
tokens: 400,
|
||||
});
|
||||
const retainedSessionId = seedEndedSession(db, videoId, BASE_MS + DAY_MS, {
|
||||
activeMs: 30_000,
|
||||
cards: 1,
|
||||
lines: 10,
|
||||
tokens: 80,
|
||||
});
|
||||
rebuildLifetimeSummaries(db);
|
||||
|
||||
// Simulate raw-session retention pruning the older session. Lifetime
|
||||
// summaries intentionally keep its contribution.
|
||||
db.prepare('DELETE FROM imm_sessions WHERE session_id = ?').run(prunedSessionId);
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId: retainedSessionId }]);
|
||||
|
||||
const globalRow = snapshotGlobal(db);
|
||||
assert.equal(globalRow.total_sessions, 1, 'pruned session contribution survives the delete');
|
||||
assert.equal(globalRow.total_active_ms, 120_000);
|
||||
assert.equal(globalRow.total_cards, 4);
|
||||
assert.equal(globalRow.episodes_started, 1);
|
||||
// The pruned session's day stays counted (pruning never subtracts); only
|
||||
// the deleted retained session's day is dropped.
|
||||
assert.equal(globalRow.active_days, 1);
|
||||
|
||||
const mediaRow = db
|
||||
.prepare(
|
||||
'SELECT total_sessions, total_active_ms, total_cards FROM imm_lifetime_media WHERE video_id = ?',
|
||||
)
|
||||
.get(videoId);
|
||||
assert.deepEqual(
|
||||
cleanRow<{ total_sessions: number; total_active_ms: number; total_cards: number }>(mediaRow),
|
||||
{ total_sessions: 1, total_active_ms: 120_000, total_cards: 4 },
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('active_days only drops when the last session of a local day is deleted', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const videoId = seedVideo(db, null, 'same-day');
|
||||
const firstSessionId = seedEndedSession(db, videoId, BASE_MS, { activeMs: 10_000 });
|
||||
const secondSessionId = seedEndedSession(db, videoId, BASE_MS + 3_600_000, {
|
||||
activeMs: 20_000,
|
||||
});
|
||||
rebuildLifetimeSummaries(db);
|
||||
assert.equal(snapshotGlobal(db).active_days, 1);
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId: firstSessionId }]);
|
||||
assert.equal(snapshotGlobal(db).active_days, 1, 'day still has a session');
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId: secondSessionId }]);
|
||||
assert.equal(snapshotGlobal(db).active_days, 0, 'day lost its last session');
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('deleting a video updates anime and global rollups without a rebuild', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const animeId = seedAnime(db, 'Two Episode Anime', 2);
|
||||
const videoEp1 = seedVideo(db, animeId, 'two-ep-1', { watched: true });
|
||||
const videoEp2 = seedVideo(db, animeId, 'two-ep-2', { watched: true });
|
||||
seedEndedSession(db, videoEp1, BASE_MS, { activeMs: 60_000, cards: 2 });
|
||||
seedEndedSession(db, videoEp2, BASE_MS + DAY_MS, { activeMs: 40_000, cards: 1 });
|
||||
rebuildLifetimeSummaries(db);
|
||||
assert.equal(snapshotGlobal(db).anime_completed, 1);
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'video', videoId: videoEp2 }]);
|
||||
|
||||
const globalRow = snapshotGlobal(db);
|
||||
assert.equal(globalRow.total_sessions, 1);
|
||||
assert.equal(globalRow.total_active_ms, 60_000);
|
||||
assert.equal(globalRow.episodes_started, 1);
|
||||
assert.equal(globalRow.episodes_completed, 1);
|
||||
assert.equal(globalRow.anime_completed, 0, 'anime no longer has all episodes completed');
|
||||
|
||||
const animeRow = db
|
||||
.prepare(
|
||||
'SELECT total_sessions, episodes_started, episodes_completed FROM imm_lifetime_anime WHERE anime_id = ?',
|
||||
)
|
||||
.get(animeId);
|
||||
assert.deepEqual(
|
||||
cleanRow<{
|
||||
total_sessions: number;
|
||||
episodes_started: number;
|
||||
episodes_completed: number;
|
||||
}>(animeRow),
|
||||
{ total_sessions: 1, episodes_started: 1, episodes_completed: 1 },
|
||||
);
|
||||
|
||||
deleteMaintenanceBatch(db, [{ kind: 'video', videoId: videoEp1 }]);
|
||||
assert.equal(
|
||||
(db.prepare('SELECT COUNT(*) AS total FROM imm_lifetime_anime').get() as { total: number })
|
||||
.total,
|
||||
0,
|
||||
'anime lifetime row is dropped once no episodes remain',
|
||||
);
|
||||
assert.deepEqual(snapshotGlobal(db), {
|
||||
total_sessions: 0,
|
||||
total_active_ms: 0,
|
||||
total_cards: 0,
|
||||
active_days: 0,
|
||||
episodes_started: 0,
|
||||
episodes_completed: 0,
|
||||
anime_completed: 0,
|
||||
});
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,94 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { rebuildLifetimeSummaries, repairLifetimeSummariesFromMedia } from '../lifetime.js';
|
||||
import {
|
||||
BASE_MS,
|
||||
DAY_MS,
|
||||
cleanRow,
|
||||
createDb,
|
||||
seedAnime,
|
||||
seedEndedSession,
|
||||
seedVideo,
|
||||
snapshotAnime,
|
||||
snapshotGlobal,
|
||||
snapshotMedia,
|
||||
} from './lifetime-test-fixtures.js';
|
||||
|
||||
test('repair after a video moves between anime matches a full rebuild', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const animeA = seedAnime(db, 'Move Source', 2);
|
||||
const animeB = seedAnime(db, 'Move Target', 2);
|
||||
const movedVideo = seedVideo(db, animeA, 'moved-ep', { watched: true });
|
||||
const stayingVideo = seedVideo(db, animeA, 'staying-ep');
|
||||
const targetVideo = seedVideo(db, animeB, 'target-ep', { watched: true });
|
||||
seedEndedSession(db, movedVideo, BASE_MS, { activeMs: 60_000, cards: 2 });
|
||||
seedEndedSession(db, stayingVideo, BASE_MS + DAY_MS, { activeMs: 30_000 });
|
||||
seedEndedSession(db, targetVideo, BASE_MS + 2 * DAY_MS, { activeMs: 45_000, cards: 1 });
|
||||
rebuildLifetimeSummaries(db);
|
||||
|
||||
// Simulate a library merge reassigning the episode to the other anime.
|
||||
db.prepare('UPDATE imm_videos SET anime_id = ? WHERE video_id = ?').run(animeB, movedVideo);
|
||||
repairLifetimeSummariesFromMedia(db);
|
||||
|
||||
const repairedGlobal = snapshotGlobal(db);
|
||||
const repairedMedia = snapshotMedia(db);
|
||||
const repairedAnime = snapshotAnime(db);
|
||||
|
||||
rebuildLifetimeSummaries(db);
|
||||
assert.deepEqual(repairedGlobal, snapshotGlobal(db));
|
||||
assert.deepEqual(repairedMedia, snapshotMedia(db));
|
||||
assert.deepEqual(repairedAnime, snapshotAnime(db));
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('repair preserves lifetime history from pruned sessions where a rebuild would not', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
const animeId = seedAnime(db, 'Repair Anime', null);
|
||||
const videoId = seedVideo(db, animeId, 'repair-ep');
|
||||
const prunedSessionId = seedEndedSession(db, videoId, BASE_MS, {
|
||||
activeMs: 90_000,
|
||||
cards: 3,
|
||||
});
|
||||
seedEndedSession(db, videoId, BASE_MS + DAY_MS, { activeMs: 30_000, cards: 1 });
|
||||
rebuildLifetimeSummaries(db);
|
||||
|
||||
db.prepare('DELETE FROM imm_sessions WHERE session_id = ?').run(prunedSessionId);
|
||||
repairLifetimeSummariesFromMedia(db);
|
||||
|
||||
const globalRow = snapshotGlobal(db);
|
||||
assert.equal(globalRow.total_sessions, 2, 'repair keeps the pruned session contribution');
|
||||
assert.equal(globalRow.total_active_ms, 120_000);
|
||||
assert.equal(globalRow.total_cards, 4);
|
||||
assert.equal(globalRow.active_days, 2, 'repair never subtracts active days');
|
||||
|
||||
const animeRow = db
|
||||
.prepare('SELECT total_sessions FROM imm_lifetime_anime WHERE anime_id = ?')
|
||||
.get(animeId);
|
||||
assert.equal(cleanRow<{ total_sessions: number }>(animeRow).total_sessions, 2);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('repair leaves a caller-owned transaction intact when its begin fails', () => {
|
||||
const db = createDb();
|
||||
try {
|
||||
db.exec('BEGIN');
|
||||
const animeId = seedAnime(db, 'Caller Transaction', null);
|
||||
|
||||
assert.throws(() => repairLifetimeSummariesFromMedia(db), /transaction/i);
|
||||
assert.ok(
|
||||
db.prepare('SELECT 1 FROM imm_anime WHERE anime_id = ?').get(animeId),
|
||||
'the repair did not roll back the caller transaction',
|
||||
);
|
||||
|
||||
db.exec('ROLLBACK');
|
||||
assert.equal(db.prepare('SELECT 1 FROM imm_anime WHERE anime_id = ?').get(animeId), undefined);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,157 @@
|
||||
import { Database } from '../sqlite.js';
|
||||
import type { DatabaseSync } from '../sqlite.js';
|
||||
import {
|
||||
applyPragmas,
|
||||
ensureSchema,
|
||||
getOrCreateAnimeRecord,
|
||||
getOrCreateVideoRecord,
|
||||
linkVideoToAnimeRecord,
|
||||
} from '../storage.js';
|
||||
import { startSessionRecord } from '../session.js';
|
||||
import { toDbTimestamp } from '../query-shared.js';
|
||||
|
||||
const SOURCE_TYPE_LOCAL = 1;
|
||||
export const DAY_MS = 86_400_000;
|
||||
// Noon UTC keeps every seeded timestamp on the same local day regardless of
|
||||
// the timezone the test host runs in.
|
||||
export const BASE_MS = Date.UTC(2026, 0, 5, 12, 0, 0);
|
||||
|
||||
export function createDb(): DatabaseSync {
|
||||
const db = new Database(':memory:');
|
||||
applyPragmas(db);
|
||||
ensureSchema(db);
|
||||
return db;
|
||||
}
|
||||
|
||||
export function seedAnime(db: DatabaseSync, title: string, episodesTotal: number | null): number {
|
||||
const animeId = getOrCreateAnimeRecord(db, {
|
||||
parsedTitle: title,
|
||||
canonicalTitle: title,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: null,
|
||||
});
|
||||
if (episodesTotal !== null) {
|
||||
db.prepare('UPDATE imm_anime SET episodes_total = ? WHERE anime_id = ?').run(
|
||||
episodesTotal,
|
||||
animeId,
|
||||
);
|
||||
}
|
||||
return animeId;
|
||||
}
|
||||
|
||||
export function seedVideo(
|
||||
db: DatabaseSync,
|
||||
animeId: number | null,
|
||||
name: string,
|
||||
options: { watched?: boolean } = {},
|
||||
): number {
|
||||
const videoId = getOrCreateVideoRecord(db, `local:/tmp/${name}.mkv`, {
|
||||
canonicalTitle: name,
|
||||
sourcePath: `/tmp/${name}.mkv`,
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
if (animeId !== null) {
|
||||
linkVideoToAnimeRecord(db, videoId, {
|
||||
animeId,
|
||||
parsedBasename: `${name}.mkv`,
|
||||
parsedTitle: name,
|
||||
parsedSeason: 1,
|
||||
parsedEpisode: 1,
|
||||
parserSource: 'test',
|
||||
parserConfidence: 1,
|
||||
parseMetadataJson: null,
|
||||
});
|
||||
}
|
||||
if (options.watched) {
|
||||
db.prepare('UPDATE imm_videos SET watched = 1 WHERE video_id = ?').run(videoId);
|
||||
}
|
||||
return videoId;
|
||||
}
|
||||
|
||||
export function seedEndedSession(
|
||||
db: DatabaseSync,
|
||||
videoId: number,
|
||||
startedAtMs: number,
|
||||
metrics: { activeMs: number; cards?: number; lines?: number; tokens?: number },
|
||||
): number {
|
||||
const sessionId = startSessionRecord(db, videoId, startedAtMs).sessionId;
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_sessions SET
|
||||
ended_at_ms = ?,
|
||||
active_watched_ms = ?,
|
||||
total_watched_ms = ?,
|
||||
cards_mined = ?,
|
||||
lines_seen = ?,
|
||||
tokens_seen = ?
|
||||
WHERE session_id = ?
|
||||
`,
|
||||
).run(
|
||||
toDbTimestamp(startedAtMs + metrics.activeMs),
|
||||
metrics.activeMs,
|
||||
metrics.activeMs,
|
||||
metrics.cards ?? 0,
|
||||
metrics.lines ?? 0,
|
||||
metrics.tokens ?? 0,
|
||||
sessionId,
|
||||
);
|
||||
return sessionId;
|
||||
}
|
||||
|
||||
// libsql attaches a per-query `_metadata` property to result rows; strip it so
|
||||
// row snapshots can be compared with deepEqual.
|
||||
export function cleanRow<T>(row: unknown): T {
|
||||
const { _metadata: _ignored, ...rest } = row as Record<string, unknown>;
|
||||
return rest as T;
|
||||
}
|
||||
|
||||
export interface GlobalSnapshot {
|
||||
total_sessions: number;
|
||||
total_active_ms: number;
|
||||
total_cards: number;
|
||||
active_days: number;
|
||||
episodes_started: number;
|
||||
episodes_completed: number;
|
||||
anime_completed: number;
|
||||
}
|
||||
|
||||
export function snapshotGlobal(db: DatabaseSync): GlobalSnapshot {
|
||||
const row = db
|
||||
.prepare(
|
||||
`SELECT total_sessions, total_active_ms, total_cards, active_days,
|
||||
episodes_started, episodes_completed, anime_completed
|
||||
FROM imm_lifetime_global WHERE global_id = 1`,
|
||||
)
|
||||
.get();
|
||||
return cleanRow<GlobalSnapshot>(row);
|
||||
}
|
||||
|
||||
export function snapshotMedia(db: DatabaseSync): unknown[] {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT video_id, total_sessions, total_active_ms, total_cards,
|
||||
total_lines_seen, total_tokens_seen, completed,
|
||||
CAST(first_watched_ms AS REAL) AS first_watched,
|
||||
CAST(last_watched_ms AS REAL) AS last_watched
|
||||
FROM imm_lifetime_media ORDER BY video_id`,
|
||||
)
|
||||
.all()
|
||||
.map((row) => cleanRow(row));
|
||||
}
|
||||
|
||||
export function snapshotAnime(db: DatabaseSync): unknown[] {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT anime_id, total_sessions, total_active_ms, total_cards,
|
||||
total_lines_seen, total_tokens_seen, episodes_started, episodes_completed,
|
||||
CAST(first_watched_ms AS REAL) AS first_watched,
|
||||
CAST(last_watched_ms AS REAL) AS last_watched
|
||||
FROM imm_lifetime_anime ORDER BY anime_id`,
|
||||
)
|
||||
.all()
|
||||
.map((row) => cleanRow(row));
|
||||
}
|
||||
@@ -1,169 +0,0 @@
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { animeSeasonsAreMergeCompatible, getParsedSeasonsForAnime } from './anime-merge';
|
||||
import { toDbTimestamp } from './query-shared';
|
||||
import { normalizeAnimeIdentityKey } from './storage';
|
||||
import { nowMs } from './time';
|
||||
|
||||
export interface AnimeMergeRecommendation {
|
||||
recommendationId: number;
|
||||
animeIds: [number, number];
|
||||
}
|
||||
|
||||
export interface AnimeConflictRecommendationOptions {
|
||||
survivor?: 'target' | 'existing';
|
||||
/** Automatic matches must be exact; manual assignment is authoritative. */
|
||||
matchConfidence?: 'exact' | 'weak' | 'manual';
|
||||
}
|
||||
|
||||
interface AnimeTitleRow {
|
||||
canonical_title: string;
|
||||
title_romaji: string | null;
|
||||
title_english: string | null;
|
||||
title_native: string | null;
|
||||
}
|
||||
|
||||
function getAnimeTitles(db: DatabaseSync, animeId: number): AnimeTitleRow | null {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT canonical_title, title_romaji, title_english, title_native
|
||||
FROM imm_anime
|
||||
WHERE anime_id = ?`,
|
||||
)
|
||||
.get(animeId) as AnimeTitleRow | null;
|
||||
}
|
||||
|
||||
function getParsedTitles(db: DatabaseSync, animeId: number): Array<string | null> {
|
||||
return (
|
||||
db.prepare('SELECT parsed_title FROM imm_videos WHERE anime_id = ?').all(animeId) as Array<{
|
||||
parsed_title: string | null;
|
||||
}>
|
||||
).map((row) => row.parsed_title);
|
||||
}
|
||||
|
||||
function stripSeasonIdentitySuffix(title: string): string {
|
||||
return title
|
||||
.replace(/\bseason\s*\d{1,2}\b/gi, ' ')
|
||||
.replace(/\b\d{1,2}(?:st|nd|rd|th)\s+season\b/gi, ' ')
|
||||
.replace(/\bs\d{1,2}\b/gi, ' ');
|
||||
}
|
||||
|
||||
export function hasExactStoredTitleMatch(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
conflictAnimeId: number,
|
||||
): boolean {
|
||||
const target = getAnimeTitles(db, targetAnimeId);
|
||||
const conflict = getAnimeTitles(db, conflictAnimeId);
|
||||
if (!target || !conflict) return false;
|
||||
const targetKeys = [target.canonical_title, ...getParsedTitles(db, targetAnimeId)]
|
||||
.filter((title): title is string => Boolean(title?.trim()))
|
||||
.map((title) => normalizeAnimeIdentityKey(stripSeasonIdentitySuffix(title)))
|
||||
.filter(Boolean);
|
||||
const anilistTitleKeys = [
|
||||
conflict.title_romaji,
|
||||
conflict.title_english,
|
||||
conflict.title_native,
|
||||
conflict.canonical_title,
|
||||
]
|
||||
.filter((title): title is string => Boolean(title?.trim()))
|
||||
.map((title) => normalizeAnimeIdentityKey(stripSeasonIdentitySuffix(title)))
|
||||
.filter(Boolean);
|
||||
return targetKeys.some((key) => anilistTitleKeys.includes(key));
|
||||
}
|
||||
|
||||
export function shouldRecommendAnilistConflict(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
conflictAnimeId: number,
|
||||
options: AnimeConflictRecommendationOptions,
|
||||
): boolean {
|
||||
if (options.survivor === 'target' || options.matchConfidence === 'manual') return false;
|
||||
if (
|
||||
!animeSeasonsAreMergeCompatible(
|
||||
getParsedSeasonsForAnime(db, targetAnimeId),
|
||||
getParsedSeasonsForAnime(db, conflictAnimeId),
|
||||
)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
options.matchConfidence === 'weak' ||
|
||||
(options.matchConfidence === undefined &&
|
||||
!hasExactStoredTitleMatch(db, targetAnimeId, conflictAnimeId))
|
||||
);
|
||||
}
|
||||
|
||||
export function recordAnimeMergeRecommendation(
|
||||
db: DatabaseSync,
|
||||
firstCandidateAnimeId: number,
|
||||
secondCandidateAnimeId: number,
|
||||
anilistId: number,
|
||||
): void {
|
||||
const firstAnimeId = Math.min(firstCandidateAnimeId, secondCandidateAnimeId);
|
||||
const secondAnimeId = Math.max(firstCandidateAnimeId, secondCandidateAnimeId);
|
||||
const timestamp = toDbTimestamp(nowMs());
|
||||
db.prepare(
|
||||
`INSERT INTO imm_anime_merge_recommendations(
|
||||
first_anime_id, second_anime_id, anilist_id, status, CREATED_DATE, LAST_UPDATE_DATE
|
||||
) VALUES (?, ?, ?, 'pending', ?, ?)
|
||||
ON CONFLICT(first_anime_id, second_anime_id, anilist_id) DO UPDATE SET
|
||||
LAST_UPDATE_DATE = excluded.LAST_UPDATE_DATE`,
|
||||
).run(firstAnimeId, secondAnimeId, anilistId, timestamp, timestamp);
|
||||
}
|
||||
|
||||
export function hasDismissedAnimeMergeRecommendation(
|
||||
db: DatabaseSync,
|
||||
firstCandidateAnimeId: number,
|
||||
secondCandidateAnimeId: number,
|
||||
): boolean {
|
||||
const firstAnimeId = Math.min(firstCandidateAnimeId, secondCandidateAnimeId);
|
||||
const secondAnimeId = Math.max(firstCandidateAnimeId, secondCandidateAnimeId);
|
||||
return Boolean(
|
||||
db
|
||||
.prepare(
|
||||
`SELECT 1
|
||||
FROM imm_anime_merge_recommendations
|
||||
WHERE first_anime_id = ?
|
||||
AND second_anime_id = ?
|
||||
AND status = 'dismissed'
|
||||
LIMIT 1`,
|
||||
)
|
||||
.get(firstAnimeId, secondAnimeId),
|
||||
);
|
||||
}
|
||||
|
||||
export function getAnimeMergeRecommendations(db: DatabaseSync): AnimeMergeRecommendation[] {
|
||||
return (
|
||||
db
|
||||
.prepare(
|
||||
`SELECT recommendation_id AS recommendationId,
|
||||
first_anime_id AS firstAnimeId,
|
||||
second_anime_id AS secondAnimeId
|
||||
FROM imm_anime_merge_recommendations
|
||||
WHERE status = 'pending'
|
||||
ORDER BY recommendation_id ASC`,
|
||||
)
|
||||
.all() as Array<{
|
||||
recommendationId: number;
|
||||
firstAnimeId: number;
|
||||
secondAnimeId: number;
|
||||
}>
|
||||
).map((row) => ({
|
||||
recommendationId: row.recommendationId,
|
||||
animeIds: [row.firstAnimeId, row.secondAnimeId],
|
||||
}));
|
||||
}
|
||||
|
||||
export function dismissAnimeMergeRecommendation(
|
||||
db: DatabaseSync,
|
||||
recommendationId: number,
|
||||
): boolean {
|
||||
const result = db
|
||||
.prepare(
|
||||
`UPDATE imm_anime_merge_recommendations
|
||||
SET status = 'dismissed', LAST_UPDATE_DATE = ?
|
||||
WHERE recommendation_id = ? AND status = 'pending'`,
|
||||
)
|
||||
.run(toDbTimestamp(nowMs()), recommendationId) as { changes: number };
|
||||
return result.changes > 0;
|
||||
}
|
||||
@@ -1,296 +0,0 @@
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { recomputeLifetimeAnimeAggregatesInTransaction } from './lifetime';
|
||||
import { toDbTimestamp } from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
|
||||
/** Thrown when a move names an episode or destination entry that is not there. */
|
||||
export const UNKNOWN_MOVE_TARGET_MESSAGE = 'Unknown episode or target library entry';
|
||||
|
||||
export interface AnimeMergeSummary {
|
||||
/** Library entry that owns every moved episode once the merge finishes. */
|
||||
survivingAnimeId: number;
|
||||
/** Entries that were folded into the survivor and deleted. */
|
||||
mergedAnimeIds: number[];
|
||||
movedVideos: number;
|
||||
}
|
||||
|
||||
export interface VideoMoveSummary {
|
||||
targetAnimeId: number;
|
||||
/** Previous owner, or null when the episode had no library entry yet. */
|
||||
previousAnimeId: number | null;
|
||||
/** True when the previous owner was left empty and pruned. */
|
||||
removedPreviousAnime: boolean;
|
||||
}
|
||||
|
||||
interface AnimeMetadataRow {
|
||||
normalized_title_key: string;
|
||||
anilist_id: number | null;
|
||||
title_romaji: string | null;
|
||||
title_english: string | null;
|
||||
title_native: string | null;
|
||||
episodes_total: number | null;
|
||||
description: string | null;
|
||||
}
|
||||
|
||||
function emptyMergeSummary(survivingAnimeId: number): AnimeMergeSummary {
|
||||
return { survivingAnimeId, mergedAnimeIds: [], movedVideos: 0 };
|
||||
}
|
||||
|
||||
function runInTransaction<T>(db: DatabaseSync, work: () => T): T {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const result = work();
|
||||
db.exec('COMMIT');
|
||||
return result;
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function readAnimeMetadata(db: DatabaseSync, animeId: number): AnimeMetadataRow | null {
|
||||
return (db
|
||||
.prepare(
|
||||
`
|
||||
SELECT normalized_title_key, anilist_id, title_romaji, title_english, title_native, episodes_total, description
|
||||
FROM imm_anime
|
||||
WHERE anime_id = ?
|
||||
`,
|
||||
)
|
||||
.get(animeId) ?? null) as AnimeMetadataRow | null;
|
||||
}
|
||||
|
||||
function animeExists(db: DatabaseSync, animeId: number): boolean {
|
||||
return Boolean(db.prepare('SELECT 1 FROM imm_anime WHERE anime_id = ?').get(animeId));
|
||||
}
|
||||
|
||||
function hasAnimeReferences(db: DatabaseSync, animeId: number): boolean {
|
||||
const row = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT 1 AS found
|
||||
WHERE EXISTS (SELECT 1 FROM imm_videos WHERE anime_id = ?)
|
||||
OR EXISTS (SELECT 1 FROM imm_subtitle_lines WHERE anime_id = ?)
|
||||
`,
|
||||
)
|
||||
.get(animeId, animeId) as { found: number } | null;
|
||||
return Boolean(row);
|
||||
}
|
||||
|
||||
/**
|
||||
* Distinct explicit seasons behind a library entry. Videos with no parsed
|
||||
* season are ignored, so an entry built from `Show - 03.mkv` style filenames
|
||||
* reports an empty set rather than a bogus season.
|
||||
*/
|
||||
export function getParsedSeasonsForAnime(db: DatabaseSync, animeId: number): Set<number> {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT DISTINCT parsed_season AS season
|
||||
FROM imm_videos
|
||||
WHERE anime_id = ?
|
||||
AND parsed_season IS NOT NULL
|
||||
AND parsed_season > 0
|
||||
`,
|
||||
)
|
||||
.all(animeId) as Array<{ season: number }>;
|
||||
return new Set(rows.map((row) => row.season));
|
||||
}
|
||||
|
||||
/**
|
||||
* Two entries are safe to fold together when neither spans more than one
|
||||
* explicit season and they do not disagree about which season that is. A
|
||||
* seasonless entry is compatible with anything single-season: those are the
|
||||
* `Show - 03.mkv` vs `Show.S01E03.mkv` splits that produce duplicate cards.
|
||||
*/
|
||||
export function animeSeasonsAreMergeCompatible(a: Set<number>, b: Set<number>): boolean {
|
||||
if (a.size > 1 || b.size > 1) return false;
|
||||
if (a.size === 0 || b.size === 0) return true;
|
||||
return [...a][0] === [...b][0];
|
||||
}
|
||||
|
||||
/**
|
||||
* Fill in whatever the target is missing from a source row that is on its way
|
||||
* out. Must run after the source row is deleted: imm_anime.anilist_id is
|
||||
* UNIQUE, so the two rows cannot hold the same id at once.
|
||||
*/
|
||||
function absorbAnimeMetadata(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
source: AnimeMetadataRow | null,
|
||||
updatedAt: string,
|
||||
): void {
|
||||
if (!source) return;
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_anime
|
||||
SET
|
||||
anilist_id = COALESCE(anilist_id, ?),
|
||||
title_romaji = COALESCE(title_romaji, ?),
|
||||
title_english = COALESCE(title_english, ?),
|
||||
title_native = COALESCE(title_native, ?),
|
||||
episodes_total = COALESCE(episodes_total, ?),
|
||||
description = COALESCE(description, ?),
|
||||
LAST_UPDATE_DATE = ?
|
||||
WHERE anime_id = ?
|
||||
`,
|
||||
).run(
|
||||
source.anilist_id,
|
||||
source.title_romaji,
|
||||
source.title_english,
|
||||
source.title_native,
|
||||
source.episodes_total,
|
||||
source.description,
|
||||
updatedAt,
|
||||
targetAnimeId,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fold `sourceAnimeIds` into `targetAnimeId`: every episode and subtitle line
|
||||
* is repointed, metadata the target is missing is inherited from the sources,
|
||||
* and the emptied source rows are deleted.
|
||||
*
|
||||
* Assumes the caller already holds a write transaction and refreshes the
|
||||
* per-anime lifetime aggregates afterwards; use {@link mergeAnimeRecords}
|
||||
* otherwise.
|
||||
*/
|
||||
export function mergeAnimeRecordsInTransaction(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
sourceAnimeIds: number[],
|
||||
): AnimeMergeSummary {
|
||||
const summary = emptyMergeSummary(targetAnimeId);
|
||||
if (!animeExists(db, targetAnimeId)) {
|
||||
return summary;
|
||||
}
|
||||
|
||||
const updatedAt = toDbTimestamp(nowMs());
|
||||
const sourceVideosStmt = db.prepare(
|
||||
'SELECT video_id AS videoId FROM imm_videos WHERE anime_id = ?',
|
||||
);
|
||||
const moveVideosStmt = db.prepare(
|
||||
'UPDATE imm_videos SET anime_id = ?, LAST_UPDATE_DATE = ? WHERE anime_id = ?',
|
||||
);
|
||||
// Repointed per video rather than by anime_id: lines recorded before the
|
||||
// async title parse assigns the link are stored with a NULL anime_id, and
|
||||
// matching on the source id would strand them unattributed.
|
||||
const moveLinesStmt = db.prepare(
|
||||
'UPDATE imm_subtitle_lines SET anime_id = ?, LAST_UPDATE_DATE = ? WHERE video_id = ?',
|
||||
);
|
||||
const dropLifetimeStmt = db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?');
|
||||
const sourceAliasesStmt = db.prepare(
|
||||
'SELECT normalized_title_key AS normalizedTitleKey FROM imm_anime_title_aliases WHERE anime_id = ?',
|
||||
);
|
||||
const upsertAliasStmt = db.prepare(
|
||||
`INSERT INTO imm_anime_title_aliases(normalized_title_key, anime_id, CREATED_DATE, LAST_UPDATE_DATE)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(normalized_title_key) DO UPDATE SET
|
||||
anime_id = excluded.anime_id,
|
||||
LAST_UPDATE_DATE = excluded.LAST_UPDATE_DATE`,
|
||||
);
|
||||
const dropSourceAliasesStmt = db.prepare(
|
||||
'DELETE FROM imm_anime_title_aliases WHERE anime_id = ?',
|
||||
);
|
||||
const dropAnimeStmt = db.prepare('DELETE FROM imm_anime WHERE anime_id = ?');
|
||||
|
||||
for (const sourceAnimeId of new Set(sourceAnimeIds)) {
|
||||
if (sourceAnimeId === targetAnimeId || !animeExists(db, sourceAnimeId)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const sourceMetadata = readAnimeMetadata(db, sourceAnimeId);
|
||||
const sourceAliases = sourceAliasesStmt.all(sourceAnimeId) as Array<{
|
||||
normalizedTitleKey: string;
|
||||
}>;
|
||||
const sourceVideoIds = (sourceVideosStmt.all(sourceAnimeId) as Array<{ videoId: number }>).map(
|
||||
(row) => row.videoId,
|
||||
);
|
||||
const moved = moveVideosStmt.run(targetAnimeId, updatedAt, sourceAnimeId) as {
|
||||
changes: number;
|
||||
};
|
||||
for (const videoId of sourceVideoIds) {
|
||||
moveLinesStmt.run(targetAnimeId, updatedAt, videoId);
|
||||
}
|
||||
dropSourceAliasesStmt.run(sourceAnimeId);
|
||||
for (const alias of [
|
||||
...(sourceMetadata ? [sourceMetadata.normalized_title_key] : []),
|
||||
...sourceAliases.map((row) => row.normalizedTitleKey),
|
||||
]) {
|
||||
upsertAliasStmt.run(alias, targetAnimeId, updatedAt, updatedAt);
|
||||
}
|
||||
dropLifetimeStmt.run(sourceAnimeId);
|
||||
dropAnimeStmt.run(sourceAnimeId);
|
||||
absorbAnimeMetadata(db, targetAnimeId, sourceMetadata, updatedAt);
|
||||
|
||||
summary.mergedAnimeIds.push(sourceAnimeId);
|
||||
summary.movedVideos += moved.changes;
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
export function mergeAnimeRecords(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
sourceAnimeIds: number[],
|
||||
): AnimeMergeSummary {
|
||||
return runInTransaction(db, () => {
|
||||
const summary = mergeAnimeRecordsInTransaction(db, targetAnimeId, sourceAnimeIds);
|
||||
if (summary.mergedAnimeIds.length > 0) {
|
||||
recomputeLifetimeAnimeAggregatesInTransaction(db);
|
||||
}
|
||||
return summary;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Move a single episode to another library entry, pruning the previous owner
|
||||
* when it is left with nothing.
|
||||
*/
|
||||
export function moveVideoToAnime(
|
||||
db: DatabaseSync,
|
||||
videoId: number,
|
||||
targetAnimeId: number,
|
||||
): VideoMoveSummary {
|
||||
return runInTransaction(db, () => {
|
||||
const videoRow = db
|
||||
.prepare('SELECT anime_id AS animeId FROM imm_videos WHERE video_id = ?')
|
||||
.get(videoId) as { animeId: number | null } | null;
|
||||
if (!videoRow || !animeExists(db, targetAnimeId)) {
|
||||
throw new Error(UNKNOWN_MOVE_TARGET_MESSAGE);
|
||||
}
|
||||
|
||||
const previousAnimeId = videoRow.animeId;
|
||||
if (previousAnimeId === targetAnimeId) {
|
||||
db.prepare(
|
||||
'UPDATE imm_videos SET anime_assignment_locked = 1, LAST_UPDATE_DATE = ? WHERE video_id = ?',
|
||||
).run(toDbTimestamp(nowMs()), videoId);
|
||||
return { targetAnimeId, previousAnimeId, removedPreviousAnime: false };
|
||||
}
|
||||
|
||||
const updatedAt = toDbTimestamp(nowMs());
|
||||
db.prepare(
|
||||
`UPDATE imm_videos
|
||||
SET anime_id = ?, anime_assignment_locked = 1, LAST_UPDATE_DATE = ?
|
||||
WHERE video_id = ?`,
|
||||
).run(targetAnimeId, updatedAt, videoId);
|
||||
db.prepare(
|
||||
'UPDATE imm_subtitle_lines SET anime_id = ?, LAST_UPDATE_DATE = ? WHERE video_id = ?',
|
||||
).run(targetAnimeId, updatedAt, videoId);
|
||||
|
||||
let removedPreviousAnime = false;
|
||||
if (previousAnimeId !== null && !hasAnimeReferences(db, previousAnimeId)) {
|
||||
// The emptied entry's metadata is deliberately dropped rather than
|
||||
// absorbed. A move says "this episode belongs elsewhere", not "these are
|
||||
// the same show", and the entry being emptied is usually a mis-parse
|
||||
// whose AniList link would be wrong for the target.
|
||||
db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?').run(previousAnimeId);
|
||||
db.prepare('DELETE FROM imm_anime WHERE anime_id = ?').run(previousAnimeId);
|
||||
removedPreviousAnime = true;
|
||||
}
|
||||
|
||||
recomputeLifetimeAnimeAggregatesInTransaction(db);
|
||||
return { targetAnimeId, previousAnimeId, removedPreviousAnime };
|
||||
});
|
||||
}
|
||||
@@ -1,16 +1,4 @@
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import {
|
||||
animeSeasonsAreMergeCompatible,
|
||||
getParsedSeasonsForAnime,
|
||||
mergeAnimeRecordsInTransaction,
|
||||
} from './anime-merge';
|
||||
import {
|
||||
hasExactStoredTitleMatch,
|
||||
hasDismissedAnimeMergeRecommendation,
|
||||
recordAnimeMergeRecommendation,
|
||||
shouldRecommendAnilistConflict,
|
||||
type AnimeConflictRecommendationOptions,
|
||||
} from './anime-merge-recommendations';
|
||||
import { getOrCreateAnimeRecord } from './storage';
|
||||
import { toDbTimestamp } from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
@@ -20,33 +8,9 @@ export interface AnimeSeasonRepairSummary {
|
||||
repaired: number;
|
||||
movedVideos: number;
|
||||
deletedAnimeRows: number;
|
||||
/**
|
||||
* Entry that owns the videos afterwards when two rows were folded together,
|
||||
* so callers can keep pointing at a row that still exists.
|
||||
*/
|
||||
survivingAnimeId: number | null;
|
||||
/** True when an ambiguous AniList collision was saved for user review. */
|
||||
mergeRecommended: boolean;
|
||||
/** True when automatic metadata must not assign the colliding AniList id. */
|
||||
anilistAssignmentBlocked: boolean;
|
||||
affectedAnimeIds: number[];
|
||||
}
|
||||
|
||||
export interface AnimeAnilistConflictOptions extends AnimeConflictRecommendationOptions {
|
||||
/**
|
||||
* Which row keeps its identity when two entries claim the same AniList id.
|
||||
* `existing` (the default) keeps the row that already held the id, so
|
||||
* automatic cover-art resolution does not rename a card under the user;
|
||||
* `target` keeps the row the user is acting on.
|
||||
*/
|
||||
survivor?: 'target' | 'existing';
|
||||
}
|
||||
|
||||
export {
|
||||
dismissAnimeMergeRecommendation,
|
||||
getAnimeMergeRecommendations,
|
||||
type AnimeMergeRecommendation,
|
||||
} from './anime-merge-recommendations';
|
||||
|
||||
interface AnimeRow {
|
||||
anime_id: number;
|
||||
anilist_id: number | null;
|
||||
@@ -61,7 +25,6 @@ interface ParsedVideoRow {
|
||||
video_id: number;
|
||||
parsed_title: string | null;
|
||||
parsed_season: number | null;
|
||||
anime_assignment_locked: number;
|
||||
}
|
||||
|
||||
interface RedistributeOptions {
|
||||
@@ -76,9 +39,7 @@ function emptySummary(scanned = 0): AnimeSeasonRepairSummary {
|
||||
repaired: 0,
|
||||
movedVideos: 0,
|
||||
deletedAnimeRows: 0,
|
||||
survivingAnimeId: null,
|
||||
mergeRecommended: false,
|
||||
anilistAssignmentBlocked: false,
|
||||
affectedAnimeIds: [],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -90,9 +51,7 @@ function mergeSummary(
|
||||
target.repaired += source.repaired;
|
||||
target.movedVideos += source.movedVideos;
|
||||
target.deletedAnimeRows += source.deletedAnimeRows;
|
||||
target.survivingAnimeId = source.survivingAnimeId ?? target.survivingAnimeId;
|
||||
target.mergeRecommended ||= source.mergeRecommended;
|
||||
target.anilistAssignmentBlocked ||= source.anilistAssignmentBlocked;
|
||||
target.affectedAnimeIds = [...new Set([...target.affectedAnimeIds, ...source.affectedAnimeIds])];
|
||||
return target;
|
||||
}
|
||||
|
||||
@@ -138,7 +97,7 @@ function getParsedVideos(db: DatabaseSync, animeId: number): ParsedVideoRow[] {
|
||||
return db
|
||||
.prepare(
|
||||
`
|
||||
SELECT video_id, parsed_title, parsed_season, anime_assignment_locked
|
||||
SELECT video_id, parsed_title, parsed_season
|
||||
FROM imm_videos
|
||||
WHERE anime_id = ?
|
||||
ORDER BY video_id ASC
|
||||
@@ -228,13 +187,11 @@ function redistributeAnimeRowByParsedSeasonsInTransaction(
|
||||
|
||||
const videos = getParsedVideos(db, animeId);
|
||||
const summary = emptySummary(1);
|
||||
summary.affectedAnimeIds.push(animeId);
|
||||
const updatedAt = toDbTimestamp(nowMs());
|
||||
const targetBySeason = new Map<number, number>();
|
||||
|
||||
for (const video of videos) {
|
||||
if (video.anime_assignment_locked === 1) {
|
||||
continue;
|
||||
}
|
||||
const parsedTitle = video.parsed_title?.trim();
|
||||
const season = normalizeSeason(video.parsed_season);
|
||||
if (!parsedTitle || season === null) {
|
||||
@@ -280,6 +237,9 @@ function redistributeAnimeRowByParsedSeasonsInTransaction(
|
||||
|
||||
if (videoUpdate.changes > 0 || lineUpdate.changes > 0) {
|
||||
summary.movedVideos += 1;
|
||||
if (!summary.affectedAnimeIds.includes(targetAnimeId)) {
|
||||
summary.affectedAnimeIds.push(targetAnimeId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -348,18 +308,10 @@ export function repairLegacySeasonlessAnimeRows(db: DatabaseSync): AnimeSeasonRe
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Two library entries cannot both hold the same AniList id
|
||||
* (`imm_anime.anilist_id` is UNIQUE). Fold an automatic collision only when
|
||||
* exact title evidence and compatible parsed seasons make it safe. Persist a
|
||||
* review recommendation for compatible weak matches. Fall back to legacy
|
||||
* season redistribution when the conflicting row spans several seasons.
|
||||
*/
|
||||
export function resolveAnimeAnilistConflict(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
anilistId: number,
|
||||
options: AnimeAnilistConflictOptions = {},
|
||||
): AnimeSeasonRepairSummary {
|
||||
const conflict = db
|
||||
.prepare(
|
||||
@@ -376,100 +328,10 @@ export function resolveAnimeAnilistConflict(
|
||||
return emptySummary();
|
||||
}
|
||||
|
||||
return runInTransaction(db, () => {
|
||||
const targetRow = getAnimeRow(db, targetAnimeId);
|
||||
if (
|
||||
options.survivor !== 'target' &&
|
||||
targetRow?.anilist_id != null &&
|
||||
targetRow.anilist_id !== anilistId
|
||||
) {
|
||||
// An automatic lookup disagreeing with an existing explicit link is a
|
||||
// mis-resolution, not evidence that either row should move or merge. The
|
||||
// colliding id must not be assigned either: another row owns it and
|
||||
// imm_anime.anilist_id is UNIQUE.
|
||||
const summary = emptySummary(1);
|
||||
summary.anilistAssignmentBlocked = true;
|
||||
return summary;
|
||||
}
|
||||
const isManual = options.survivor === 'target' || options.matchConfidence === 'manual';
|
||||
if (!isManual && hasDismissedAnimeMergeRecommendation(db, targetAnimeId, conflict.animeId)) {
|
||||
const summary = emptySummary(1);
|
||||
summary.anilistAssignmentBlocked = true;
|
||||
return summary;
|
||||
}
|
||||
const targetSeasons = getParsedSeasonsForAnime(db, targetAnimeId);
|
||||
const conflictSeasons = getParsedSeasonsForAnime(db, conflict.animeId);
|
||||
if (
|
||||
!isManual &&
|
||||
targetSeasons.size === 1 &&
|
||||
conflictSeasons.size === 1 &&
|
||||
[...targetSeasons][0] !== [...conflictSeasons][0]
|
||||
) {
|
||||
const summary = emptySummary(1);
|
||||
summary.anilistAssignmentBlocked = true;
|
||||
return summary;
|
||||
}
|
||||
if (canMergeAnilistConflict(db, targetAnimeId, conflict.animeId, anilistId, options)) {
|
||||
const survivingAnimeId = options.survivor === 'target' ? targetAnimeId : conflict.animeId;
|
||||
const absorbedAnimeId = survivingAnimeId === targetAnimeId ? conflict.animeId : targetAnimeId;
|
||||
const merge = mergeAnimeRecordsInTransaction(db, survivingAnimeId, [absorbedAnimeId]);
|
||||
const summary = emptySummary(1);
|
||||
summary.movedVideos = merge.movedVideos;
|
||||
summary.deletedAnimeRows = merge.mergedAnimeIds.length;
|
||||
if (merge.mergedAnimeIds.length > 0) {
|
||||
summary.repaired = 1;
|
||||
// Only reported once a row really absorbed the other, so callers never
|
||||
// follow this to an anime id that was never written.
|
||||
summary.survivingAnimeId = survivingAnimeId;
|
||||
}
|
||||
// Lifetime summaries are rebuilt by the caller off this summary, the same
|
||||
// as the redistribution path below.
|
||||
return summary;
|
||||
}
|
||||
|
||||
if (shouldRecommendAnilistConflict(db, targetAnimeId, conflict.animeId, options)) {
|
||||
recordAnimeMergeRecommendation(db, targetAnimeId, conflict.animeId, anilistId);
|
||||
const summary = emptySummary(1);
|
||||
summary.mergeRecommended = true;
|
||||
return summary;
|
||||
}
|
||||
|
||||
return redistributeAnimeRowByParsedSeasonsInTransaction(db, conflict.animeId, {
|
||||
return runInTransaction(db, () =>
|
||||
redistributeAnimeRowByParsedSeasonsInTransaction(db, conflict.animeId, {
|
||||
transferAnilistToAnimeId: targetAnimeId,
|
||||
overwriteTargetAnilist: true,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function canMergeAnilistConflict(
|
||||
db: DatabaseSync,
|
||||
targetAnimeId: number,
|
||||
conflictAnimeId: number,
|
||||
anilistId: number,
|
||||
options: AnimeAnilistConflictOptions,
|
||||
): boolean {
|
||||
const targetRow = getAnimeRow(db, targetAnimeId);
|
||||
if (!targetRow) {
|
||||
// Nothing to merge with a row that no longer exists (a stale id from the
|
||||
// caller); fall through to the redistribution path.
|
||||
return false;
|
||||
}
|
||||
if (options.survivor !== 'target') {
|
||||
// The target is the row about to disappear here, so an existing link of its
|
||||
// own means this is a mis-resolution rather than a duplicate: leave it be.
|
||||
if (targetRow.anilist_id != null && targetRow.anilist_id !== anilistId) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (
|
||||
options.matchConfidence === 'weak' ||
|
||||
(options.matchConfidence === undefined &&
|
||||
!hasExactStoredTitleMatch(db, targetAnimeId, conflictAnimeId))
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return animeSeasonsAreMergeCompatible(
|
||||
getParsedSeasonsForAnime(db, targetAnimeId),
|
||||
getParsedSeasonsForAnime(db, conflictAnimeId),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -32,7 +32,7 @@ function createFakeWorker() {
|
||||
|
||||
type FakeWorker = ReturnType<typeof createFakeWorker>['worker'];
|
||||
|
||||
test('a delete batch rebuilds lifetime summaries once', () => {
|
||||
test('a delete batch never runs a full lifetime rebuild', () => {
|
||||
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-delete-batch-test-'));
|
||||
const dbPath = path.join(tempDir, 'immersion.sqlite');
|
||||
let db = new Database(dbPath);
|
||||
@@ -87,8 +87,8 @@ test('a delete batch rebuilds lifetime summaries once', () => {
|
||||
assert.equal(deletedVideo, undefined);
|
||||
assert.equal(
|
||||
audit.total,
|
||||
2,
|
||||
'one rebuild performs exactly its reset and final global summary writes',
|
||||
0,
|
||||
'delete maintenance subtracts incrementally instead of rewriting last_rebuilt_ms',
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
@@ -100,6 +100,14 @@ test('a delete batch rebuilds lifetime summaries once', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('delete worker module resolves in the current layout', () => {
|
||||
// If this resolves to null, every delete silently runs on the serving thread
|
||||
// and blocks the stats API for the whole maintenance run.
|
||||
const workerPath = resolveDeleteMaintenanceWorkerPath();
|
||||
assert.ok(workerPath, 'delete-maintenance worker module must resolve');
|
||||
assert.ok(workerPath.endsWith(__filename.endsWith('.ts') ? '.ts' : '.js'));
|
||||
});
|
||||
|
||||
test(
|
||||
'compiled delete worker removes data through its separate database connection',
|
||||
{ skip: resolveDeleteMaintenanceWorkerPath() === null },
|
||||
@@ -178,19 +186,72 @@ test('worker runtime terminates a worker after successful settlement', async ()
|
||||
assert.equal(terminationState.calls, 1);
|
||||
});
|
||||
|
||||
test('worker runtime terminates a worker after failed settlement', async () => {
|
||||
test('worker runtime falls back to the current thread when the worker crashes', async () => {
|
||||
const { worker, listeners, terminationState } = createFakeWorker();
|
||||
const fallbackTasks: unknown[] = [];
|
||||
const warnings: string[] = [];
|
||||
const runtime = new DeleteMaintenanceWorkerRuntime({
|
||||
resolveWorkerPath: () => '/tmp/delete-worker.js',
|
||||
createWorker: async () => worker,
|
||||
executeFallback: (dbPath, task) => {
|
||||
fallbackTasks.push({ dbPath, task });
|
||||
},
|
||||
warn: (message) => {
|
||||
warnings.push(message);
|
||||
},
|
||||
});
|
||||
|
||||
const result = runtime.run('/tmp/test.sqlite', { kind: 'session', sessionId: 1 });
|
||||
await new Promise<void>((resolve) => setTimeout(resolve, 0));
|
||||
listeners.get('error')?.(new Error('worker failed') as never);
|
||||
|
||||
await assert.rejects(result, /worker failed/);
|
||||
await result;
|
||||
assert.equal(terminationState.calls, 1);
|
||||
assert.deepEqual(fallbackTasks, [
|
||||
{ dbPath: '/tmp/test.sqlite', task: { kind: 'session', sessionId: 1 } },
|
||||
]);
|
||||
assert.equal(warnings.length, 1);
|
||||
});
|
||||
|
||||
test('worker runtime falls back when a worker exits cleanly without a response', async () => {
|
||||
const { worker, listeners, terminationState } = createFakeWorker();
|
||||
const fallbackTasks: unknown[] = [];
|
||||
const runtime = new DeleteMaintenanceWorkerRuntime({
|
||||
resolveWorkerPath: () => '/tmp/delete-worker.js',
|
||||
createWorker: async () => worker,
|
||||
executeFallback: (dbPath, task) => {
|
||||
fallbackTasks.push({ dbPath, task });
|
||||
},
|
||||
});
|
||||
const task = { kind: 'session' as const, sessionId: 1 };
|
||||
|
||||
const result = runtime.run('/tmp/test.sqlite', task);
|
||||
await new Promise<void>((resolve) => setTimeout(resolve, 0));
|
||||
listeners.get('exit')?.(0 as never);
|
||||
|
||||
await result;
|
||||
assert.equal(terminationState.calls, 1);
|
||||
assert.deepEqual(fallbackTasks, [{ dbPath: '/tmp/test.sqlite', task }]);
|
||||
});
|
||||
|
||||
test('worker runtime surfaces a task failure without rerunning it', async () => {
|
||||
const { worker, listeners, terminationState } = createFakeWorker();
|
||||
const fallbackTasks: unknown[] = [];
|
||||
const runtime = new DeleteMaintenanceWorkerRuntime({
|
||||
resolveWorkerPath: () => '/tmp/delete-worker.js',
|
||||
createWorker: async () => worker,
|
||||
executeFallback: () => {
|
||||
fallbackTasks.push('ran');
|
||||
},
|
||||
});
|
||||
|
||||
const result = runtime.run('/tmp/test.sqlite', { kind: 'session', sessionId: 1 });
|
||||
await new Promise<void>((resolve) => setTimeout(resolve, 0));
|
||||
listeners.get('message')?.({ ok: false, error: 'constraint violated' } as never);
|
||||
|
||||
await assert.rejects(result, /constraint violated/);
|
||||
assert.equal(terminationState.calls, 1);
|
||||
assert.equal(fallbackTasks.length, 0);
|
||||
});
|
||||
|
||||
test('worker runtime terminates a worker created after shutdown begins', async () => {
|
||||
|
||||
@@ -31,7 +31,13 @@ interface DeleteMaintenanceWorkerRuntimeOptions {
|
||||
}
|
||||
|
||||
export function resolveDeleteMaintenanceWorkerPath(): string | null {
|
||||
const workerPath = path.join(__dirname, 'delete-maintenance-worker-thread.js');
|
||||
// When the process runs TypeScript directly (Bun from source), the emitted
|
||||
// .js sibling doesn't exist — spawn the .ts module instead, which such
|
||||
// runtimes transpile for workers too. Compiled layouts keep using the .js.
|
||||
const fileName = __filename.endsWith('.ts')
|
||||
? 'delete-maintenance-worker-thread.ts'
|
||||
: 'delete-maintenance-worker-thread.js';
|
||||
const workerPath = path.join(__dirname, fileName);
|
||||
return fs.existsSync(workerPath) ? workerPath : null;
|
||||
}
|
||||
|
||||
@@ -76,38 +82,61 @@ export class DeleteMaintenanceWorkerRuntime {
|
||||
throw new Error('Delete maintenance worker is shut down');
|
||||
}
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
type WorkerOutcome =
|
||||
| { kind: 'ok' }
|
||||
| { kind: 'task-error'; detail: string }
|
||||
| { kind: 'worker-failure'; error: Error };
|
||||
|
||||
const outcome = await new Promise<WorkerOutcome>((resolve) => {
|
||||
let settled = false;
|
||||
this.activeWorkers.add(worker);
|
||||
|
||||
const settle = (error?: Error) => {
|
||||
const settle = (result: WorkerOutcome) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
this.activeWorkers.delete(worker);
|
||||
if (error) reject(error);
|
||||
else resolve();
|
||||
resolve(result);
|
||||
void worker.terminate();
|
||||
};
|
||||
|
||||
worker.once('message', (message: DeleteMaintenanceWorkerResponse) => {
|
||||
if (message.ok === true) {
|
||||
settle();
|
||||
settle({ kind: 'ok' });
|
||||
return;
|
||||
}
|
||||
const detail = typeof message.error === 'string' ? message.error : 'unknown worker error';
|
||||
settle(new Error(`Delete maintenance failed: ${detail}`));
|
||||
settle({ kind: 'task-error', detail });
|
||||
});
|
||||
worker.once('error', (error) => settle(error));
|
||||
worker.once('error', (error) => settle({ kind: 'worker-failure', error }));
|
||||
worker.once('exit', (code) => {
|
||||
settle(
|
||||
new Error(
|
||||
settle({
|
||||
kind: 'worker-failure',
|
||||
error: new Error(
|
||||
code === 0
|
||||
? 'Delete maintenance worker exited without a response'
|
||||
: `Delete maintenance worker exited with code ${code}`,
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
if (outcome.kind === 'ok') return;
|
||||
// The maintenance itself failed inside the worker — rerunning it on this
|
||||
// thread would hit the same error, so surface it instead.
|
||||
if (outcome.kind === 'task-error') {
|
||||
throw new Error(`Delete maintenance failed: ${outcome.detail}`);
|
||||
}
|
||||
if (this.destroyed) {
|
||||
throw new Error('Delete maintenance worker is shut down');
|
||||
}
|
||||
// The worker died without reporting a result (failed to load, crashed).
|
||||
// Its transaction rolled back with its connection, and a rerun re-plans
|
||||
// against the current rows, so falling back on this thread is safe.
|
||||
(this.options.warn ?? logger.warn)(
|
||||
'Delete maintenance worker failed; running maintenance on the current thread',
|
||||
outcome.error,
|
||||
);
|
||||
(this.options.executeFallback ?? executeDeleteMaintenanceTask)(dbPath, task);
|
||||
}
|
||||
|
||||
destroy(): void {
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { Database } from './sqlite';
|
||||
import { applyPragmas } from './storage';
|
||||
import { deleteAnime, deleteSession, deleteSessions, deleteVideo } from './query-maintenance';
|
||||
import {
|
||||
deleteMaintenanceBatch,
|
||||
type DeleteMaintenanceOperation,
|
||||
@@ -12,35 +11,11 @@ export type DeleteMaintenanceTask =
|
||||
| DeleteMaintenanceOperation
|
||||
| { kind: 'batch'; tasks: DeleteMaintenanceOperation[] };
|
||||
|
||||
function executeDeleteMaintenanceOperation(
|
||||
db: InstanceType<typeof Database>,
|
||||
task: DeleteMaintenanceOperation,
|
||||
): void {
|
||||
switch (task.kind) {
|
||||
case 'session':
|
||||
deleteSession(db, task.sessionId);
|
||||
return;
|
||||
case 'sessions':
|
||||
deleteSessions(db, task.sessionIds);
|
||||
return;
|
||||
case 'video':
|
||||
deleteVideo(db, task.videoId);
|
||||
return;
|
||||
case 'anime':
|
||||
deleteAnime(db, task.animeId);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
export function executeDeleteMaintenanceTask(dbPath: string, task: DeleteMaintenanceTask): void {
|
||||
const db = new Database(dbPath);
|
||||
try {
|
||||
applyPragmas(db);
|
||||
if (task.kind === 'batch') {
|
||||
deleteMaintenanceBatch(db, task.tasks);
|
||||
return;
|
||||
}
|
||||
executeDeleteMaintenanceOperation(db, task);
|
||||
deleteMaintenanceBatch(db, task.kind === 'batch' ? task.tasks : [task]);
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,423 @@
|
||||
/*
|
||||
* Retroactive removal of animation-burst subtitle lines from the stats database.
|
||||
*
|
||||
* Before the live ingest gate existed, a karaoke OP recorded one line -- and one count
|
||||
* for every word in it -- per animation frame, which is enough to put an OP lyric at the
|
||||
* top of "Top Repeated Words" for good. This module finds those runs in what is already
|
||||
* stored and takes them back down to one line.
|
||||
*
|
||||
* Only timing is available here: the stored text has been stripped of ASS markup, so the
|
||||
* authoring evidence the file-level parser uses (`\t`, `\move`, karaoke timing, a
|
||||
* changing override signature) is long gone. What is left is a run of identical,
|
||||
* contiguous, short-lived lines inside a single session.
|
||||
*
|
||||
* The run has to be as long as the timing-only rule in `subtitle-cue-dedup` demands, but
|
||||
* its short frames may be as long as the animation-frame bound rather than the much
|
||||
* tighter timing-only one. A qualifying run may end with one longer hold, which is a
|
||||
* common karaoke shape. Five or more repeats of the same text, each ending where the next
|
||||
* begins, is already conclusive on its own -- no dialogue does that -- and the tighter
|
||||
* bound would walk straight past the heavier typesetting that motivated this, where
|
||||
* frames sit nearer a quarter of a second. Both bounds are options, so a cautious run can
|
||||
* ask for more, and a dry run always reports before anything is removed.
|
||||
*
|
||||
* Scope: subtitle lines, their word/kanji occurrences, and the `imm_words`/`imm_kanji`
|
||||
* aggregates those occurrences feed. Session telemetry (`lines_seen`, `tokens_seen`) and
|
||||
* the rollups derived from it are left alone; they are cumulative samples taken at record
|
||||
* time, and for sessions whose raw rows have since been pruned they cannot be recomputed.
|
||||
*/
|
||||
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import {
|
||||
ANIMATION_FRAME_MAX_SECONDS,
|
||||
DUPLICATE_CUE_GAP_TOLERANCE_SECONDS,
|
||||
MIN_STREAM_RESIDUE_FRAMES,
|
||||
MIN_TIMING_ONLY_FRAMES,
|
||||
TIMING_ONLY_FRAME_MAX_SECONDS,
|
||||
} from '../subtitle-burst-constants';
|
||||
import {
|
||||
applyLexicalRemovals,
|
||||
makePlaceholders,
|
||||
planLexicalRemovalsForLines,
|
||||
toDbTimestamp,
|
||||
} from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
|
||||
const MS_PER_DAY = 86_400_000;
|
||||
/** SQLite caps bound parameters per statement; stay well under it. */
|
||||
const ID_BATCH_SIZE = 400;
|
||||
const DEFAULT_SAMPLE_LIMIT = 20;
|
||||
|
||||
export interface DuplicateSubtitleLineCleanupOptions {
|
||||
/** Only consider lines recorded within this many days. Null or omitted = all history. */
|
||||
lookbackDays?: number | null;
|
||||
/** Measure without writing. */
|
||||
dryRun?: boolean;
|
||||
/** Identical contiguous lines needed before a run counts as an animation. */
|
||||
minRunLength?: number;
|
||||
/** Longest a single event may last and still look like an animation frame. */
|
||||
maxFrameSeconds?: number;
|
||||
/** How many of the largest runs to describe in the summary. */
|
||||
sampleLimit?: number;
|
||||
}
|
||||
|
||||
export interface DuplicateSubtitleLineBurst {
|
||||
sessionId: number;
|
||||
videoId: number;
|
||||
text: string;
|
||||
/** Kept line, extended to cover the whole run. */
|
||||
keptLineId: number;
|
||||
removedLineIds: number[];
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
export interface DuplicateSubtitleLineSample {
|
||||
videoId: number;
|
||||
videoTitle: string | null;
|
||||
text: string;
|
||||
frames: number;
|
||||
removedLines: number;
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
export interface DuplicateSubtitleLineCleanupSummary {
|
||||
dryRun: boolean;
|
||||
lookbackDays: number | null;
|
||||
scannedLines: number;
|
||||
burstGroups: number;
|
||||
removedLines: number;
|
||||
removedWordOccurrences: number;
|
||||
removedKanjiOccurrences: number;
|
||||
samples: DuplicateSubtitleLineSample[];
|
||||
}
|
||||
|
||||
export interface StoredSubtitleLineRow {
|
||||
lineId: number;
|
||||
sessionId: number;
|
||||
videoId: number;
|
||||
text: string;
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
interface ResolvedBounds {
|
||||
lookbackDays: number | null;
|
||||
minRunLength: number;
|
||||
maxFrameMs: number;
|
||||
/** Shorter runs qualify only when every event sits under this much stricter bound. */
|
||||
residueMinRunLength: number;
|
||||
strictFrameMs: number;
|
||||
gapToleranceMs: number;
|
||||
sampleLimit: number;
|
||||
}
|
||||
|
||||
function resolveBounds(options: DuplicateSubtitleLineCleanupOptions): ResolvedBounds {
|
||||
const lookbackDays =
|
||||
typeof options.lookbackDays === 'number' && Number.isFinite(options.lookbackDays)
|
||||
? Math.max(1, Math.floor(options.lookbackDays))
|
||||
: null;
|
||||
const minRunLength =
|
||||
typeof options.minRunLength === 'number' && Number.isFinite(options.minRunLength)
|
||||
? Math.max(2, Math.floor(options.minRunLength))
|
||||
: MIN_TIMING_ONLY_FRAMES;
|
||||
const maxFrameSeconds =
|
||||
typeof options.maxFrameSeconds === 'number' &&
|
||||
Number.isFinite(options.maxFrameSeconds) &&
|
||||
options.maxFrameSeconds > 0
|
||||
? options.maxFrameSeconds
|
||||
: ANIMATION_FRAME_MAX_SECONDS;
|
||||
const sampleLimit =
|
||||
typeof options.sampleLimit === 'number' && options.sampleLimit >= 0
|
||||
? Math.floor(options.sampleLimit)
|
||||
: DEFAULT_SAMPLE_LIMIT;
|
||||
return {
|
||||
lookbackDays,
|
||||
minRunLength,
|
||||
maxFrameMs: Math.round(maxFrameSeconds * 1000),
|
||||
residueMinRunLength: Math.max(MIN_STREAM_RESIDUE_FRAMES, minRunLength - 1),
|
||||
strictFrameMs: Math.round(TIMING_ONLY_FRAME_MAX_SECONDS * 1000),
|
||||
gapToleranceMs: Math.round(DUPLICATE_CUE_GAP_TOLERANCE_SECONDS * 1000),
|
||||
sampleLimit,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* `CREATED_DATE` holds epoch milliseconds on rows this app wrote, but older and synced
|
||||
* rows can carry seconds, so normalize before comparing against the cutoff.
|
||||
*/
|
||||
const CREATED_MS_SQL = `
|
||||
CASE
|
||||
WHEN sl.CREATED_DATE < 10000000000 THEN sl.CREATED_DATE * 1000
|
||||
ELSE sl.CREATED_DATE
|
||||
END`;
|
||||
|
||||
function readCandidateLines(db: DatabaseSync, bounds: ResolvedBounds): StoredSubtitleLineRow[] {
|
||||
const scope =
|
||||
bounds.lookbackDays === null
|
||||
? ''
|
||||
: `AND sl.CREATED_DATE IS NOT NULL AND ${CREATED_MS_SQL} >= ?`;
|
||||
const params = bounds.lookbackDays === null ? [] : [nowMs() - bounds.lookbackDays * MS_PER_DAY];
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT
|
||||
sl.line_id AS lineId,
|
||||
sl.session_id AS sessionId,
|
||||
sl.video_id AS videoId,
|
||||
sl.text AS text,
|
||||
sl.segment_start_ms AS startMs,
|
||||
sl.segment_end_ms AS endMs
|
||||
FROM imm_subtitle_lines sl
|
||||
WHERE sl.segment_start_ms IS NOT NULL
|
||||
AND sl.segment_end_ms IS NOT NULL
|
||||
${scope}
|
||||
ORDER BY sl.session_id, sl.video_id, sl.segment_start_ms, sl.line_id`,
|
||||
)
|
||||
.all(...params) as StoredSubtitleLineRow[];
|
||||
}
|
||||
|
||||
function isBurst(run: StoredSubtitleLineRow[], bounds: ResolvedBounds): boolean {
|
||||
const isShortFrame = (row: StoredSubtitleLineRow): boolean =>
|
||||
row.endMs - row.startMs <= bounds.maxFrameMs;
|
||||
// The residue the live gate leaves behind: it records the first frames of a burst
|
||||
// before the run is long enough to recognise, so one frame fewer than the timing-only
|
||||
// minimum, every one under the strict timing-only bound. No dialogue holds identical
|
||||
// sub-tenth-second lines back to back that many times.
|
||||
if (
|
||||
run.length >= bounds.residueMinRunLength &&
|
||||
run.every((row) => row.endMs - row.startMs <= bounds.strictFrameMs)
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
if (run.length < bounds.minRunLength) {
|
||||
return false;
|
||||
}
|
||||
if (run.every(isShortFrame)) {
|
||||
return true;
|
||||
}
|
||||
// Karaoke commonly finishes its short animation frames with one long hold. Only the
|
||||
// final event may exceed the frame bound, and the short frames before it must already
|
||||
// meet the minimum run length on their own.
|
||||
return (
|
||||
run.length - 1 >= bounds.minRunLength &&
|
||||
run.slice(0, -1).every(isShortFrame) &&
|
||||
!isShortFrame(run[run.length - 1]!)
|
||||
);
|
||||
}
|
||||
|
||||
function toBurst(run: StoredSubtitleLineRow[]): DuplicateSubtitleLineBurst {
|
||||
const [first] = run;
|
||||
return {
|
||||
sessionId: first!.sessionId,
|
||||
videoId: first!.videoId,
|
||||
text: first!.text,
|
||||
keptLineId: first!.lineId,
|
||||
removedLineIds: run.slice(1).map((row) => row.lineId),
|
||||
startMs: first!.startMs,
|
||||
endMs: run.reduce((latest, row) => Math.max(latest, row.endMs), first!.endMs),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Group stored lines into animation runs.
|
||||
*
|
||||
* Rows are bucketed per (session, video, text) before chaining, the way the file-level
|
||||
* dedup buckets cues: dual-line karaoke interleaves two texts frame by frame, and
|
||||
* chaining across the interleave would break every run at length one.
|
||||
*
|
||||
* Runs never cross a session, which is what keeps a rewatch intact: the same episode
|
||||
* watched twice stores the same line twice, and those two belong to different sessions.
|
||||
*/
|
||||
export function findDuplicateSubtitleLineBursts(
|
||||
rows: readonly StoredSubtitleLineRow[],
|
||||
options: DuplicateSubtitleLineCleanupOptions = {},
|
||||
): DuplicateSubtitleLineBurst[] {
|
||||
const bounds = resolveBounds(options);
|
||||
|
||||
// Insertion order preserves the query's startMs ordering within each bucket.
|
||||
const rowsByKey = new Map<string, StoredSubtitleLineRow[]>();
|
||||
for (const row of rows) {
|
||||
const key = `${row.sessionId}|${row.videoId}|${row.text}`;
|
||||
const bucket = rowsByKey.get(key);
|
||||
if (bucket) {
|
||||
bucket.push(row);
|
||||
} else {
|
||||
rowsByKey.set(key, [row]);
|
||||
}
|
||||
}
|
||||
|
||||
const bursts: DuplicateSubtitleLineBurst[] = [];
|
||||
for (const bucket of rowsByKey.values()) {
|
||||
if (bucket.length < 2) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let run: StoredSubtitleLineRow[] = [];
|
||||
let chainEndMs = 0;
|
||||
|
||||
const closeRun = (): void => {
|
||||
if (run.length > 1 && isBurst(run, bounds)) {
|
||||
bursts.push(toBurst(run));
|
||||
}
|
||||
run = [];
|
||||
};
|
||||
|
||||
for (const row of bucket) {
|
||||
if (run.length > 0 && row.startMs <= chainEndMs + bounds.gapToleranceMs) {
|
||||
run.push(row);
|
||||
chainEndMs = Math.max(chainEndMs, row.endMs);
|
||||
continue;
|
||||
}
|
||||
closeRun();
|
||||
run = [row];
|
||||
chainEndMs = row.endMs;
|
||||
}
|
||||
closeRun();
|
||||
}
|
||||
|
||||
return bursts;
|
||||
}
|
||||
|
||||
function chunk<T>(values: T[], size: number): T[][] {
|
||||
const chunks: T[][] = [];
|
||||
for (let i = 0; i < values.length; i += size) {
|
||||
chunks.push(values.slice(i, i + size));
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
|
||||
function buildSamples(
|
||||
db: DatabaseSync,
|
||||
bursts: DuplicateSubtitleLineBurst[],
|
||||
sampleLimit: number,
|
||||
): DuplicateSubtitleLineSample[] {
|
||||
if (sampleLimit === 0 || bursts.length === 0) {
|
||||
return [];
|
||||
}
|
||||
const largest = [...bursts]
|
||||
.sort((a, b) => b.removedLineIds.length - a.removedLineIds.length)
|
||||
.slice(0, sampleLimit);
|
||||
const videoIds = [...new Set(largest.map((burst) => burst.videoId))];
|
||||
const titles = new Map<number, string>();
|
||||
for (const batch of chunk(videoIds, ID_BATCH_SIZE)) {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`SELECT video_id AS videoId, canonical_title AS title
|
||||
FROM imm_videos
|
||||
WHERE video_id IN (${makePlaceholders(batch)})`,
|
||||
)
|
||||
.all(...batch) as Array<{ videoId: number; title: string | null }>;
|
||||
for (const row of rows) {
|
||||
if (row.title) titles.set(row.videoId, row.title);
|
||||
}
|
||||
}
|
||||
|
||||
return largest.map((burst) => ({
|
||||
videoId: burst.videoId,
|
||||
videoTitle: titles.get(burst.videoId) ?? null,
|
||||
text: burst.text,
|
||||
frames: burst.removedLineIds.length + 1,
|
||||
removedLines: burst.removedLineIds.length,
|
||||
startMs: burst.startMs,
|
||||
endMs: burst.endMs,
|
||||
}));
|
||||
}
|
||||
|
||||
function sumRemovedOccurrences(
|
||||
db: DatabaseSync,
|
||||
table: 'imm_word_line_occurrences' | 'imm_kanji_line_occurrences',
|
||||
lineIds: number[],
|
||||
): number {
|
||||
let total = 0;
|
||||
for (const batch of chunk(lineIds, ID_BATCH_SIZE)) {
|
||||
const row = db
|
||||
.prepare(
|
||||
`SELECT COALESCE(SUM(occurrence_count), 0) AS total
|
||||
FROM ${table}
|
||||
WHERE line_id IN (${makePlaceholders(batch)})`,
|
||||
)
|
||||
.get(...batch) as { total: number } | null;
|
||||
total += row?.total ?? 0;
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
function applyBursts(db: DatabaseSync, bursts: DuplicateSubtitleLineBurst[]): void {
|
||||
const removedLineIds = bursts.flatMap((burst) => burst.removedLineIds);
|
||||
const currentMs = toDbTimestamp(nowMs());
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
for (const batch of chunk(removedLineIds, ID_BATCH_SIZE)) {
|
||||
const placeholders = makePlaceholders(batch);
|
||||
// Measured before the delete, applied after it: `applyLexicalRemovals` checks the
|
||||
// surviving occurrences to decide whether a zeroed count really means the word is
|
||||
// gone, so the rows it inspects have to be the post-delete ones.
|
||||
const plan = planLexicalRemovalsForLines(db, batch);
|
||||
db.prepare(`DELETE FROM imm_word_line_occurrences WHERE line_id IN (${placeholders})`).run(
|
||||
...batch,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_kanji_line_occurrences WHERE line_id IN (${placeholders})`).run(
|
||||
...batch,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_subtitle_lines WHERE line_id IN (${placeholders})`).run(...batch);
|
||||
applyLexicalRemovals(db, plan);
|
||||
}
|
||||
|
||||
const extendStmt = db.prepare(
|
||||
`UPDATE imm_subtitle_lines
|
||||
SET segment_end_ms = ?, LAST_UPDATE_DATE = ?
|
||||
WHERE line_id = ? AND (segment_end_ms IS NULL OR segment_end_ms < ?)`,
|
||||
);
|
||||
for (const burst of bursts) {
|
||||
extendStmt.run(burst.endMs, currentMs, burst.keptLineId, burst.endMs);
|
||||
}
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
try {
|
||||
db.exec('ROLLBACK');
|
||||
} catch {
|
||||
// Surface the transaction failure, not the rollback's.
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse stored animation bursts down to one line each.
|
||||
*
|
||||
* A dry run measures exactly what an apply would remove, using the same scan, so the
|
||||
* numbers shown in a confirmation prompt are the numbers that will happen.
|
||||
*/
|
||||
export function cleanupDuplicateSubtitleLines(
|
||||
db: DatabaseSync,
|
||||
options: DuplicateSubtitleLineCleanupOptions = {},
|
||||
): DuplicateSubtitleLineCleanupSummary {
|
||||
const bounds = resolveBounds(options);
|
||||
const dryRun = options.dryRun === true;
|
||||
const rows = readCandidateLines(db, bounds);
|
||||
const bursts = findDuplicateSubtitleLineBursts(rows, options);
|
||||
const removedLineIds = bursts.flatMap((burst) => burst.removedLineIds);
|
||||
|
||||
const summary: DuplicateSubtitleLineCleanupSummary = {
|
||||
dryRun,
|
||||
lookbackDays: bounds.lookbackDays,
|
||||
scannedLines: rows.length,
|
||||
burstGroups: bursts.length,
|
||||
removedLines: removedLineIds.length,
|
||||
removedWordOccurrences: sumRemovedOccurrences(db, 'imm_word_line_occurrences', removedLineIds),
|
||||
removedKanjiOccurrences: sumRemovedOccurrences(
|
||||
db,
|
||||
'imm_kanji_line_occurrences',
|
||||
removedLineIds,
|
||||
),
|
||||
samples: buildSamples(db, bursts, bounds.sampleLimit),
|
||||
};
|
||||
|
||||
if (dryRun || removedLineIds.length === 0) {
|
||||
return summary;
|
||||
}
|
||||
|
||||
applyBursts(db, bursts);
|
||||
return summary;
|
||||
}
|
||||
@@ -8,8 +8,6 @@ import type { JellyfinLinkRepairSummary } from './types';
|
||||
type LegacyJellyfinVideoRow = {
|
||||
video_id: number;
|
||||
video_key: string;
|
||||
anime_id: number | null;
|
||||
anime_assignment_locked: number;
|
||||
source_url: string | null;
|
||||
canonical_title: string;
|
||||
};
|
||||
@@ -17,7 +15,6 @@ type LegacyJellyfinVideoRow = {
|
||||
type JellyfinTargetVideoRow = {
|
||||
video_id: number;
|
||||
anime_id: number | null;
|
||||
anime_assignment_locked: number;
|
||||
canonical_title: string;
|
||||
parsed_basename: string | null;
|
||||
parsed_title: string | null;
|
||||
@@ -261,13 +258,7 @@ export function repairJellyfinStreamVideoLinks(db: DatabaseSync): JellyfinLinkRe
|
||||
const candidates = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
video_id,
|
||||
video_key,
|
||||
anime_id,
|
||||
anime_assignment_locked,
|
||||
source_url,
|
||||
canonical_title
|
||||
SELECT video_id, video_key, source_url, canonical_title
|
||||
FROM imm_videos
|
||||
WHERE source_type = 2
|
||||
AND (
|
||||
@@ -319,7 +310,6 @@ export function repairJellyfinStreamVideoLinks(db: DatabaseSync): JellyfinLinkRe
|
||||
SELECT
|
||||
video_id,
|
||||
anime_id,
|
||||
anime_assignment_locked,
|
||||
canonical_title,
|
||||
parsed_basename,
|
||||
parsed_title,
|
||||
@@ -367,17 +357,12 @@ export function repairJellyfinStreamVideoLinks(db: DatabaseSync): JellyfinLinkRe
|
||||
continue;
|
||||
}
|
||||
|
||||
const assignmentAnimeId =
|
||||
candidate.anime_assignment_locked === 1 ? candidate.anime_id : target.anime_id;
|
||||
const assignmentLocked =
|
||||
candidate.anime_assignment_locked === 1 || target.anime_assignment_locked === 1 ? 1 : 0;
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_videos
|
||||
SET
|
||||
video_key = ?,
|
||||
anime_id = ?,
|
||||
anime_assignment_locked = ?,
|
||||
canonical_title = ?,
|
||||
source_url = ?,
|
||||
parsed_basename = ?,
|
||||
@@ -392,8 +377,7 @@ export function repairJellyfinStreamVideoLinks(db: DatabaseSync): JellyfinLinkRe
|
||||
`,
|
||||
).run(
|
||||
sanitizedVideoKey,
|
||||
assignmentAnimeId,
|
||||
assignmentLocked,
|
||||
target.anime_id,
|
||||
target.canonical_title,
|
||||
statsUrl,
|
||||
target.parsed_basename,
|
||||
@@ -406,14 +390,14 @@ export function repairJellyfinStreamVideoLinks(db: DatabaseSync): JellyfinLinkRe
|
||||
currentTimestamp,
|
||||
candidate.video_id,
|
||||
);
|
||||
if (assignmentAnimeId !== null) {
|
||||
if (target.anime_id !== null) {
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_subtitle_lines
|
||||
SET anime_id = ?, LAST_UPDATE_DATE = ?
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
).run(assignmentAnimeId, currentTimestamp, candidate.video_id);
|
||||
).run(target.anime_id, currentTimestamp, candidate.video_id);
|
||||
}
|
||||
summary.repaired += 1;
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { finalizeSessionRecord } from './session';
|
||||
import { nowMs } from './time';
|
||||
import { toDbTimestamp } from './query-shared';
|
||||
import { forEachIdChunk, makePlaceholders, toDbTimestamp } from './query-shared';
|
||||
import type { LifetimeRebuildSummary, SessionState } from './types';
|
||||
|
||||
interface TelemetryRow {
|
||||
@@ -21,10 +21,8 @@ interface AnimeRow {
|
||||
}
|
||||
|
||||
function asPositiveNumber(value: number | null, fallback: number): number {
|
||||
if (value === null || !Number.isFinite(value)) {
|
||||
return fallback;
|
||||
}
|
||||
return Math.max(0, Math.floor(value));
|
||||
const resolved = value !== null && Number.isFinite(value) ? value : fallback;
|
||||
return Number.isFinite(resolved) ? Math.floor(Math.max(resolved, 0)) : 0;
|
||||
}
|
||||
|
||||
interface ExistenceRow {
|
||||
@@ -68,10 +66,10 @@ const RETAINED_SESSION_METRICS_CTE = `
|
||||
v.anime_id,
|
||||
s.started_at_ms,
|
||||
s.ended_at_ms,
|
||||
MAX(COALESCE(t.active_watched_ms, s.active_watched_ms, 0), 0) AS active_ms,
|
||||
MAX(COALESCE(t.cards_mined, s.cards_mined, 0), 0) AS cards_mined,
|
||||
MAX(COALESCE(t.lines_seen, s.lines_seen, 0), 0) AS lines_seen,
|
||||
MAX(COALESCE(t.tokens_seen, s.tokens_seen, 0), 0) AS tokens_seen,
|
||||
CAST(MAX(COALESCE(t.active_watched_ms, s.active_watched_ms, 0), 0) AS INTEGER) AS active_ms,
|
||||
CAST(MAX(COALESCE(t.cards_mined, s.cards_mined, 0), 0) AS INTEGER) AS cards_mined,
|
||||
CAST(MAX(COALESCE(t.lines_seen, s.lines_seen, 0), 0) AS INTEGER) AS lines_seen,
|
||||
CAST(MAX(COALESCE(t.tokens_seen, s.tokens_seen, 0), 0) AS INTEGER) AS tokens_seen,
|
||||
CASE WHEN v.watched > 0 THEN 1 ELSE 0 END AS completed
|
||||
FROM imm_sessions s
|
||||
JOIN imm_videos v
|
||||
@@ -599,18 +597,10 @@ export function applySessionLifetimeSummary(
|
||||
.get(video.anime_id) as AnimeRow | null | undefined) ?? null)
|
||||
: null;
|
||||
|
||||
const activeMs = telemetry
|
||||
? asPositiveNumber(telemetry.active_watched_ms, session.activeWatchedMs)
|
||||
: session.activeWatchedMs;
|
||||
const cardsMined = telemetry
|
||||
? asPositiveNumber(telemetry.cards_mined, session.cardsMined)
|
||||
: session.cardsMined;
|
||||
const linesSeen = telemetry
|
||||
? asPositiveNumber(telemetry.lines_seen, session.linesSeen)
|
||||
: session.linesSeen;
|
||||
const tokensSeen = telemetry
|
||||
? asPositiveNumber(telemetry.tokens_seen, session.tokensSeen)
|
||||
: session.tokensSeen;
|
||||
const activeMs = asPositiveNumber(telemetry?.active_watched_ms ?? null, session.activeWatchedMs);
|
||||
const cardsMined = asPositiveNumber(telemetry?.cards_mined ?? null, session.cardsMined);
|
||||
const linesSeen = asPositiveNumber(telemetry?.lines_seen ?? null, session.linesSeen);
|
||||
const tokensSeen = asPositiveNumber(telemetry?.tokens_seen ?? null, session.tokensSeen);
|
||||
const watched = video?.watched ?? 0;
|
||||
const isFirstSessionForVideoRun =
|
||||
mediaLifetime === null &&
|
||||
@@ -708,26 +698,284 @@ export function rebuildLifetimeSummariesInTransaction(
|
||||
return rebuildLifetimeSummariesInternal(db, rebuiltAtMs);
|
||||
}
|
||||
|
||||
const LOCAL_DAY_EXPR = `CAST(
|
||||
julianday(CAST(started_at_ms AS REAL) / 1000, 'unixepoch', 'localtime') - 2440587.5
|
||||
AS INTEGER
|
||||
)`;
|
||||
|
||||
interface LifetimeMediaRemoval {
|
||||
videoId: number;
|
||||
sessions: number;
|
||||
activeMs: number;
|
||||
cards: number;
|
||||
linesSeen: number;
|
||||
tokensSeen: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-derive every per-anime lifetime row from the per-video summaries after
|
||||
* episodes changed owners (merge, move, season repair).
|
||||
* What a pending delete removes from the lifetime summary tables.
|
||||
*
|
||||
* Deliberately NOT a full rebuild: {@link rebuildLifetimeSummariesInTransaction}
|
||||
* recomputes from raw sessions, which are pruned after the retention window, so
|
||||
* it silently truncates lifetime history. `imm_lifetime_media` is keyed by
|
||||
* video and survives repointing, so aggregating it preserves all-time totals;
|
||||
* `imm_lifetime_global` only needs `anime_completed` refreshed because moving
|
||||
* attribution between entries cannot change the global counters.
|
||||
*
|
||||
* Assumes the caller holds a write transaction; use
|
||||
* {@link recomputeLifetimeAnimeAggregates} otherwise.
|
||||
* Lifetime totals intentionally outlive raw-session retention, so they can
|
||||
* never be rebuilt from `imm_sessions` without collapsing history to the
|
||||
* retention window. Deletes instead subtract exactly what the deleted rows
|
||||
* contributed: this plan is measured before the rows are removed and applied
|
||||
* after.
|
||||
*/
|
||||
export function recomputeLifetimeAnimeAggregatesInTransaction(db: DatabaseSync): void {
|
||||
const updatedAt = toDbTimestamp(nowMs());
|
||||
db.exec('DELETE FROM imm_lifetime_anime');
|
||||
db.prepare(
|
||||
export interface LifetimeRemovalPlan {
|
||||
/** Per surviving video: summed metrics of its deleted, lifetime-applied sessions. */
|
||||
mediaRemovals: LifetimeMediaRemoval[];
|
||||
/** Surviving anime whose lifetime rows must be recomputed from their media rows. */
|
||||
affectedAnimeIds: number[];
|
||||
/** Local-day keys touched by deleted applied sessions, for active_days upkeep. */
|
||||
affectedDayKeys: number[];
|
||||
}
|
||||
|
||||
export function planLifetimeRemovals(
|
||||
db: DatabaseSync,
|
||||
args: {
|
||||
/** Every session being deleted, including ones expanded from video/anime deletes. */
|
||||
deletedSessionIds: number[];
|
||||
/** Deleted sessions whose video survives the delete. */
|
||||
sessionIdsOnSurvivingVideos: number[];
|
||||
deletedVideoIds: number[];
|
||||
deletedAnimeIds: number[];
|
||||
},
|
||||
): LifetimeRemovalPlan {
|
||||
const mediaRemovalsByVideo = new Map<number, LifetimeMediaRemoval>();
|
||||
forEachIdChunk(args.sessionIdsOnSurvivingVideos, (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
s.video_id AS videoId,
|
||||
COUNT(*) AS sessions,
|
||||
COALESCE(SUM(CAST(MAX(COALESCE(t.active_watched_ms, s.active_watched_ms, 0), 0) AS INTEGER)), 0) AS activeMs,
|
||||
COALESCE(SUM(CAST(MAX(COALESCE(t.cards_mined, s.cards_mined, 0), 0) AS INTEGER)), 0) AS cards,
|
||||
COALESCE(SUM(CAST(MAX(COALESCE(t.lines_seen, s.lines_seen, 0), 0) AS INTEGER)), 0) AS linesSeen,
|
||||
COALESCE(SUM(CAST(MAX(COALESCE(t.tokens_seen, s.tokens_seen, 0), 0) AS INTEGER)), 0) AS tokensSeen
|
||||
FROM imm_sessions s
|
||||
JOIN imm_lifetime_applied_sessions a ON a.session_id = s.session_id
|
||||
LEFT JOIN imm_session_telemetry t
|
||||
ON t.telemetry_id = (
|
||||
SELECT telemetry_id
|
||||
FROM imm_session_telemetry
|
||||
WHERE session_id = s.session_id
|
||||
ORDER BY sample_ms DESC, telemetry_id DESC
|
||||
LIMIT 1
|
||||
)
|
||||
WHERE s.session_id IN (${makePlaceholders(chunk)})
|
||||
GROUP BY s.video_id
|
||||
`,
|
||||
)
|
||||
.all(...chunk) as LifetimeMediaRemoval[];
|
||||
for (const row of rows) {
|
||||
const existing = mediaRemovalsByVideo.get(row.videoId);
|
||||
if (!existing) {
|
||||
mediaRemovalsByVideo.set(row.videoId, { ...row });
|
||||
continue;
|
||||
}
|
||||
existing.sessions += row.sessions;
|
||||
existing.activeMs += row.activeMs;
|
||||
existing.cards += row.cards;
|
||||
existing.linesSeen += row.linesSeen;
|
||||
existing.tokensSeen += row.tokensSeen;
|
||||
}
|
||||
});
|
||||
|
||||
const deletedAnimeIds = new Set(args.deletedAnimeIds);
|
||||
const affectedAnimeIds = new Set<number>();
|
||||
forEachIdChunk(args.deletedVideoIds, (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`SELECT DISTINCT anime_id AS animeId FROM imm_videos
|
||||
WHERE video_id IN (${makePlaceholders(chunk)}) AND anime_id IS NOT NULL`,
|
||||
)
|
||||
.all(...chunk) as Array<{ animeId: number }>;
|
||||
for (const row of rows) affectedAnimeIds.add(row.animeId);
|
||||
});
|
||||
forEachIdChunk(args.sessionIdsOnSurvivingVideos, (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`SELECT DISTINCT v.anime_id AS animeId
|
||||
FROM imm_sessions s
|
||||
JOIN imm_videos v ON v.video_id = s.video_id
|
||||
WHERE s.session_id IN (${makePlaceholders(chunk)}) AND v.anime_id IS NOT NULL`,
|
||||
)
|
||||
.all(...chunk) as Array<{ animeId: number }>;
|
||||
for (const row of rows) affectedAnimeIds.add(row.animeId);
|
||||
});
|
||||
for (const animeId of deletedAnimeIds) affectedAnimeIds.delete(animeId);
|
||||
|
||||
const affectedDayKeys = new Set<number>();
|
||||
forEachIdChunk(args.deletedSessionIds, (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`SELECT DISTINCT ${LOCAL_DAY_EXPR} AS dayKey
|
||||
FROM imm_sessions s
|
||||
JOIN imm_lifetime_applied_sessions a ON a.session_id = s.session_id
|
||||
WHERE s.session_id IN (${makePlaceholders(chunk)})`,
|
||||
)
|
||||
.all(...chunk) as Array<{ dayKey: number }>;
|
||||
for (const row of rows) affectedDayKeys.add(row.dayKey);
|
||||
});
|
||||
|
||||
return {
|
||||
mediaRemovals: [...mediaRemovalsByVideo.values()],
|
||||
affectedAnimeIds: [...affectedAnimeIds],
|
||||
affectedDayKeys: [...affectedDayKeys],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a removal plan after the underlying rows are gone.
|
||||
*
|
||||
* Media rows are adjusted by subtraction (pruned-session history stays intact),
|
||||
* affected anime rows are recomputed from their surviving media rows, and the
|
||||
* global row is re-derived from the media/anime tables. `active_days` is the
|
||||
* one metric that can't be derived, so a touched day is only decremented when
|
||||
* no ended session remains on that local day; days whose sessions were pruned
|
||||
* by retention keep their count because pruning never subtracts.
|
||||
*/
|
||||
export function applyLifetimeRemovals(db: DatabaseSync, plan: LifetimeRemovalPlan): void {
|
||||
const updatedAtMs = toDbTimestamp(nowMs());
|
||||
|
||||
const subtractMediaStmt = db.prepare(
|
||||
`
|
||||
INSERT INTO imm_lifetime_anime (
|
||||
UPDATE imm_lifetime_media SET
|
||||
total_sessions = MAX(total_sessions - ?, 0),
|
||||
total_active_ms = MAX(total_active_ms - ?, 0),
|
||||
total_cards = MAX(total_cards - ?, 0),
|
||||
total_lines_seen = MAX(total_lines_seen - ?, 0),
|
||||
total_tokens_seen = MAX(total_tokens_seen - ?, 0),
|
||||
LAST_UPDATE_DATE = ?
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
);
|
||||
const dropEmptyMediaStmt = db.prepare(
|
||||
'DELETE FROM imm_lifetime_media WHERE video_id = ? AND total_sessions <= 0',
|
||||
);
|
||||
const remainingSessionRangeStmt = db.prepare(
|
||||
`
|
||||
SELECT
|
||||
MIN(CAST(started_at_ms AS REAL)) AS minStartedMs,
|
||||
MAX(CAST(ended_at_ms AS REAL)) AS maxEndedMs
|
||||
FROM imm_sessions
|
||||
WHERE video_id = ? AND ended_at_ms IS NOT NULL
|
||||
`,
|
||||
);
|
||||
const storedMediaRangeStmt = db.prepare(
|
||||
`
|
||||
SELECT CAST(first_watched_ms AS REAL) AS firstWatchedMs
|
||||
FROM imm_lifetime_media
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
);
|
||||
const refreshMediaRangeStmt = db.prepare(
|
||||
`
|
||||
UPDATE imm_lifetime_media SET
|
||||
first_watched_ms = ?,
|
||||
last_watched_ms = ?
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
);
|
||||
|
||||
for (const removal of plan.mediaRemovals) {
|
||||
subtractMediaStmt.run(
|
||||
removal.sessions,
|
||||
removal.activeMs,
|
||||
removal.cards,
|
||||
removal.linesSeen,
|
||||
removal.tokensSeen,
|
||||
updatedAtMs,
|
||||
removal.videoId,
|
||||
);
|
||||
dropEmptyMediaStmt.run(removal.videoId);
|
||||
const stored = storedMediaRangeStmt.get(removal.videoId) as {
|
||||
firstWatchedMs: number | null;
|
||||
} | null;
|
||||
if (!stored) continue;
|
||||
const range = remainingSessionRangeStmt.get(removal.videoId) as {
|
||||
minStartedMs: number | null;
|
||||
maxEndedMs: number | null;
|
||||
} | null;
|
||||
// Retained sessions are always newer than pruned ones, so the surviving
|
||||
// range is authoritative for last_watched while first_watched can only
|
||||
// keep or extend the stored (possibly pruned-history) minimum. When no
|
||||
// session survives, the stored values are all that's left.
|
||||
if (range && range.minStartedMs !== null && range.maxEndedMs !== null) {
|
||||
const firstWatchedMs =
|
||||
stored.firstWatchedMs === null
|
||||
? range.minStartedMs
|
||||
: Math.min(stored.firstWatchedMs, range.minStartedMs);
|
||||
refreshMediaRangeStmt.run(
|
||||
toDbTimestamp(firstWatchedMs),
|
||||
toDbTimestamp(range.maxEndedMs),
|
||||
removal.videoId,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
recomputeLifetimeAnimeFromMedia(db, plan.affectedAnimeIds, updatedAtMs);
|
||||
|
||||
// One pass over the sessions rather than a probe per affected day: the local
|
||||
// day is a computed expression with no index, so each probe would be a table
|
||||
// scan — and the miss case (the day we need to count) is the full-scan one.
|
||||
let removedDays = 0;
|
||||
if (plan.affectedDayKeys.length > 0) {
|
||||
const survivingDayKeys = new Set(
|
||||
(
|
||||
db
|
||||
.prepare(
|
||||
`SELECT DISTINCT ${LOCAL_DAY_EXPR} AS dayKey
|
||||
FROM imm_sessions
|
||||
WHERE ended_at_ms IS NOT NULL`,
|
||||
)
|
||||
.all() as Array<{ dayKey: number }>
|
||||
).map((row) => row.dayKey),
|
||||
);
|
||||
for (const dayKey of plan.affectedDayKeys) {
|
||||
if (!survivingDayKeys.has(dayKey)) removedDays += 1;
|
||||
}
|
||||
}
|
||||
|
||||
recomputeLifetimeGlobalFromSummaries(db, { removedActiveDays: removedDays, updatedAtMs });
|
||||
}
|
||||
|
||||
/**
|
||||
* Recompute lifetime anime rows exactly from their surviving media rows.
|
||||
*
|
||||
* Media rows are the durable per-video ledger (they outlive session pruning and
|
||||
* follow a video when it moves between anime), so this is the correct refresh
|
||||
* after merges, moves, and deletes. Anime with no media rows left are dropped.
|
||||
*/
|
||||
export function recomputeLifetimeAnimeFromMedia(
|
||||
db: DatabaseSync,
|
||||
animeIds: number[],
|
||||
updatedAtMs = toDbTimestamp(nowMs()),
|
||||
): void {
|
||||
if (animeIds.length === 0) return;
|
||||
|
||||
const animeSummaryStmt = db.prepare(
|
||||
`
|
||||
SELECT
|
||||
COUNT(*) AS episodeRows,
|
||||
COALESCE(SUM(m.total_sessions), 0) AS totalSessions,
|
||||
COALESCE(SUM(m.total_active_ms), 0) AS totalActiveMs,
|
||||
COALESCE(SUM(m.total_cards), 0) AS totalCards,
|
||||
COALESCE(SUM(m.total_lines_seen), 0) AS totalLinesSeen,
|
||||
COALESCE(SUM(m.total_tokens_seen), 0) AS totalTokensSeen,
|
||||
COALESCE(SUM(CASE WHEN m.completed > 0 THEN 1 ELSE 0 END), 0) AS episodesCompleted,
|
||||
MIN(CAST(m.first_watched_ms AS REAL)) AS firstWatchedMs,
|
||||
MAX(CAST(m.last_watched_ms AS REAL)) AS lastWatchedMs
|
||||
FROM imm_lifetime_media m
|
||||
JOIN imm_videos v ON v.video_id = m.video_id
|
||||
WHERE v.anime_id = ?
|
||||
`,
|
||||
);
|
||||
const dropAnimeStmt = db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?');
|
||||
const upsertAnimeStmt = db.prepare(
|
||||
`
|
||||
INSERT INTO imm_lifetime_anime(
|
||||
anime_id,
|
||||
total_sessions,
|
||||
total_active_ms,
|
||||
@@ -741,50 +989,171 @@ export function recomputeLifetimeAnimeAggregatesInTransaction(db: DatabaseSync):
|
||||
CREATED_DATE,
|
||||
LAST_UPDATE_DATE
|
||||
)
|
||||
SELECT
|
||||
v.anime_id,
|
||||
COALESCE(SUM(m.total_sessions), 0),
|
||||
COALESCE(SUM(m.total_active_ms), 0),
|
||||
COALESCE(SUM(m.total_cards), 0),
|
||||
COALESCE(SUM(m.total_lines_seen), 0),
|
||||
COALESCE(SUM(m.total_tokens_seen), 0),
|
||||
COUNT(*),
|
||||
COUNT(CASE WHEN m.completed > 0 THEN 1 END),
|
||||
MIN(m.first_watched_ms),
|
||||
MAX(m.last_watched_ms),
|
||||
?,
|
||||
?
|
||||
FROM imm_lifetime_media m
|
||||
JOIN imm_videos v ON v.video_id = m.video_id
|
||||
WHERE v.anime_id IS NOT NULL
|
||||
GROUP BY v.anime_id
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(anime_id) DO UPDATE SET
|
||||
total_sessions = excluded.total_sessions,
|
||||
total_active_ms = excluded.total_active_ms,
|
||||
total_cards = excluded.total_cards,
|
||||
total_lines_seen = excluded.total_lines_seen,
|
||||
total_tokens_seen = excluded.total_tokens_seen,
|
||||
episodes_started = excluded.episodes_started,
|
||||
episodes_completed = excluded.episodes_completed,
|
||||
first_watched_ms = excluded.first_watched_ms,
|
||||
last_watched_ms = excluded.last_watched_ms,
|
||||
LAST_UPDATE_DATE = excluded.LAST_UPDATE_DATE
|
||||
`,
|
||||
).run(updatedAt, updatedAt);
|
||||
);
|
||||
|
||||
for (const animeId of animeIds) {
|
||||
const summary = animeSummaryStmt.get(animeId) as {
|
||||
episodeRows: number;
|
||||
totalSessions: number;
|
||||
totalActiveMs: number;
|
||||
totalCards: number;
|
||||
totalLinesSeen: number;
|
||||
totalTokensSeen: number;
|
||||
episodesCompleted: number;
|
||||
firstWatchedMs: number | null;
|
||||
lastWatchedMs: number | null;
|
||||
};
|
||||
if (Number(summary.episodeRows) === 0) {
|
||||
dropAnimeStmt.run(animeId);
|
||||
continue;
|
||||
}
|
||||
upsertAnimeStmt.run(
|
||||
animeId,
|
||||
summary.totalSessions,
|
||||
summary.totalActiveMs,
|
||||
summary.totalCards,
|
||||
summary.totalLinesSeen,
|
||||
summary.totalTokensSeen,
|
||||
summary.episodeRows,
|
||||
summary.episodesCompleted,
|
||||
summary.firstWatchedMs === null ? null : toDbTimestamp(summary.firstWatchedMs),
|
||||
summary.lastWatchedMs === null ? null : toDbTimestamp(summary.lastWatchedMs),
|
||||
updatedAtMs,
|
||||
updatedAtMs,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-derive the global lifetime row from the media/anime summary tables.
|
||||
*
|
||||
* Every global metric except active_days is a pure aggregate of those tables;
|
||||
* active_days can't be derived, so callers pass how many day slots their change
|
||||
* removed (0 for moves/merges, which never touch sessions).
|
||||
*/
|
||||
export function recomputeLifetimeGlobalFromSummaries(
|
||||
db: DatabaseSync,
|
||||
options: { removedActiveDays?: number; updatedAtMs?: string } = {},
|
||||
): void {
|
||||
const updatedAtMs = options.updatedAtMs ?? toDbTimestamp(nowMs());
|
||||
const mediaTotals = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
COUNT(*) AS episodesStarted,
|
||||
COALESCE(SUM(total_sessions), 0) AS totalSessions,
|
||||
COALESCE(SUM(total_active_ms), 0) AS totalActiveMs,
|
||||
COALESCE(SUM(total_cards), 0) AS totalCards,
|
||||
COALESCE(SUM(CASE WHEN completed > 0 THEN 1 ELSE 0 END), 0) AS episodesCompleted
|
||||
FROM imm_lifetime_media
|
||||
`,
|
||||
)
|
||||
.get() as {
|
||||
episodesStarted: number;
|
||||
totalSessions: number;
|
||||
totalActiveMs: number;
|
||||
totalCards: number;
|
||||
episodesCompleted: number;
|
||||
};
|
||||
const animeCompletedRow = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT COUNT(*) AS animeCompleted
|
||||
FROM imm_lifetime_anime la
|
||||
JOIN imm_anime a ON a.anime_id = la.anime_id
|
||||
WHERE a.episodes_total IS NOT NULL
|
||||
AND a.episodes_total > 0
|
||||
AND la.episodes_completed >= a.episodes_total
|
||||
`,
|
||||
)
|
||||
.get() as { animeCompleted: number };
|
||||
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_lifetime_global
|
||||
SET
|
||||
anime_completed = (
|
||||
SELECT COUNT(*)
|
||||
FROM imm_lifetime_anime la
|
||||
JOIN imm_anime a ON a.anime_id = la.anime_id
|
||||
WHERE a.episodes_total IS NOT NULL
|
||||
AND a.episodes_total > 0
|
||||
AND la.episodes_completed >= a.episodes_total
|
||||
),
|
||||
UPDATE imm_lifetime_global SET
|
||||
total_sessions = ?,
|
||||
total_active_ms = ?,
|
||||
total_cards = ?,
|
||||
episodes_started = ?,
|
||||
episodes_completed = ?,
|
||||
anime_completed = ?,
|
||||
active_days = MAX(active_days - ?, 0),
|
||||
LAST_UPDATE_DATE = ?
|
||||
WHERE global_id = 1
|
||||
`,
|
||||
).run(updatedAt);
|
||||
).run(
|
||||
mediaTotals.totalSessions,
|
||||
mediaTotals.totalActiveMs,
|
||||
mediaTotals.totalCards,
|
||||
mediaTotals.episodesStarted,
|
||||
mediaTotals.episodesCompleted,
|
||||
animeCompletedRow.animeCompleted,
|
||||
options.removedActiveDays ?? 0,
|
||||
updatedAtMs,
|
||||
);
|
||||
}
|
||||
|
||||
export function recomputeLifetimeAnimeAggregates(db: DatabaseSync): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
export interface LifetimeRepairSummary {
|
||||
recomputedAnime: number;
|
||||
repairedAtMs: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Non-destructive lifetime repair: recompute every anime row and the global row
|
||||
* from the per-video media ledger.
|
||||
*
|
||||
* Unlike {@link rebuildLifetimeSummaries}, this never resets the tables from
|
||||
* retained sessions, so lifetime history older than the session retention
|
||||
* window survives. The one exception is a database whose lifetime tables were
|
||||
* never populated — there is no ledger to repair from, so it bootstraps with
|
||||
* the full rebuild instead.
|
||||
*/
|
||||
export function repairLifetimeSummariesFromMedia(db: DatabaseSync): LifetimeRepairSummary {
|
||||
const repairedAtMs = nowMs();
|
||||
let transactionStarted = false;
|
||||
try {
|
||||
recomputeLifetimeAnimeAggregatesInTransaction(db);
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
transactionStarted = true;
|
||||
if (shouldBackfillLifetimeSummaries(db)) {
|
||||
const rebuilt = rebuildLifetimeSummariesInTransaction(db, repairedAtMs);
|
||||
const animeRow = db
|
||||
.prepare('SELECT COUNT(*) AS count FROM imm_lifetime_anime')
|
||||
.get() as ExistenceRow;
|
||||
db.exec('COMMIT');
|
||||
return { recomputedAnime: Number(animeRow.count), repairedAtMs: rebuilt.rebuiltAtMs };
|
||||
}
|
||||
|
||||
const animeIds = new Set<number>();
|
||||
for (const row of db
|
||||
.prepare('SELECT DISTINCT anime_id AS animeId FROM imm_videos WHERE anime_id IS NOT NULL')
|
||||
.all() as Array<{ animeId: number }>) {
|
||||
animeIds.add(row.animeId);
|
||||
}
|
||||
for (const row of db
|
||||
.prepare('SELECT anime_id AS animeId FROM imm_lifetime_anime')
|
||||
.all() as Array<{ animeId: number }>) {
|
||||
animeIds.add(row.animeId);
|
||||
}
|
||||
const updatedAtMs = toDbTimestamp(repairedAtMs);
|
||||
recomputeLifetimeAnimeFromMedia(db, [...animeIds], updatedAtMs);
|
||||
recomputeLifetimeGlobalFromSummaries(db, { updatedAtMs });
|
||||
db.exec('COMMIT');
|
||||
return { recomputedAnime: animeIds.size, repairedAtMs };
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
if (transactionStarted) db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { rebuildLifetimeSummariesInTransaction } from './lifetime';
|
||||
import { applyLifetimeRemovals, planLifetimeRemovals } from './lifetime';
|
||||
import { getRollupGroupsForSessions, refreshRollupsForGroupsInTransaction } from './maintenance';
|
||||
import {
|
||||
applyLexicalRemovals,
|
||||
@@ -8,9 +8,10 @@ import {
|
||||
forEachIdChunk,
|
||||
makePlaceholders,
|
||||
planLexicalRemovalsForSessions,
|
||||
SQLITE_ID_CHUNK_SIZE,
|
||||
planLexicalRemovalsForVideos,
|
||||
type LexicalRemovalPlan,
|
||||
} from './query-shared';
|
||||
import type { RollupGroup } from './maintenance';
|
||||
|
||||
export type DeleteMaintenanceOperation =
|
||||
| { kind: 'session'; sessionId: number }
|
||||
@@ -59,40 +60,72 @@ function selectIds(
|
||||
return ids;
|
||||
}
|
||||
|
||||
function planLexicalRemovalsInChunks(db: DatabaseSync, sessionIds: number[]): LexicalRemovalPlan {
|
||||
const combined: LexicalRemovalPlan = { words: [], kanji: [] };
|
||||
const merge = (target: LexicalRemovalPlan['words'], source: LexicalRemovalPlan['words']) => {
|
||||
const byId = new Map(target.map((entry) => [entry.id, entry]));
|
||||
for (const entry of source) {
|
||||
const existing = byId.get(entry.id);
|
||||
if (!existing) {
|
||||
const added = { ...entry };
|
||||
target.push(added);
|
||||
byId.set(entry.id, added);
|
||||
continue;
|
||||
}
|
||||
existing.removedFrequency += entry.removedFrequency;
|
||||
if (
|
||||
entry.removedFirstSeenMs !== null &&
|
||||
(existing.removedFirstSeenMs === null ||
|
||||
entry.removedFirstSeenMs < existing.removedFirstSeenMs)
|
||||
) {
|
||||
existing.removedFirstSeenMs = entry.removedFirstSeenMs;
|
||||
}
|
||||
if (
|
||||
entry.removedLastSeenMs !== null &&
|
||||
(existing.removedLastSeenMs === null ||
|
||||
entry.removedLastSeenMs > existing.removedLastSeenMs)
|
||||
) {
|
||||
existing.removedLastSeenMs = entry.removedLastSeenMs;
|
||||
}
|
||||
function mergeLexicalPlanEntries(
|
||||
target: LexicalRemovalPlan['words'],
|
||||
byId: Map<number, LexicalRemovalPlan['words'][number]>,
|
||||
source: LexicalRemovalPlan['words'],
|
||||
): void {
|
||||
for (const entry of source) {
|
||||
const existing = byId.get(entry.id);
|
||||
if (!existing) {
|
||||
const added = { ...entry };
|
||||
target.push(added);
|
||||
byId.set(entry.id, added);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
existing.removedFrequency += entry.removedFrequency;
|
||||
if (
|
||||
entry.removedFirstSeenMs !== null &&
|
||||
(existing.removedFirstSeenMs === null ||
|
||||
entry.removedFirstSeenMs < existing.removedFirstSeenMs)
|
||||
) {
|
||||
existing.removedFirstSeenMs = entry.removedFirstSeenMs;
|
||||
}
|
||||
if (
|
||||
entry.removedLastSeenMs !== null &&
|
||||
(existing.removedLastSeenMs === null || entry.removedLastSeenMs > existing.removedLastSeenMs)
|
||||
) {
|
||||
existing.removedLastSeenMs = entry.removedLastSeenMs;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
forEachIdChunk(sessionIds, (chunk) => {
|
||||
const plan = planLexicalRemovalsForSessions(db, chunk);
|
||||
merge(combined.words, plan.words);
|
||||
merge(combined.kanji, plan.kanji);
|
||||
interface LexicalPlanEntryMaps {
|
||||
words: Map<number, LexicalRemovalPlan['words'][number]>;
|
||||
kanji: Map<number, LexicalRemovalPlan['kanji'][number]>;
|
||||
}
|
||||
|
||||
function mergeLexicalPlans(
|
||||
target: LexicalRemovalPlan,
|
||||
byId: LexicalPlanEntryMaps,
|
||||
source: LexicalRemovalPlan,
|
||||
): void {
|
||||
mergeLexicalPlanEntries(target.words, byId.words, source.words);
|
||||
mergeLexicalPlanEntries(target.kanji, byId.kanji, source.kanji);
|
||||
}
|
||||
|
||||
/**
|
||||
* Plan what the delete removes from imm_words/imm_kanji.
|
||||
*
|
||||
* Deleted videos are planned by video so orphaned subtitle lines (whose session
|
||||
* is already gone) still get subtracted; sessions on surviving videos are
|
||||
* planned by session. The two scopes are disjoint, so nothing is counted twice.
|
||||
*/
|
||||
function planLexicalRemovalsForDelete(
|
||||
db: DatabaseSync,
|
||||
sessionIdsOnSurvivingVideos: number[],
|
||||
videoIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
const combined: LexicalRemovalPlan = { words: [], kanji: [] };
|
||||
const byId: LexicalPlanEntryMaps = {
|
||||
words: new Map(),
|
||||
kanji: new Map(),
|
||||
};
|
||||
forEachIdChunk(sessionIdsOnSurvivingVideos, (chunk) => {
|
||||
mergeLexicalPlans(combined, byId, planLexicalRemovalsForSessions(db, chunk));
|
||||
});
|
||||
forEachIdChunk(videoIds, (chunk) => {
|
||||
mergeLexicalPlans(combined, byId, planLexicalRemovalsForVideos(db, chunk));
|
||||
});
|
||||
return combined;
|
||||
}
|
||||
@@ -121,24 +154,37 @@ export function deleteMaintenanceBatch(
|
||||
}
|
||||
|
||||
const videoIdList = [...videoIds];
|
||||
for (const sessionId of selectIds(
|
||||
db,
|
||||
(placeholders) => `SELECT session_id FROM imm_sessions WHERE video_id IN (${placeholders})`,
|
||||
videoIdList,
|
||||
'session_id',
|
||||
)) {
|
||||
sessionIds.add(sessionId);
|
||||
}
|
||||
const sessionIdsOnDeletedVideos = new Set(
|
||||
selectIds(
|
||||
db,
|
||||
(placeholders) => `SELECT session_id FROM imm_sessions WHERE video_id IN (${placeholders})`,
|
||||
videoIdList,
|
||||
'session_id',
|
||||
),
|
||||
);
|
||||
for (const sessionId of sessionIdsOnDeletedVideos) sessionIds.add(sessionId);
|
||||
|
||||
const sessionIdList = [...sessionIds];
|
||||
const lexicalRemovals = planLexicalRemovalsInChunks(db, sessionIdList);
|
||||
const affectedRollupGroups = sessionIdList
|
||||
.flatMap((_, index) =>
|
||||
index % SQLITE_ID_CHUNK_SIZE === 0
|
||||
? getRollupGroupsForSessions(db, sessionIdList.slice(index, index + SQLITE_ID_CHUNK_SIZE))
|
||||
: [],
|
||||
)
|
||||
.filter((group) => !videoIds.has(group.videoId));
|
||||
const sessionIdsOnSurvivingVideos = sessionIdList.filter(
|
||||
(sessionId) => !sessionIdsOnDeletedVideos.has(sessionId),
|
||||
);
|
||||
|
||||
// Both plans must be measured before any rows are removed.
|
||||
const lexicalRemovals = planLexicalRemovalsForDelete(
|
||||
db,
|
||||
sessionIdsOnSurvivingVideos,
|
||||
videoIdList,
|
||||
);
|
||||
const lifetimeRemovals = planLifetimeRemovals(db, {
|
||||
deletedSessionIds: sessionIdList,
|
||||
sessionIdsOnSurvivingVideos,
|
||||
deletedVideoIds: videoIdList,
|
||||
deletedAnimeIds: animeIdList,
|
||||
});
|
||||
const affectedRollupGroups: RollupGroup[] = [];
|
||||
forEachIdChunk(sessionIdsOnSurvivingVideos, (chunk) => {
|
||||
affectedRollupGroups.push(...getRollupGroupsForSessions(db, chunk));
|
||||
});
|
||||
const coverBlobHashes = new Set<string>();
|
||||
if (videoIdList.length > 0) {
|
||||
forEachIdChunk(videoIdList, (chunk) => {
|
||||
@@ -152,26 +198,31 @@ export function deleteMaintenanceBatch(
|
||||
.all(...chunk) as Array<{ coverBlobHash: string }>;
|
||||
for (const row of artRows) coverBlobHashes.add(row.coverBlobHash);
|
||||
});
|
||||
|
||||
deleteSessionsByIds(db, sessionIdList);
|
||||
forEachIdChunk(videoIdList, (chunk) => {
|
||||
const placeholders = makePlaceholders(chunk);
|
||||
db.prepare(`DELETE FROM imm_subtitle_lines WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_daily_rollups WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_monthly_rollups WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_media_art WHERE video_id IN (${placeholders})`).run(...chunk);
|
||||
db.prepare(`DELETE FROM imm_videos WHERE video_id IN (${placeholders})`).run(...chunk);
|
||||
});
|
||||
} else {
|
||||
deleteSessionsByIds(db, sessionIdList);
|
||||
}
|
||||
|
||||
deleteSessionsByIds(db, sessionIdList);
|
||||
forEachIdChunk(sessionIdList, (chunk) => {
|
||||
const placeholders = makePlaceholders(chunk);
|
||||
db.prepare(
|
||||
`DELETE FROM imm_lifetime_applied_sessions WHERE session_id IN (${placeholders})`,
|
||||
).run(...chunk);
|
||||
});
|
||||
forEachIdChunk(videoIdList, (chunk) => {
|
||||
const placeholders = makePlaceholders(chunk);
|
||||
db.prepare(`DELETE FROM imm_subtitle_lines WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_daily_rollups WHERE video_id IN (${placeholders})`).run(...chunk);
|
||||
db.prepare(`DELETE FROM imm_monthly_rollups WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_media_art WHERE video_id IN (${placeholders})`).run(...chunk);
|
||||
db.prepare(`DELETE FROM imm_lifetime_media WHERE video_id IN (${placeholders})`).run(
|
||||
...chunk,
|
||||
);
|
||||
db.prepare(`DELETE FROM imm_videos WHERE video_id IN (${placeholders})`).run(...chunk);
|
||||
});
|
||||
|
||||
for (const coverBlobHash of coverBlobHashes) {
|
||||
cleanupUnusedCoverArtBlobHash(db, coverBlobHash);
|
||||
}
|
||||
@@ -186,7 +237,7 @@ export function deleteMaintenanceBatch(
|
||||
}
|
||||
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
applyLifetimeRemovals(db, lifetimeRemovals);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
|
||||
@@ -2,22 +2,20 @@ import { createHash } from 'node:crypto';
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { buildCoverBlobReference, normalizeCoverBlobBytes } from './storage';
|
||||
import {
|
||||
recomputeLifetimeAnimeAggregates,
|
||||
rebuildLifetimeSummariesInTransaction,
|
||||
recomputeLifetimeAnimeFromMedia,
|
||||
recomputeLifetimeGlobalFromSummaries,
|
||||
repairLifetimeSummariesFromMedia,
|
||||
shouldBackfillLifetimeSummaries,
|
||||
} from './lifetime';
|
||||
import { getRollupGroupsForSessions, refreshRollupsForGroupsInTransaction } from './maintenance';
|
||||
import { nowMs } from './time';
|
||||
import { resolveAnimeAnilistConflict } from './anime-season-repair';
|
||||
import { deleteMaintenanceBatch } from './query-delete-maintenance';
|
||||
import { PartOfSpeech, type MergedToken } from '../../../types';
|
||||
import { shouldExcludeTokenFromVocabularyPersistence } from '../tokenizer/annotation-stage';
|
||||
import { deriveStoredPartOfSpeech } from '../tokenizer/part-of-speech';
|
||||
import {
|
||||
applyLexicalRemovals,
|
||||
cleanupUnusedCoverArtBlobHash,
|
||||
deleteSessionsByIds,
|
||||
findSharedCoverBlobHash,
|
||||
planLexicalRemovalsForSessions,
|
||||
planLexicalRemovalsForVideos,
|
||||
toDbMs,
|
||||
toDbTimestamp,
|
||||
} from './query-shared';
|
||||
@@ -421,7 +419,6 @@ export function updateAnimeAnilistInfo(
|
||||
titleEnglish: string | null;
|
||||
titleNative: string | null;
|
||||
episodesTotal: number | null;
|
||||
exactTitleMatch?: boolean;
|
||||
},
|
||||
): void {
|
||||
const row = db.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?').get(videoId) as {
|
||||
@@ -429,11 +426,7 @@ export function updateAnimeAnilistInfo(
|
||||
} | null;
|
||||
if (!row?.anime_id) return;
|
||||
|
||||
const repair = resolveAnimeAnilistConflict(db, row.anime_id, info.anilistId, {
|
||||
matchConfidence:
|
||||
info.exactTitleMatch === true ? 'exact' : info.exactTitleMatch === false ? 'weak' : undefined,
|
||||
});
|
||||
if (repair.mergeRecommended || repair.anilistAssignmentBlocked) return;
|
||||
const conflictRepair = resolveAnimeAnilistConflict(db, row.anime_id, info.anilistId);
|
||||
const targetRow = db
|
||||
.prepare('SELECT anime_id FROM imm_videos WHERE video_id = ?')
|
||||
.get(videoId) as {
|
||||
@@ -462,8 +455,14 @@ export function updateAnimeAnilistInfo(
|
||||
toDbTimestamp(nowMs()),
|
||||
targetRow.anime_id,
|
||||
);
|
||||
if (repair.movedVideos > 0 || repair.deletedAnimeRows > 0) {
|
||||
recomputeLifetimeAnimeAggregates(db);
|
||||
if (shouldBackfillLifetimeSummaries(db)) {
|
||||
repairLifetimeSummariesFromMedia(db);
|
||||
} else {
|
||||
const affectedAnimeIds = new Set(conflictRepair.affectedAnimeIds);
|
||||
affectedAnimeIds.add(row.anime_id);
|
||||
affectedAnimeIds.add(targetRow.anime_id);
|
||||
recomputeLifetimeAnimeFromMedia(db, [...affectedAnimeIds]);
|
||||
recomputeLifetimeGlobalFromSummaries(db);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -490,136 +489,22 @@ export function isVideoWatched(db: DatabaseSync, videoId: number): boolean {
|
||||
}
|
||||
|
||||
export function deleteSession(db: DatabaseSync, sessionId: number): void {
|
||||
const sessionIds = [sessionId];
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
// Measured inside the write lock: the plan records what the delete removes,
|
||||
// and applying a plan taken against a different snapshot would subtract the
|
||||
// wrong totals from imm_words/imm_kanji.
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
deleteMaintenanceBatch(db, [{ kind: 'session', sessionId }]);
|
||||
}
|
||||
|
||||
export function deleteSessions(db: DatabaseSync, sessionIds: number[]): void {
|
||||
if (sessionIds.length === 0) return;
|
||||
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const lexicalRemovals = planLexicalRemovalsForSessions(db, sessionIds);
|
||||
const affectedRollupGroups = getRollupGroupsForSessions(db, sessionIds);
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
refreshRollupsForGroupsInTransaction(db, affectedRollupGroups);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
deleteMaintenanceBatch(db, [{ kind: 'sessions', sessionIds }]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an entire library entry: every episode of the anime, all of their
|
||||
* sessions and derived stats, and the anime row itself.
|
||||
*
|
||||
* Mirrors {@link deleteVideo} per episode, but batches the lexical refresh and
|
||||
* lifetime rebuild into a single transaction so a multi-episode title doesn't
|
||||
* pay for one full rebuild per episode.
|
||||
*/
|
||||
export function deleteAnime(db: DatabaseSync, animeId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const videoIds = (
|
||||
db.prepare('SELECT video_id FROM imm_videos WHERE anime_id = ?').all(animeId) as Array<{
|
||||
video_id: number;
|
||||
}>
|
||||
).map((row) => row.video_id);
|
||||
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, videoIds);
|
||||
const coverBlobHashes: string[] = [];
|
||||
const sessionIds: number[] = [];
|
||||
for (const videoId of videoIds) {
|
||||
const artRow = db
|
||||
.prepare('SELECT cover_blob_hash AS coverBlobHash FROM imm_media_art WHERE video_id = ?')
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
if (artRow?.coverBlobHash) {
|
||||
coverBlobHashes.push(artRow.coverBlobHash);
|
||||
}
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
sessionIds.push(...sessions.map((session) => session.session_id));
|
||||
}
|
||||
|
||||
deleteSessionsByIds(db, sessionIds);
|
||||
const deleteLinesStmt = db.prepare('DELETE FROM imm_subtitle_lines WHERE video_id = ?');
|
||||
const deleteDailyStmt = db.prepare('DELETE FROM imm_daily_rollups WHERE video_id = ?');
|
||||
const deleteMonthlyStmt = db.prepare('DELETE FROM imm_monthly_rollups WHERE video_id = ?');
|
||||
const deleteArtStmt = db.prepare('DELETE FROM imm_media_art WHERE video_id = ?');
|
||||
const deleteVideoStmt = db.prepare('DELETE FROM imm_videos WHERE video_id = ?');
|
||||
for (const videoId of videoIds) {
|
||||
deleteLinesStmt.run(videoId);
|
||||
deleteDailyStmt.run(videoId);
|
||||
deleteMonthlyStmt.run(videoId);
|
||||
deleteArtStmt.run(videoId);
|
||||
deleteVideoStmt.run(videoId);
|
||||
}
|
||||
for (const coverBlobHash of new Set(coverBlobHashes)) {
|
||||
cleanupUnusedCoverArtBlobHash(db, coverBlobHash);
|
||||
}
|
||||
db.prepare('DELETE FROM imm_lifetime_anime WHERE anime_id = ?').run(animeId);
|
||||
db.prepare('DELETE FROM imm_anime WHERE anime_id = ?').run(animeId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
deleteMaintenanceBatch(db, [{ kind: 'anime', animeId }]);
|
||||
}
|
||||
|
||||
export function deleteVideo(db: DatabaseSync, videoId: number): void {
|
||||
db.exec('BEGIN IMMEDIATE');
|
||||
try {
|
||||
const artRow = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT cover_blob_hash AS coverBlobHash
|
||||
FROM imm_media_art
|
||||
WHERE video_id = ?
|
||||
`,
|
||||
)
|
||||
.get(videoId) as { coverBlobHash: string | null } | undefined;
|
||||
const lexicalRemovals = planLexicalRemovalsForVideos(db, [videoId]);
|
||||
const sessions = db
|
||||
.prepare('SELECT session_id FROM imm_sessions WHERE video_id = ?')
|
||||
.all(videoId) as Array<{ session_id: number }>;
|
||||
|
||||
deleteSessionsByIds(
|
||||
db,
|
||||
sessions.map((session) => session.session_id),
|
||||
);
|
||||
db.prepare('DELETE FROM imm_subtitle_lines WHERE video_id = ?').run(videoId);
|
||||
db.prepare('DELETE FROM imm_daily_rollups WHERE video_id = ?').run(videoId);
|
||||
db.prepare('DELETE FROM imm_monthly_rollups WHERE video_id = ?').run(videoId);
|
||||
db.prepare('DELETE FROM imm_media_art WHERE video_id = ?').run(videoId);
|
||||
cleanupUnusedCoverArtBlobHash(db, artRow?.coverBlobHash ?? null);
|
||||
db.prepare('DELETE FROM imm_videos WHERE video_id = ?').run(videoId);
|
||||
applyLexicalRemovals(db, lexicalRemovals);
|
||||
rebuildLifetimeSummariesInTransaction(db);
|
||||
db.exec('COMMIT');
|
||||
} catch (error) {
|
||||
db.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
deleteMaintenanceBatch(db, [{ kind: 'video', videoId }]);
|
||||
}
|
||||
|
||||
@@ -276,6 +276,19 @@ export function planLexicalRemovalsForSessions(
|
||||
return planLexicalRemovals(db, `sl.session_id IN (${makePlaceholders(sessionIds)})`, sessionIds);
|
||||
}
|
||||
|
||||
/**
|
||||
* Measure what deleting these individual subtitle lines removes from the vocabulary
|
||||
* tables. Used by the duplicate-line cleanup, which drops animation frames out of the
|
||||
* middle of sessions that otherwise stay intact.
|
||||
*/
|
||||
export function planLexicalRemovalsForLines(
|
||||
db: DatabaseSync,
|
||||
lineIds: number[],
|
||||
): LexicalRemovalPlan {
|
||||
if (lineIds.length === 0) return EMPTY_LEXICAL_REMOVAL_PLAN;
|
||||
return planLexicalRemovals(db, `sl.line_id IN (${makePlaceholders(lineIds)})`, lineIds);
|
||||
}
|
||||
|
||||
/** Measure what deleting these videos removes from the vocabulary tables. */
|
||||
export function planLexicalRemovalsForVideos(
|
||||
db: DatabaseSync,
|
||||
@@ -294,10 +307,13 @@ function toStoredSeenSeconds(ms: number | null): number | null {
|
||||
* Apply a removal plan to the vocabulary aggregates.
|
||||
*
|
||||
* Frequencies are adjusted by subtraction, which is exact and touches only the
|
||||
* affected rows. `first_seen`/`last_seen` only need a rescan when the removed
|
||||
* lines held the current extreme, and rows whose frequency reaches zero are
|
||||
* verified against the surviving occurrences before deletion — so stored counts
|
||||
* that have drifted still converge on the truth instead of dropping a live row.
|
||||
* affected rows. When the removed lines held a `first_seen`/`last_seen`
|
||||
* extreme, the new extremes come from MIN/MAX index-endpoint seeks on the
|
||||
* occurrence covering index — never a re-aggregation of every occurrence, which
|
||||
* for common particles means scanning the whole library. The full re-aggregate
|
||||
* survives only as the repair path: rows whose stored frequency reaches zero
|
||||
* while occurrences remain (drift), and rows with undated pre-migration
|
||||
* occurrences the seeks would skip.
|
||||
*/
|
||||
export function applyLexicalRemovals(db: DatabaseSync, plan: LexicalRemovalPlan): void {
|
||||
applyRemovalsForEntity(db, 'word', plan.words);
|
||||
@@ -326,6 +342,21 @@ function applyRemovalsForEntity(
|
||||
`SELECT 1 AS found FROM ${occurrenceTable} WHERE ${col} = ? LIMIT 1`,
|
||||
);
|
||||
const deleteStmt = db.prepare(`DELETE FROM ${entityTable} WHERE id = ?`);
|
||||
// Seeks to the front of this entity's index range, where NULL seen_ms sorts.
|
||||
const hasUndatedOccurrenceStmt = db.prepare(
|
||||
`SELECT 1 AS found FROM ${occurrenceTable} WHERE ${col} = ? AND seen_ms IS NULL LIMIT 1`,
|
||||
);
|
||||
// Kept as separate single-aggregate statements so SQLite's min/max
|
||||
// optimization turns each into an index-endpoint seek instead of a scan.
|
||||
const minSeenStmt = db.prepare(
|
||||
`SELECT MIN(seen_ms) AS value FROM ${occurrenceTable} WHERE ${col} = ?`,
|
||||
);
|
||||
const maxSeenStmt = db.prepare(
|
||||
`SELECT MAX(seen_ms) AS value FROM ${occurrenceTable} WHERE ${col} = ?`,
|
||||
);
|
||||
const updateAggregatesStmt = db.prepare(
|
||||
`UPDATE ${entityTable} SET frequency = ?, first_seen = ?, last_seen = ? WHERE id = ?`,
|
||||
);
|
||||
|
||||
const needsExactRefresh: number[] = [];
|
||||
|
||||
@@ -358,7 +389,26 @@ function applyRemovalsForEntity(
|
||||
current.lastSeen === null ||
|
||||
(removedLastSeen !== null && removedLastSeen >= current.lastSeen);
|
||||
if (firstSeenMayHaveMoved || lastSeenMayHaveMoved) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
// Undated pre-migration occurrences are invisible to the seeks below;
|
||||
// fall back to the full re-aggregate that resolves their dates.
|
||||
if (hasUndatedOccurrenceStmt.get(removal.id)) {
|
||||
needsExactRefresh.push(removal.id);
|
||||
continue;
|
||||
}
|
||||
const minSeenMs = (minSeenStmt.get(removal.id) as { value: number | null }).value;
|
||||
const maxSeenMs = (maxSeenStmt.get(removal.id) as { value: number | null }).value;
|
||||
if (minSeenMs === null || maxSeenMs === null) {
|
||||
// Frequency says occurrences remain but none exist: stale row, let the
|
||||
// exact refresh reconcile (it deletes rows with nothing left).
|
||||
needsExactRefresh.push(removal.id);
|
||||
continue;
|
||||
}
|
||||
updateAggregatesStmt.run(
|
||||
nextFrequency,
|
||||
Math.floor(Number(minSeenMs) / 1000),
|
||||
Math.floor(Number(maxSeenMs) / 1000),
|
||||
removal.id,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
@@ -20,7 +20,6 @@ import {
|
||||
} from './storage';
|
||||
import {
|
||||
EVENT_SUBTITLE_LINE,
|
||||
SCHEMA_VERSION,
|
||||
SESSION_STATUS_ENDED,
|
||||
SOURCE_TYPE_LOCAL,
|
||||
SOURCE_TYPE_REMOTE,
|
||||
@@ -133,7 +132,6 @@ test('ensureSchema creates immersion core tables', () => {
|
||||
assert.ok(videoColumns.has('parser_source'));
|
||||
assert.ok(videoColumns.has('parser_confidence'));
|
||||
assert.ok(videoColumns.has('parse_metadata_json'));
|
||||
assert.ok(videoColumns.has('anime_assignment_locked'));
|
||||
|
||||
const mediaArtColumns = new Set(
|
||||
(
|
||||
@@ -157,33 +155,6 @@ test('ensureSchema creates immersion core tables', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('ensureSchema adds manual assignment locks when upgrading the previous schema', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
db.exec('ALTER TABLE imm_videos DROP COLUMN anime_assignment_locked');
|
||||
db.prepare('UPDATE imm_schema_version SET schema_version = ?').run(SCHEMA_VERSION - 1);
|
||||
|
||||
ensureSchema(db);
|
||||
|
||||
const columns = new Set(
|
||||
(db.prepare('PRAGMA table_info(imm_videos)').all() as Array<{ name: string }>).map(
|
||||
(row) => row.name,
|
||||
),
|
||||
);
|
||||
assert.ok(columns.has('anime_assignment_locked'));
|
||||
const version = db
|
||||
.prepare('SELECT MAX(schema_version) AS version FROM imm_schema_version')
|
||||
.get() as { version: number };
|
||||
assert.equal(version.version, SCHEMA_VERSION);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('stats excluded words are replaced and read from sqlite storage', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
@@ -836,7 +807,6 @@ test('ensureSchema migrates legacy videos and backfills anime metadata from file
|
||||
assert.ok(videoColumns.has('parser_source'));
|
||||
assert.ok(videoColumns.has('parser_confidence'));
|
||||
assert.ok(videoColumns.has('parse_metadata_json'));
|
||||
assert.ok(videoColumns.has('anime_assignment_locked'));
|
||||
|
||||
const animeRows = db
|
||||
.prepare('SELECT canonical_title FROM imm_anime ORDER BY canonical_title')
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import path from 'node:path';
|
||||
import { parseMediaInfo } from '../../../jimaku/utils';
|
||||
import { normalizeTitleIdentity } from '../../utils/title-normalization';
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import { nowMs } from './time';
|
||||
import { SCHEMA_VERSION } from './types';
|
||||
@@ -321,7 +319,14 @@ export function applyPragmas(db: DatabaseSync): void {
|
||||
db.exec(`PRAGMA journal_size_limit = ${WAL_JOURNAL_SIZE_LIMIT_BYTES}`);
|
||||
}
|
||||
|
||||
export const normalizeAnimeIdentityKey = normalizeTitleIdentity;
|
||||
export function normalizeAnimeIdentityKey(title: string): string {
|
||||
return title
|
||||
.normalize('NFKC')
|
||||
.toLowerCase()
|
||||
.replace(/[^\p{L}\p{N}]+/gu, ' ')
|
||||
.trim()
|
||||
.replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
function normalizeSeasonScope(value: number | null | undefined): number | null {
|
||||
if (typeof value !== 'number' || !Number.isSafeInteger(value) || value <= 0) {
|
||||
@@ -525,36 +530,6 @@ function ensureStatsExcludedWordsTable(db: DatabaseSync): void {
|
||||
`);
|
||||
}
|
||||
|
||||
function ensureAnimeMergeTables(db: DatabaseSync): void {
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS imm_anime_title_aliases(
|
||||
normalized_title_key TEXT PRIMARY KEY,
|
||||
anime_id INTEGER NOT NULL,
|
||||
CREATED_DATE TEXT,
|
||||
LAST_UPDATE_DATE TEXT,
|
||||
FOREIGN KEY(anime_id) REFERENCES imm_anime(anime_id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_anime_title_aliases_anime_id
|
||||
ON imm_anime_title_aliases(anime_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS imm_anime_merge_recommendations(
|
||||
recommendation_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
first_anime_id INTEGER NOT NULL,
|
||||
second_anime_id INTEGER NOT NULL,
|
||||
anilist_id INTEGER NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'dismissed')),
|
||||
CREATED_DATE TEXT,
|
||||
LAST_UPDATE_DATE TEXT,
|
||||
CHECK(first_anime_id < second_anime_id),
|
||||
UNIQUE(first_anime_id, second_anime_id, anilist_id),
|
||||
FOREIGN KEY(first_anime_id) REFERENCES imm_anime(anime_id) ON DELETE CASCADE,
|
||||
FOREIGN KEY(second_anime_id) REFERENCES imm_anime(anime_id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_anime_merge_recommendations_status
|
||||
ON imm_anime_merge_recommendations(status, recommendation_id);
|
||||
`);
|
||||
}
|
||||
|
||||
export function getOrCreateAnimeRecord(db: DatabaseSync, input: AnimeRecordInput): number {
|
||||
const seasonScope = normalizeSeasonScope(input.seasonScope);
|
||||
const identityTitle = buildSeasonScopedAnimeTitle(input.parsedTitle, seasonScope);
|
||||
@@ -575,14 +550,8 @@ export function getOrCreateAnimeRecord(db: DatabaseSync, input: AnimeRecordInput
|
||||
const byNormalizedTitle = db
|
||||
.prepare('SELECT anime_id FROM imm_anime WHERE normalized_title_key = ?')
|
||||
.get(normalizedTitleKey) as { anime_id: number } | null;
|
||||
const byTitleAlias = db
|
||||
.prepare('SELECT anime_id FROM imm_anime_title_aliases WHERE normalized_title_key = ?')
|
||||
.get(normalizedTitleKey) as { anime_id: number } | null;
|
||||
const existing = byAnilistId ?? byNormalizedTitle ?? byTitleAlias;
|
||||
const existing = byAnilistId ?? byNormalizedTitle;
|
||||
if (existing?.anime_id) {
|
||||
// An alias remembers an intentionally merged-away spelling. Reusing it
|
||||
// must not rename the survivor back to that discarded display title.
|
||||
const canonicalTitleUpdate = byAnilistId || byNormalizedTitle ? canonicalTitle : null;
|
||||
db.prepare(
|
||||
`
|
||||
UPDATE imm_anime
|
||||
@@ -597,7 +566,7 @@ export function getOrCreateAnimeRecord(db: DatabaseSync, input: AnimeRecordInput
|
||||
WHERE anime_id = ?
|
||||
`,
|
||||
).run(
|
||||
canonicalTitleUpdate,
|
||||
canonicalTitle,
|
||||
input.anilistId,
|
||||
input.titleRomaji,
|
||||
input.titleEnglish,
|
||||
@@ -649,10 +618,7 @@ export function linkVideoToAnimeRecord(
|
||||
`
|
||||
UPDATE imm_videos
|
||||
SET
|
||||
anime_id = CASE
|
||||
WHEN anime_assignment_locked = 1 THEN anime_id
|
||||
ELSE ?
|
||||
END,
|
||||
anime_id = ?,
|
||||
parsed_basename = ?,
|
||||
parsed_title = ?,
|
||||
parsed_season = ?,
|
||||
@@ -677,67 +643,6 @@ export function linkVideoToAnimeRecord(
|
||||
);
|
||||
}
|
||||
|
||||
export function getManualAnimeAssignment(db: DatabaseSync, videoId: number): number | null {
|
||||
const row = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT anime_id AS animeId
|
||||
FROM imm_videos
|
||||
WHERE video_id = ?
|
||||
AND anime_assignment_locked = 1
|
||||
`,
|
||||
)
|
||||
.get(videoId) as { animeId: number | null } | null;
|
||||
return row?.animeId ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* A manual correction in the same folder is a useful grouping hint, but only
|
||||
* when every season-compatible correction agrees on the destination.
|
||||
*/
|
||||
export function findManualDirectoryAnimeAssignment(
|
||||
db: DatabaseSync,
|
||||
videoId: number,
|
||||
mediaPath: string,
|
||||
parsedSeason: number | null,
|
||||
): number | null {
|
||||
const directory = path.dirname(path.resolve(mediaPath));
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
anime_id AS animeId,
|
||||
source_path AS sourcePath,
|
||||
parsed_season AS parsedSeason
|
||||
FROM imm_videos
|
||||
WHERE video_id != ?
|
||||
AND anime_assignment_locked = 1
|
||||
AND anime_id IS NOT NULL
|
||||
AND source_path IS NOT NULL
|
||||
`,
|
||||
)
|
||||
.all(videoId) as Array<{
|
||||
animeId: number;
|
||||
sourcePath: string;
|
||||
parsedSeason: number | null;
|
||||
}>;
|
||||
|
||||
const candidates = new Set<number>();
|
||||
for (const row of rows) {
|
||||
if (path.dirname(path.resolve(row.sourcePath)) !== directory) {
|
||||
continue;
|
||||
}
|
||||
if (parsedSeason !== null && row.parsedSeason !== null && parsedSeason !== row.parsedSeason) {
|
||||
continue;
|
||||
}
|
||||
candidates.add(row.animeId);
|
||||
if (candidates.size > 1) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return candidates.values().next().value ?? null;
|
||||
}
|
||||
|
||||
export function linkYoutubeVideoToAnimeRecord(
|
||||
db: DatabaseSync,
|
||||
videoId: number,
|
||||
@@ -846,7 +751,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
if (currentVersion?.schema_version === SCHEMA_VERSION) {
|
||||
ensureLifetimeSummaryTables(db);
|
||||
ensureStatsExcludedWordsTable(db);
|
||||
ensureAnimeMergeTables(db);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -882,7 +786,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
parser_source TEXT,
|
||||
parser_confidence REAL,
|
||||
parse_metadata_json TEXT,
|
||||
anime_assignment_locked INTEGER NOT NULL DEFAULT 0 CHECK(anime_assignment_locked IN (0, 1)),
|
||||
watched INTEGER NOT NULL DEFAULT 0,
|
||||
duration_ms INTEGER NOT NULL CHECK(duration_ms>=0),
|
||||
file_size_bytes INTEGER CHECK(file_size_bytes>=0),
|
||||
@@ -896,13 +799,6 @@ export function ensureSchema(db: DatabaseSync): void {
|
||||
FOREIGN KEY(anime_id) REFERENCES imm_anime(anime_id) ON DELETE SET NULL
|
||||
);
|
||||
`);
|
||||
addColumnIfMissing(
|
||||
db,
|
||||
'imm_videos',
|
||||
'anime_assignment_locked',
|
||||
'INTEGER NOT NULL DEFAULT 0 CHECK(anime_assignment_locked IN (0, 1))',
|
||||
);
|
||||
ensureAnimeMergeTables(db);
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS imm_sessions(
|
||||
session_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
export const SCHEMA_VERSION = 21;
|
||||
export const SCHEMA_VERSION = 19;
|
||||
export const DEFAULT_QUEUE_CAP = 1_000;
|
||||
export const DEFAULT_BATCH_SIZE = 25;
|
||||
export const DEFAULT_FLUSH_INTERVAL_MS = 500;
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
import type { Hono } from 'hono';
|
||||
import { statsJson } from '../../../types/stats-http-contract.js';
|
||||
import { UNKNOWN_MOVE_TARGET_MESSAGE } from '../immersion-tracker/anime-merge.js';
|
||||
import type { ImmersionTrackerService } from '../immersion-tracker-service.js';
|
||||
import {
|
||||
buildSentenceSearchOptions,
|
||||
enrichSessionsWithKnownWordMetrics,
|
||||
parseBooleanQuery,
|
||||
parseDuplicateLineCleanupBody,
|
||||
parseExcludedWordsBody,
|
||||
parseIntQuery,
|
||||
parsePositiveIdList,
|
||||
} from './route-support.js';
|
||||
|
||||
export function registerStatsLibraryRoutes(
|
||||
@@ -42,6 +41,19 @@ export function registerStatsLibraryRoutes(
|
||||
return c.json(statsJson('setExcludedWords', { ok: true }));
|
||||
});
|
||||
|
||||
// Collapse animation bursts older versions recorded frame by frame. `dryRun` measures
|
||||
// the same scan without writing, so the confirmation the user sees is the real cost.
|
||||
app.post('/api/stats/maintenance/duplicate-lines', async (c) => {
|
||||
const contentType = c.req.header('content-type')?.split(';', 1)[0]?.trim().toLowerCase();
|
||||
if (contentType !== 'application/json') return c.body(null, 415);
|
||||
const body = await c.req.json().catch(() => null);
|
||||
const options = parseDuplicateLineCleanupBody(body);
|
||||
if (!options) return c.body(null, 400);
|
||||
const { dryRun, lookbackDays } = options;
|
||||
const result = await tracker.cleanupDuplicateSubtitleLines({ dryRun, lookbackDays });
|
||||
return c.json(statsJson('duplicateLineCleanup', result));
|
||||
});
|
||||
|
||||
app.get('/api/stats/vocabulary/occurrences', async (c) => {
|
||||
const headword = (c.req.query('headword') ?? '').trim();
|
||||
const word = (c.req.query('word') ?? '').trim();
|
||||
@@ -132,19 +144,6 @@ export function registerStatsLibraryRoutes(
|
||||
return c.json(statsJson('animeLibrary', rows));
|
||||
});
|
||||
|
||||
app.get('/api/stats/anime/merge-recommendations', async (c) => {
|
||||
const recommendations = await tracker.getAnimeMergeRecommendations();
|
||||
return c.json(statsJson('animeMergeRecommendations', { recommendations }));
|
||||
});
|
||||
|
||||
app.delete('/api/stats/anime/merge-recommendations/:recommendationId', async (c) => {
|
||||
const recommendationId = parseIntQuery(c.req.param('recommendationId'), 0);
|
||||
if (recommendationId <= 0) return c.body(null, 400);
|
||||
const dismissed = await tracker.dismissAnimeMergeRecommendation(recommendationId);
|
||||
if (!dismissed) return c.body(null, 404);
|
||||
return c.json(statsJson('dismissAnimeMergeRecommendation', { ok: true }));
|
||||
});
|
||||
|
||||
app.get('/api/stats/anime/:animeId', async (c) => {
|
||||
const animeId = parseIntQuery(c.req.param('animeId'), 0);
|
||||
if (animeId <= 0) return c.body(null, 400);
|
||||
@@ -212,50 +211,4 @@ export function registerStatsLibraryRoutes(
|
||||
await tracker.deleteAnime(animeId);
|
||||
return c.json(statsJson('deleteAnime', { ok: true }));
|
||||
});
|
||||
|
||||
app.post('/api/stats/anime/:animeId/merge', async (c) => {
|
||||
const animeId = parseIntQuery(c.req.param('animeId'), 0);
|
||||
if (animeId <= 0) return c.body(null, 400);
|
||||
const body = await c.req.json().catch(() => null);
|
||||
const sourceAnimeIds = parsePositiveIdList(body?.sourceAnimeIds).filter((id) => id !== animeId);
|
||||
if (sourceAnimeIds.length === 0) return c.body(null, 400);
|
||||
const summary = await tracker.mergeAnime(animeId, sourceAnimeIds);
|
||||
// Nothing folded means the target or every source was already gone, so the
|
||||
// caller should not be told the merge succeeded.
|
||||
if (summary.mergedAnimeIds.length === 0) return c.body(null, 404);
|
||||
return c.json(
|
||||
statsJson('mergeAnime', {
|
||||
ok: true,
|
||||
animeId: summary.survivingAnimeId,
|
||||
mergedAnimeIds: summary.mergedAnimeIds,
|
||||
movedVideos: summary.movedVideos,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
app.patch('/api/stats/media/:videoId/anime', async (c) => {
|
||||
const videoId = parseIntQuery(c.req.param('videoId'), 0);
|
||||
if (videoId <= 0) return c.body(null, 400);
|
||||
const body = await c.req.json().catch(() => null);
|
||||
const animeId = Number.isSafeInteger(body?.animeId) ? (body.animeId as number) : 0;
|
||||
if (animeId <= 0) return c.body(null, 400);
|
||||
try {
|
||||
const summary = await tracker.moveVideoToAnime(videoId, animeId);
|
||||
return c.json(
|
||||
statsJson('moveVideoToAnime', {
|
||||
ok: true,
|
||||
animeId: summary.targetAnimeId,
|
||||
previousAnimeId: summary.previousAnimeId,
|
||||
removedPreviousAnime: summary.removedPreviousAnime,
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
// Only a missing episode or entry is a 404; storage failures must not be
|
||||
// reported to the caller as "not found".
|
||||
if (error instanceof Error && error.message === UNKNOWN_MOVE_TARGET_MESSAGE) {
|
||||
return c.body(null, 404);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -88,6 +88,35 @@ export function parseExcludedWordsBody(body: unknown): StatsExcludedWord[] | nul
|
||||
return words;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a duplicate-line cleanup request. An explicit object with no lookback scans all
|
||||
* history. Invalid bodies and invalid windows are rejected instead of broadening scope.
|
||||
*/
|
||||
export function parseDuplicateLineCleanupBody(body: unknown): {
|
||||
dryRun: boolean;
|
||||
lookbackDays: number | null;
|
||||
} | null {
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body)) {
|
||||
return null;
|
||||
}
|
||||
const source = body as Record<string, unknown>;
|
||||
if (source.dryRun !== undefined && typeof source.dryRun !== 'boolean') {
|
||||
return null;
|
||||
}
|
||||
const rawLookback = source.lookbackDays;
|
||||
if (
|
||||
rawLookback !== undefined &&
|
||||
rawLookback !== null &&
|
||||
(typeof rawLookback !== 'number' || !Number.isFinite(rawLookback) || rawLookback < 1)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
dryRun: source.dryRun === true,
|
||||
lookbackDays: typeof rawLookback === 'number' ? Math.floor(rawLookback) : null,
|
||||
};
|
||||
}
|
||||
|
||||
export function loadKnownWordsSet(cachePath: string | undefined): Set<string> | null {
|
||||
if (!cachePath || !existsSync(cachePath)) return null;
|
||||
try {
|
||||
@@ -170,18 +199,6 @@ export async function enrichSessionsWithKnownWordMetrics<
|
||||
);
|
||||
}
|
||||
|
||||
/** Deduplicated positive integer ids from an untrusted JSON body field. */
|
||||
export function parsePositiveIdList(raw: unknown): number[] {
|
||||
if (!Array.isArray(raw)) return [];
|
||||
const ids = new Set<number>();
|
||||
for (const value of raw) {
|
||||
if (Number.isSafeInteger(value) && (value as number) > 0) {
|
||||
ids.add(value as number);
|
||||
}
|
||||
}
|
||||
return [...ids];
|
||||
}
|
||||
|
||||
export function parseBooleanQuery(raw: string | undefined, fallback: boolean): boolean {
|
||||
if (raw === undefined) return fallback;
|
||||
const normalized = raw.trim().toLowerCase();
|
||||
|
||||
@@ -28,7 +28,6 @@ const VIDEO_COPY_COLUMNS = [
|
||||
'parser_source',
|
||||
'parser_confidence',
|
||||
'parse_metadata_json',
|
||||
'anime_assignment_locked',
|
||||
'watched',
|
||||
'duration_ms',
|
||||
'file_size_bytes',
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
/*
|
||||
* Thresholds that decide when a run of repeated subtitle events is one animation.
|
||||
*
|
||||
* Three consumers have to agree on these numbers or the same karaoke line is one cue in
|
||||
* the sidebar and two hundred in the stats: the file-level cue dedup
|
||||
* (`subtitle-cue-dedup`), the live gate that decides what immersion stats record
|
||||
* (`subtitle-line-dedup-gate`), and the retroactive database cleanup
|
||||
* (`immersion-tracker/duplicate-line-cleanup`).
|
||||
*/
|
||||
|
||||
/**
|
||||
* Back-to-back frames of the same animation are authored flush against each other; a
|
||||
* tiny tolerance absorbs the centisecond rounding of the ASS timestamp format.
|
||||
*/
|
||||
export const DUPLICATE_CUE_GAP_TOLERANCE_SECONDS = 0.05;
|
||||
|
||||
/**
|
||||
* A burst is a *sequence*. Two adjacent events are two events, not an animation --
|
||||
* characters do repeat each other, and a repeated line can legitimately be short.
|
||||
*/
|
||||
export const MIN_BURST_EVENTS = 3;
|
||||
|
||||
/**
|
||||
* Real dialogue holds on screen for about a second, so a run with a couple of much
|
||||
* shorter events among them looks like frames. Used only alongside authoring evidence.
|
||||
*/
|
||||
export const ANIMATION_FRAME_MAX_SECONDS = 0.3;
|
||||
|
||||
/** A karaoke run usually ends on a long "hold" frame, so not every event is short. */
|
||||
export const MIN_TAGGED_BURST_FRAMES = 2;
|
||||
|
||||
/**
|
||||
* SRT and VTT carry no authoring metadata at all, so timing is the only signal available
|
||||
* -- which makes it the easiest one to get wrong. ASS->SRT conversion leaves frames at
|
||||
* ~0.04s, well under any real utterance, and a burst leaves many of them behind. Both
|
||||
* bounds are deliberately far stricter than the ASS path: a run of ordinary short lines
|
||||
* (`えっ` traded between characters) must not clear them.
|
||||
*/
|
||||
export const TIMING_ONLY_FRAME_MAX_SECONDS = 0.1;
|
||||
export const MIN_TIMING_ONLY_FRAMES = 5;
|
||||
|
||||
/**
|
||||
* The live gate can only recognise a streaming burst from the inside, so it records the
|
||||
* first `MIN_TIMING_ONLY_FRAMES - 1` frames before it starts dropping. That stored
|
||||
* residue is one frame short of the timing-only minimum, and the retroactive cleanup
|
||||
* accepts it only when every event also sits under the strict timing-only frame bound.
|
||||
*/
|
||||
export const MIN_STREAM_RESIDUE_FRAMES = MIN_TIMING_ONLY_FRAMES - 1;
|
||||
@@ -7,31 +7,20 @@
|
||||
*/
|
||||
|
||||
import { hasAssTemporalOverride, isAnimatedAssEffectKind } from './ass-text';
|
||||
import {
|
||||
ANIMATION_FRAME_MAX_SECONDS,
|
||||
DUPLICATE_CUE_GAP_TOLERANCE_SECONDS,
|
||||
MIN_BURST_EVENTS,
|
||||
MIN_TAGGED_BURST_FRAMES,
|
||||
MIN_TIMING_ONLY_FRAMES,
|
||||
TIMING_ONLY_FRAME_MAX_SECONDS,
|
||||
} from './subtitle-burst-constants';
|
||||
import type {
|
||||
AnnotatedSubtitleCue,
|
||||
SubtitleCue,
|
||||
SubtitleSourceFormat,
|
||||
} from './subtitle-cue-parser';
|
||||
|
||||
// Back-to-back frames of the same animation are authored flush against each other; a
|
||||
// tiny tolerance absorbs the centisecond rounding of the ASS timestamp format.
|
||||
const DUPLICATE_CUE_GAP_TOLERANCE_SECONDS = 0.05;
|
||||
// A burst is a *sequence*. Two adjacent events are two events, not an animation --
|
||||
// characters do repeat each other, and a repeated line can legitimately be short.
|
||||
const MIN_BURST_EVENTS = 3;
|
||||
// Real dialogue holds on screen for about a second, so a run with a couple of much
|
||||
// shorter events among them looks like frames. Used only alongside authoring evidence.
|
||||
const ANIMATION_FRAME_MAX_SECONDS = 0.3;
|
||||
// A karaoke run usually ends on a long "hold" frame, so not every event is short.
|
||||
const MIN_TAGGED_BURST_FRAMES = 2;
|
||||
// SRT and VTT carry no authoring metadata at all, so timing is the only signal available
|
||||
// -- which makes it the easiest one to get wrong. ASS->SRT conversion leaves frames at
|
||||
// ~0.04s, well under any real utterance, and a burst leaves many of them behind. Both
|
||||
// bounds are deliberately far stricter than the ASS path: a run of ordinary short lines
|
||||
// (`えっ` traded between characters) must not clear them.
|
||||
const TIMING_ONLY_FRAME_MAX_SECONDS = 0.1;
|
||||
const MIN_TIMING_ONLY_FRAMES = 5;
|
||||
|
||||
function cueKey(cue: SubtitleCue): string {
|
||||
return `${cue.startTime}|${cue.endTime}|${cue.text}`;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { createSubtitleLineDedupGate } from './subtitle-line-dedup-gate';
|
||||
import type { SubtitleCue } from '../../types';
|
||||
|
||||
function karaokeFrames(text: string, start: number, frames: number, frameSeconds: number) {
|
||||
return Array.from({ length: frames }, (_, index) => ({
|
||||
text,
|
||||
startSec: start + index * frameSeconds,
|
||||
endSec: start + (index + 1) * frameSeconds,
|
||||
}));
|
||||
}
|
||||
|
||||
test('parsed cues drop the frames the sidebar already collapsed', () => {
|
||||
// What `mergeDuplicateCues` leaves behind for a karaoke run: one cue over the run.
|
||||
const cues: SubtitleCue[] = [
|
||||
{ startTime: 10, endTime: 14, text: '飛び上がる' },
|
||||
{ startTime: 14, endTime: 16, text: 'もしも' },
|
||||
];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
const recorded = karaokeFrames('飛び上がる', 10, 40, 0.04).filter((sample) =>
|
||||
gate.shouldRecord(sample),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 1);
|
||||
assert.equal(recorded[0]!.startSec, 10);
|
||||
assert.equal(gate.shouldRecord({ text: 'もしも', startSec: 14, endSec: 16 }), true);
|
||||
});
|
||||
|
||||
test('parsed cues keep separate lines that merely repeat', () => {
|
||||
const cues: SubtitleCue[] = [
|
||||
{ startTime: 3, endTime: 3.4, text: 'えっ' },
|
||||
{ startTime: 3.4, endTime: 3.9, text: 'えっ' },
|
||||
{ startTime: 3.9, endTime: 4.5, text: 'えっ' },
|
||||
];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
const recorded = cues.filter((cue) =>
|
||||
gate.shouldRecord({ text: cue.text, startSec: cue.startTime, endSec: cue.endTime }),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 3);
|
||||
});
|
||||
|
||||
test('parsed cues outrank the streaming heuristic for short repeated cues', () => {
|
||||
// Long enough to trip the timing-only rule, but the parser saw these with full
|
||||
// lookahead and kept them, so every one of them is a line the sidebar shows.
|
||||
const cues: SubtitleCue[] = Array.from({ length: 8 }, (_, index) => ({
|
||||
startTime: 3 + index * 0.08,
|
||||
endTime: 3 + (index + 1) * 0.08,
|
||||
text: 'えっ',
|
||||
}));
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
const recorded = cues.filter((cue) =>
|
||||
gate.shouldRecord({ text: cue.text, startSec: cue.startTime, endSec: cue.endTime }),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 8);
|
||||
});
|
||||
|
||||
test('parsed cues preserve legitimately separate cues only 40ms apart', () => {
|
||||
const cues: SubtitleCue[] = Array.from({ length: 8 }, (_, index) => ({
|
||||
startTime: 3 + index * 0.04,
|
||||
endTime: 3 + (index + 1) * 0.04,
|
||||
text: 'えっ',
|
||||
}));
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
const recorded = cues.filter((cue) =>
|
||||
gate.shouldRecord({ text: cue.text, startSec: cue.startTime, endSec: cue.endTime }),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 8);
|
||||
});
|
||||
|
||||
test('a line whose timing does not match any cue still records', () => {
|
||||
// A shifted track, an embedded sub nobody parsed: no match, no drop.
|
||||
const cues: SubtitleCue[] = [{ startTime: 10, endTime: 14, text: '飛び上がる' }];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
assert.equal(gate.shouldRecord({ text: '飛び上がる', startSec: 42, endSec: 44 }), true);
|
||||
});
|
||||
|
||||
test('shifted parsed text falls back to streaming burst detection', () => {
|
||||
const cues: SubtitleCue[] = [{ startTime: 10, endTime: 14, text: '飛び上がる' }];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
const recorded = karaokeFrames('飛び上がる', 42, 40, 0.04).filter((sample) =>
|
||||
gate.shouldRecord(sample),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 4);
|
||||
});
|
||||
|
||||
test('replacing the parsed cue source forgets a streaming run', () => {
|
||||
let cues: SubtitleCue[] = [];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
karaokeFrames('飛び上がる', 42, 20, 0.04).forEach((sample) => gate.shouldRecord(sample));
|
||||
// A new source publishes its own cue list; the old run must not carry over.
|
||||
cues = [{ startTime: 100, endTime: 104, text: 'もしも' }];
|
||||
|
||||
assert.equal(gate.shouldRecord({ text: '飛び上がる', startSec: 42.8, endSec: 42.84 }), true);
|
||||
});
|
||||
|
||||
test('without parsed cues a long run of identical short frames stops recording', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
const recorded = karaokeFrames('ひとしずく', 0, 200, 0.04).filter((sample) =>
|
||||
gate.shouldRecord(sample),
|
||||
);
|
||||
|
||||
assert.equal(recorded.length, 4);
|
||||
});
|
||||
|
||||
test('without parsed cues interleaved dual-line karaoke stops recording per line', () => {
|
||||
// Fansub OPs typically run two typeset lines at once -- kanji and romaji -- and mpv
|
||||
// reports their frames interleaved. Each line must build its own run.
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
const samples = Array.from({ length: 40 }, (_, index) => {
|
||||
const start = 10 + Math.floor(index / 2) * 0.06;
|
||||
return index % 2 === 0
|
||||
? { text: '歌詞', startSec: start, endSec: start + 0.06 }
|
||||
: { text: 'kashi', startSec: start + 0.001, endSec: start + 0.061 };
|
||||
});
|
||||
const recorded = samples.filter((sample) => gate.shouldRecord(sample));
|
||||
|
||||
assert.equal(recorded.filter((sample) => sample.text === '歌詞').length, 4);
|
||||
assert.equal(recorded.filter((sample) => sample.text === 'kashi').length, 4);
|
||||
});
|
||||
|
||||
test('interleaved dialogue between two speakers keeps recording', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
// Two characters trading normal-length lines back and forth.
|
||||
const samples = Array.from({ length: 12 }, (_, index) => {
|
||||
const start = 5 + index * 0.7;
|
||||
return {
|
||||
text: index % 2 === 0 ? 'えっ' : 'なに',
|
||||
startSec: start,
|
||||
endSec: start + 0.7,
|
||||
};
|
||||
});
|
||||
const recorded = samples.filter((sample) => gate.shouldRecord(sample));
|
||||
|
||||
assert.equal(recorded.length, 12);
|
||||
});
|
||||
|
||||
test('without parsed cues ordinary repeated dialogue keeps recording', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
// Six contiguous `えっ`, each held for a normal beat rather than an animation frame.
|
||||
const recorded = karaokeFrames('えっ', 0, 6, 0.6).filter((sample) => gate.shouldRecord(sample));
|
||||
|
||||
assert.equal(recorded.length, 6);
|
||||
});
|
||||
|
||||
test('the same event offered twice does not advance the run', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
// mpv fires the timing handler once for `sub-start` and once for `sub-end`.
|
||||
for (let i = 0; i < 8; i += 1) {
|
||||
assert.equal(gate.shouldRecord({ text: '待って', startSec: 5, endSec: 5.05 }), true);
|
||||
}
|
||||
});
|
||||
|
||||
test('a gap between frames starts a new run', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
const first = karaokeFrames('もし', 0, 6, 0.04).filter((sample) => gate.shouldRecord(sample));
|
||||
const second = karaokeFrames('もし', 30, 6, 0.04).filter((sample) => gate.shouldRecord(sample));
|
||||
|
||||
assert.equal(first.length, 4);
|
||||
assert.equal(second.length, 4);
|
||||
});
|
||||
|
||||
test('reset forgets the streaming run', () => {
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => null });
|
||||
|
||||
karaokeFrames('もし', 0, 20, 0.04).forEach((sample) => gate.shouldRecord(sample));
|
||||
gate.reset();
|
||||
|
||||
assert.equal(gate.shouldRecord({ text: 'もし', startSec: 0.8, endSec: 0.84 }), true);
|
||||
});
|
||||
|
||||
test('reset ignores stale parsed cues until the source publishes a new cue list', () => {
|
||||
let cues: SubtitleCue[] = [{ startTime: 10, endTime: 14, text: '飛び上がる' }];
|
||||
const gate = createSubtitleLineDedupGate({ getParsedCues: () => cues });
|
||||
|
||||
assert.equal(gate.shouldRecord({ text: '飛び上がる', startSec: 10, endSec: 10.04 }), true);
|
||||
gate.reset();
|
||||
|
||||
const recordedWithStaleCues = karaokeFrames('飛び上がる', 10.04, 8, 0.04).filter((sample) =>
|
||||
gate.shouldRecord(sample),
|
||||
);
|
||||
assert.equal(recordedWithStaleCues.length, 4);
|
||||
|
||||
cues = [{ startTime: 20, endTime: 24, text: '飛び上がる' }];
|
||||
assert.equal(gate.shouldRecord({ text: '飛び上がる', startSec: 20, endSec: 20.04 }), true);
|
||||
assert.equal(gate.shouldRecord({ text: '飛び上がる', startSec: 20.04, endSec: 20.08 }), false);
|
||||
});
|
||||
@@ -0,0 +1,241 @@
|
||||
/*
|
||||
* Decides which live mpv subtitle lines reach the immersion stats.
|
||||
*
|
||||
* The sidebar reads a parsed subtitle file, so it can collapse an animation burst with
|
||||
* full lookahead (`subtitle-cue-dedup`). Stats are fed from mpv's `sub-start`/`sub-end`
|
||||
* properties instead -- one event per animation frame, each with its own start time --
|
||||
* so without a gate a karaoke OP counts its lyrics once per frame and buries every real
|
||||
* word in the vocabulary charts.
|
||||
*
|
||||
* Two layers, in order:
|
||||
*
|
||||
* 1. When the active source has been parsed, its cue list has *already* been collapsed.
|
||||
* A live line that lands inside a surviving cue of the same text, but after that
|
||||
* cue's start, is a frame the sidebar merged away, so stats drop it too. This is the
|
||||
* layer that keeps the two views consistent by construction.
|
||||
* 2. Otherwise (embedded track nobody parsed, a source whose timings mpv has shifted)
|
||||
* fall back to timing alone. No authoring metadata is available live -- mpv delivers
|
||||
* `sub-text-ass` after `sub-start`/`sub-end`, so any ASS text read here belongs to the
|
||||
* previous event -- which puts this layer in the same position as the SRT path in
|
||||
* `subtitle-cue-dedup`, and it uses that path's deliberately strict bounds.
|
||||
*/
|
||||
|
||||
import { normalizePlainSubtitleText } from './ass-text';
|
||||
import {
|
||||
DUPLICATE_CUE_GAP_TOLERANCE_SECONDS,
|
||||
MIN_TIMING_ONLY_FRAMES,
|
||||
TIMING_ONLY_FRAME_MAX_SECONDS,
|
||||
} from './subtitle-burst-constants';
|
||||
import type { SubtitleCue } from './subtitle-cue-parser';
|
||||
|
||||
export interface SubtitleLineSample {
|
||||
text: string;
|
||||
startSec: number;
|
||||
endSec: number;
|
||||
}
|
||||
|
||||
export interface SubtitleLineDedupGateDeps {
|
||||
/** Cues for the active source, already collapsed by the parser. */
|
||||
getParsedCues: () => readonly SubtitleCue[] | null | undefined;
|
||||
}
|
||||
|
||||
export interface SubtitleLineDedupGate {
|
||||
/** False when this line is an animation frame of a line already recorded. */
|
||||
shouldRecord: (sample: SubtitleLineSample) => boolean;
|
||||
/** Forget run state and ignore the current cue list until its source is replaced. */
|
||||
reset: () => void;
|
||||
}
|
||||
|
||||
interface CueSpan {
|
||||
startTime: number;
|
||||
endTime: number;
|
||||
}
|
||||
|
||||
interface StreamingRunState {
|
||||
startMs: number;
|
||||
chainEndSec: number;
|
||||
/** Contiguous identical short frames seen so far, including the recorded first one. */
|
||||
frames: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dual-line karaoke interleaves two texts frame by frame, so runs are tracked per text.
|
||||
* Dead runs are pruned as playback moves past them; the cap only matters after a
|
||||
* backward seek leaves runs whose ends sit ahead of the new position.
|
||||
*/
|
||||
const MAX_ACTIVE_STREAMING_RUNS = 32;
|
||||
|
||||
/** Exact cue identity, separate from the looser tolerance used to chain adjacent frames. */
|
||||
const CUE_START_IDENTITY_TOLERANCE_SECONDS = 0.005;
|
||||
|
||||
function normalizeLineText(text: string): string {
|
||||
return normalizePlainSubtitleText(text, { collapseLineBreaks: true });
|
||||
}
|
||||
|
||||
function buildSpansByText(cues: readonly SubtitleCue[]): Map<string, CueSpan[]> {
|
||||
const spansByText = new Map<string, CueSpan[]>();
|
||||
for (const cue of cues) {
|
||||
const key = normalizeLineText(cue.text);
|
||||
if (!key) continue;
|
||||
const span = { startTime: cue.startTime, endTime: cue.endTime };
|
||||
const existing = spansByText.get(key);
|
||||
if (existing) {
|
||||
existing.push(span);
|
||||
} else {
|
||||
spansByText.set(key, [span]);
|
||||
}
|
||||
}
|
||||
return spansByText;
|
||||
}
|
||||
|
||||
/**
|
||||
* A frame the parser merged away: the same text, starting inside a surviving cue but
|
||||
* after it began.
|
||||
*
|
||||
* Starting a cue always wins over falling inside one. The first frame of a collapsed run
|
||||
* starts *at* the merged cue, and a line the parser deliberately kept separate -- three
|
||||
* characters trading `えっ` back to back -- begins exactly where the one before it ends.
|
||||
*/
|
||||
function isMergedAwayFrame(spans: readonly CueSpan[], startSec: number): boolean | null {
|
||||
const coveringSpans = spans.filter(
|
||||
(span) =>
|
||||
startSec >= span.startTime - CUE_START_IDENTITY_TOLERANCE_SECONDS &&
|
||||
startSec <= span.endTime + CUE_START_IDENTITY_TOLERANCE_SECONDS,
|
||||
);
|
||||
if (coveringSpans.length === 0) {
|
||||
return null;
|
||||
}
|
||||
const startsOwnCue = spans.some(
|
||||
(span) => Math.abs(startSec - span.startTime) <= CUE_START_IDENTITY_TOLERANCE_SECONDS,
|
||||
);
|
||||
if (startsOwnCue) {
|
||||
return false;
|
||||
}
|
||||
return coveringSpans.some(
|
||||
(span) =>
|
||||
startSec > span.startTime + CUE_START_IDENTITY_TOLERANCE_SECONDS &&
|
||||
startSec <= span.endTime + CUE_START_IDENTITY_TOLERANCE_SECONDS,
|
||||
);
|
||||
}
|
||||
|
||||
export function createSubtitleLineDedupGate(
|
||||
deps: SubtitleLineDedupGateDeps,
|
||||
): SubtitleLineDedupGate {
|
||||
let indexedCues: readonly SubtitleCue[] | null | undefined;
|
||||
let ignoredCuesAfterReset: readonly SubtitleCue[] | null | undefined;
|
||||
let spansByText: Map<string, CueSpan[]> = new Map();
|
||||
const runs = new Map<string, StreamingRunState>();
|
||||
|
||||
const lookupSpans = (text: string): CueSpan[] | null => {
|
||||
const cues = deps.getParsedCues() ?? null;
|
||||
if (ignoredCuesAfterReset !== undefined) {
|
||||
if (cues === ignoredCuesAfterReset) {
|
||||
return null;
|
||||
}
|
||||
ignoredCuesAfterReset = undefined;
|
||||
}
|
||||
if (cues !== indexedCues) {
|
||||
indexedCues = cues;
|
||||
spansByText = cues?.length ? buildSpansByText(cues) : new Map();
|
||||
runs.clear();
|
||||
}
|
||||
return spansByText.get(text) ?? null;
|
||||
};
|
||||
|
||||
/**
|
||||
* A run this sample cannot continue is a run no later sample can continue either --
|
||||
* continuation needs a start inside the running end plus tolerance, and starts only
|
||||
* move forward outside of seeks.
|
||||
*/
|
||||
const pruneDeadRuns = (startSec: number): void => {
|
||||
for (const [text, state] of runs) {
|
||||
if (state.chainEndSec + DUPLICATE_CUE_GAP_TOLERANCE_SECONDS < startSec) {
|
||||
runs.delete(text);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Timing-only burst detection over a stream. Without lookahead the run can only be
|
||||
* recognised from the inside, so the first frames of a burst are recorded and the rest
|
||||
* dropped -- an OP costs a handful of counted lines instead of several hundred.
|
||||
*/
|
||||
const advanceStreamingRun = (text: string, sample: SubtitleLineSample): boolean => {
|
||||
pruneDeadRuns(sample.startSec);
|
||||
const startMs = Math.round(sample.startSec * 1000);
|
||||
const run = runs.get(text);
|
||||
// mpv reports `sub-start` and `sub-end` separately, so one event can be offered
|
||||
// twice. The same start is the same frame, never the next one in a run.
|
||||
if (run && run.startMs === startMs) {
|
||||
run.chainEndSec = Math.max(run.chainEndSec, sample.endSec);
|
||||
return run.frames < MIN_TIMING_ONLY_FRAMES;
|
||||
}
|
||||
|
||||
const isShortFrame = sample.endSec - sample.startSec < TIMING_ONLY_FRAME_MAX_SECONDS;
|
||||
// Frames are authored flush against each other, but typesetters do overlap them, so
|
||||
// the chain only requires forward progress that stays inside the running end.
|
||||
const continuesRun =
|
||||
run !== undefined &&
|
||||
isShortFrame &&
|
||||
startMs > run.startMs &&
|
||||
sample.startSec <= run.chainEndSec + DUPLICATE_CUE_GAP_TOLERANCE_SECONDS;
|
||||
|
||||
if (continuesRun && run) {
|
||||
run.startMs = startMs;
|
||||
run.chainEndSec = Math.max(run.chainEndSec, sample.endSec);
|
||||
run.frames += 1;
|
||||
return run.frames < MIN_TIMING_ONLY_FRAMES;
|
||||
}
|
||||
|
||||
const fresh: StreamingRunState = {
|
||||
startMs,
|
||||
chainEndSec: sample.endSec,
|
||||
frames: isShortFrame ? 1 : 0,
|
||||
};
|
||||
runs.set(text, fresh);
|
||||
if (runs.size > MAX_ACTIVE_STREAMING_RUNS) {
|
||||
let oldestText: string | undefined;
|
||||
let oldestEnd = Infinity;
|
||||
for (const [runText, state] of runs) {
|
||||
if (runText !== text && state.chainEndSec < oldestEnd) {
|
||||
oldestEnd = state.chainEndSec;
|
||||
oldestText = runText;
|
||||
}
|
||||
}
|
||||
if (oldestText !== undefined) {
|
||||
runs.delete(oldestText);
|
||||
}
|
||||
}
|
||||
return fresh.frames < MIN_TIMING_ONLY_FRAMES;
|
||||
};
|
||||
|
||||
return {
|
||||
shouldRecord: (sample) => {
|
||||
const text = normalizeLineText(sample.text);
|
||||
if (!text) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// The parsed cue list has the final say wherever it covers this line. Falling
|
||||
// through to the streaming heuristic would let it drop cues the parser looked at
|
||||
// with full lookahead and deliberately kept apart, which is the disagreement
|
||||
// between sidebar and stats this gate exists to prevent.
|
||||
const spans = lookupSpans(text);
|
||||
if (spans) {
|
||||
const mergedAway = isMergedAwayFrame(spans, sample.startSec);
|
||||
if (mergedAway !== null) {
|
||||
runs.delete(text);
|
||||
return !mergedAway;
|
||||
}
|
||||
}
|
||||
|
||||
return advanceStreamingRun(text, sample);
|
||||
},
|
||||
reset: () => {
|
||||
runs.clear();
|
||||
ignoredCuesAfterReset = deps.getParsedCues() ?? null;
|
||||
indexedCues = undefined;
|
||||
spansByText = new Map();
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { normalizeTitleIdentity } from './title-normalization';
|
||||
|
||||
test('normalizeTitleIdentity produces a Unicode-aware comparison key', () => {
|
||||
assert.equal(normalizeTitleIdentity(' ＢＯＣＣＨＩ・The ROCK!! '), 'bocchi the rock');
|
||||
});
|
||||
@@ -1,8 +0,0 @@
|
||||
export function normalizeTitleIdentity(title: string): string {
|
||||
return title
|
||||
.normalize('NFKC')
|
||||
.toLowerCase()
|
||||
.replace(/[^\p{L}\p{N}]+/gu, ' ')
|
||||
.trim()
|
||||
.replace(/\s+/g, ' ');
|
||||
}
|
||||
@@ -267,3 +267,123 @@ test('flushPlaybackPositionOnMediaPathClear ignores disconnected mpv time-pos re
|
||||
|
||||
assert.deepEqual(recorded, [42]);
|
||||
});
|
||||
|
||||
test('media and subtitle-track transitions reset live subtitle-line deduplication', () => {
|
||||
const recordedStarts: number[] = [];
|
||||
const handlers = createBuildBindMpvMainEventHandlersMainDepsHandler({
|
||||
appState: {
|
||||
initialArgs: null,
|
||||
overlayRuntimeInitialized: true,
|
||||
mpvClient: null,
|
||||
immersionTracker: {
|
||||
recordSubtitleLine: (_text: string, start: number) => recordedStarts.push(start),
|
||||
},
|
||||
subtitleTimingTracker: null,
|
||||
activeParsedSubtitleCues: null,
|
||||
currentMediaPath: '/video-a.mkv',
|
||||
currentSubText: '',
|
||||
currentSubAssText: '',
|
||||
playbackPaused: null,
|
||||
previousSecondarySubVisibility: false,
|
||||
},
|
||||
getQuitOnDisconnectArmed: () => false,
|
||||
scheduleQuitCheck: () => {},
|
||||
quitApp: () => {},
|
||||
reportJellyfinRemoteStopped: () => {},
|
||||
syncOverlayMpvSubtitleSuppression: () => {},
|
||||
maybeRunAnilistPostWatchUpdate: async () => {},
|
||||
logSubtitleTimingError: () => {},
|
||||
broadcastToOverlayWindows: () => {},
|
||||
onSubtitleChange: () => {},
|
||||
ensureImmersionTrackerInitialized: () => {},
|
||||
updateCurrentMediaPath: () => {},
|
||||
restoreMpvSubVisibility: () => {},
|
||||
resetSubtitleSidebarEmbeddedLayout: () => {},
|
||||
getCurrentAnilistMediaKey: () => null,
|
||||
resetAnilistMediaTracking: () => {},
|
||||
maybeProbeAnilistDuration: () => {},
|
||||
ensureAnilistMediaGuess: () => {},
|
||||
syncImmersionMediaState: () => {},
|
||||
updateCurrentMediaTitle: () => {},
|
||||
resetAnilistMediaGuessState: () => {},
|
||||
reportJellyfinRemoteProgress: () => {},
|
||||
updateSubtitleRenderMetrics: () => {},
|
||||
refreshDiscordPresence: () => {},
|
||||
})();
|
||||
|
||||
for (let index = 0; index < 8; index += 1) {
|
||||
handlers.recordImmersionSubtitleLine('待って', index * 0.04, (index + 1) * 0.04);
|
||||
}
|
||||
assert.equal(recordedStarts.length, 4);
|
||||
|
||||
handlers.updateCurrentMediaPath('/video-b.mkv');
|
||||
handlers.recordImmersionSubtitleLine('待って', 0.32, 0.36);
|
||||
assert.equal(recordedStarts.length, 5);
|
||||
|
||||
for (let index = 9; index < 16; index += 1) {
|
||||
handlers.recordImmersionSubtitleLine('待って', index * 0.04, (index + 1) * 0.04);
|
||||
}
|
||||
assert.equal(recordedStarts.length, 8);
|
||||
|
||||
assert.equal(typeof handlers.onSubtitleTrackChange, 'function');
|
||||
handlers.onSubtitleTrackChange?.(2);
|
||||
handlers.recordImmersionSubtitleLine('待って', 0.64, 0.68);
|
||||
assert.equal(recordedStarts.length, 9);
|
||||
});
|
||||
|
||||
test('subtitle-track transitions ignore stale parsed cues until replacement cues arrive', () => {
|
||||
const recordedStarts: number[] = [];
|
||||
const appState = {
|
||||
initialArgs: null,
|
||||
overlayRuntimeInitialized: true,
|
||||
mpvClient: null,
|
||||
immersionTracker: {
|
||||
recordSubtitleLine: (_text: string, start: number) => recordedStarts.push(start),
|
||||
},
|
||||
subtitleTimingTracker: null,
|
||||
activeParsedSubtitleCues: [{ startTime: 10, endTime: 14, text: '飛び上がる' }],
|
||||
currentMediaPath: '/video-a.mkv',
|
||||
currentSubText: '',
|
||||
currentSubAssText: '',
|
||||
playbackPaused: null,
|
||||
previousSecondarySubVisibility: false,
|
||||
};
|
||||
const handlers = createBuildBindMpvMainEventHandlersMainDepsHandler({
|
||||
appState,
|
||||
getQuitOnDisconnectArmed: () => false,
|
||||
scheduleQuitCheck: () => {},
|
||||
quitApp: () => {},
|
||||
reportJellyfinRemoteStopped: () => {},
|
||||
syncOverlayMpvSubtitleSuppression: () => {},
|
||||
maybeRunAnilistPostWatchUpdate: async () => {},
|
||||
logSubtitleTimingError: () => {},
|
||||
broadcastToOverlayWindows: () => {},
|
||||
onSubtitleChange: () => {},
|
||||
ensureImmersionTrackerInitialized: () => {},
|
||||
updateCurrentMediaPath: () => {},
|
||||
restoreMpvSubVisibility: () => {},
|
||||
resetSubtitleSidebarEmbeddedLayout: () => {},
|
||||
getCurrentAnilistMediaKey: () => null,
|
||||
resetAnilistMediaTracking: () => {},
|
||||
maybeProbeAnilistDuration: () => {},
|
||||
ensureAnilistMediaGuess: () => {},
|
||||
syncImmersionMediaState: () => {},
|
||||
updateCurrentMediaTitle: () => {},
|
||||
resetAnilistMediaGuessState: () => {},
|
||||
reportJellyfinRemoteProgress: () => {},
|
||||
updateSubtitleRenderMetrics: () => {},
|
||||
refreshDiscordPresence: () => {},
|
||||
})();
|
||||
|
||||
handlers.recordImmersionSubtitleLine('飛び上がる', 10, 10.04);
|
||||
handlers.onSubtitleTrackChange?.(2);
|
||||
for (let index = 1; index <= 8; index += 1) {
|
||||
handlers.recordImmersionSubtitleLine('飛び上がる', 10 + index * 0.04, 10 + (index + 1) * 0.04);
|
||||
}
|
||||
assert.equal(recordedStarts.length, 5);
|
||||
|
||||
appState.activeParsedSubtitleCues = [{ startTime: 20, endTime: 24, text: '飛び上がる' }];
|
||||
handlers.recordImmersionSubtitleLine('飛び上がる', 20, 20.04);
|
||||
handlers.recordImmersionSubtitleLine('飛び上がる', 20.04, 20.08);
|
||||
assert.deepEqual(recordedStarts.slice(-1), [20]);
|
||||
});
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { MergedToken, SubtitleData } from '../../types';
|
||||
import { createSubtitleLineDedupGate } from '../../core/services/subtitle-line-dedup-gate';
|
||||
import type { MergedToken, SubtitleCue, SubtitleData } from '../../types';
|
||||
|
||||
type AnilistPostWatchRunOptions = {
|
||||
watchedSeconds?: number;
|
||||
@@ -34,6 +35,7 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
|
||||
subtitleTimingTracker: {
|
||||
recordSubtitle?: (text: string, start: number, end: number, secondaryText?: string) => void;
|
||||
} | null;
|
||||
activeParsedSubtitleCues?: SubtitleCue[] | null;
|
||||
currentMediaPath?: string | null;
|
||||
currentSubText: string;
|
||||
currentSubAssText: string;
|
||||
@@ -86,6 +88,11 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
|
||||
deps.ensureImmersionTrackerInitialized();
|
||||
deps.appState.immersionTracker?.recordPlaybackPosition?.(normalizedTimeSec);
|
||||
};
|
||||
// mpv reports every animation frame of a typeset line as its own subtitle event, so
|
||||
// stats have to collapse bursts the same way the parsed cue list already does.
|
||||
const immersionLineDedupGate = createSubtitleLineDedupGate({
|
||||
getParsedCues: () => deps.appState.activeParsedSubtitleCues,
|
||||
});
|
||||
const hasInitialPlaybackQuitOnDisconnectArg = (): boolean =>
|
||||
Boolean(
|
||||
deps.appState.initialArgs?.managedPlayback ||
|
||||
@@ -110,6 +117,9 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
|
||||
if (!tracker?.recordSubtitleLine) {
|
||||
return;
|
||||
}
|
||||
if (!immersionLineDedupGate.shouldRecord({ text, startSec: start, endSec: end })) {
|
||||
return;
|
||||
}
|
||||
const secondaryText = deps.appState.mpvClient?.currentSecondarySubText || null;
|
||||
const cachedTokens =
|
||||
deps.appState.currentSubtitleData?.text === text
|
||||
@@ -159,9 +169,10 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
|
||||
logSubtitleProcessingDebug: deps.logSubtitleProcessingDebug
|
||||
? (message: string) => deps.logSubtitleProcessingDebug!(message)
|
||||
: undefined,
|
||||
onSubtitleTrackChange: deps.onSubtitleTrackChange
|
||||
? (sid: number | null) => deps.onSubtitleTrackChange!(sid)
|
||||
: undefined,
|
||||
onSubtitleTrackChange: (sid: number | null) => {
|
||||
immersionLineDedupGate.reset();
|
||||
deps.onSubtitleTrackChange?.(sid);
|
||||
},
|
||||
onSubtitleTrackListChange: deps.onSubtitleTrackListChange
|
||||
? (trackList: unknown[] | null) => deps.onSubtitleTrackListChange!(trackList)
|
||||
: undefined,
|
||||
@@ -173,7 +184,10 @@ export function createBuildBindMpvMainEventHandlersMainDepsHandler(deps: {
|
||||
deps.broadcastToOverlayWindows('subtitle-ass:set', text),
|
||||
broadcastSecondarySubtitle: (text: string) =>
|
||||
deps.broadcastToOverlayWindows('secondary-subtitle:set', text),
|
||||
updateCurrentMediaPath: (path: string) => deps.updateCurrentMediaPath(path),
|
||||
updateCurrentMediaPath: (path: string) => {
|
||||
immersionLineDedupGate.reset();
|
||||
deps.updateCurrentMediaPath(path);
|
||||
},
|
||||
restoreMpvSubVisibility: () => deps.restoreMpvSubVisibility(),
|
||||
resetSubtitleSidebarEmbeddedLayout: () => deps.resetSubtitleSidebarEmbeddedLayout?.(),
|
||||
getCurrentAnilistMediaKey: () => deps.getCurrentAnilistMediaKey(),
|
||||
|
||||
@@ -200,6 +200,59 @@ test('stats cli command fails when immersion tracking is disabled', async () =>
|
||||
]);
|
||||
});
|
||||
|
||||
test('stats cli command runs a duplicate-line cleanup preview without touching the dashboard', async () => {
|
||||
const { handler, calls, responses } = makeHandler({
|
||||
getImmersionTracker: () => ({
|
||||
cleanupDuplicateSubtitleLines: async (options: {
|
||||
dryRun?: boolean;
|
||||
lookbackDays?: number | null;
|
||||
}) => ({
|
||||
dryRun: options.dryRun === true,
|
||||
lookbackDays: options.lookbackDays ?? null,
|
||||
scannedLines: 900,
|
||||
burstGroups: 2,
|
||||
removedLines: 180,
|
||||
removedWordOccurrences: 540,
|
||||
removedKanjiOccurrences: 120,
|
||||
samples: [
|
||||
{
|
||||
videoId: 7,
|
||||
videoTitle: 'Ep 1',
|
||||
text: '飛び上がる',
|
||||
frames: 90,
|
||||
removedLines: 89,
|
||||
startMs: 1000,
|
||||
endMs: 5000,
|
||||
},
|
||||
],
|
||||
}),
|
||||
}),
|
||||
});
|
||||
|
||||
await handler(
|
||||
{
|
||||
statsResponsePath: '/tmp/subminer-stats-response.json',
|
||||
statsCleanup: true,
|
||||
statsCleanupDuplicateLines: true,
|
||||
statsCleanupDryRun: true,
|
||||
statsCleanupLookbackDays: 30,
|
||||
},
|
||||
'initial',
|
||||
);
|
||||
|
||||
assert.deepEqual(calls, [
|
||||
'ensureImmersionTrackerStarted',
|
||||
'info:Stats duplicate-line cleanup preview (last 30d): scanned=900 bursts=2 removedLines=180 removedWordCounts=540 removedKanjiCounts=120',
|
||||
'info: Ep 1: "飛び上がる" x90',
|
||||
]);
|
||||
assert.deepEqual(responses, [
|
||||
{
|
||||
responsePath: '/tmp/subminer-stats-response.json',
|
||||
payload: { ok: true },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test('stats cli command runs vocab cleanup instead of opening dashboard when cleanup mode is requested', async () => {
|
||||
const { handler, calls, responses } = makeHandler({
|
||||
getImmersionTracker: () => ({
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import type { CliArgs, CliCommandSource } from '../../cli/args';
|
||||
import type { DuplicateSubtitleLineCleanupSummary } from '../../core/services/immersion-tracker/duplicate-line-cleanup';
|
||||
import type {
|
||||
LifetimeRebuildSummary,
|
||||
VocabularyCleanupSummary,
|
||||
@@ -50,6 +51,10 @@ export function createRunStatsCliCommandHandler(deps: {
|
||||
ensureVocabularyCleanupTokenizerReady?: () => Promise<void> | void;
|
||||
getImmersionTracker: () => {
|
||||
cleanupVocabularyStats?: () => Promise<VocabularyCleanupSummary>;
|
||||
cleanupDuplicateSubtitleLines?: (options: {
|
||||
dryRun?: boolean;
|
||||
lookbackDays?: number | null;
|
||||
}) => Promise<DuplicateSubtitleLineCleanupSummary>;
|
||||
rebuildLifetimeSummaries?: () => Promise<LifetimeRebuildSummary>;
|
||||
} | null;
|
||||
ensureStatsServerStarted: () => string;
|
||||
@@ -83,6 +88,9 @@ export function createRunStatsCliCommandHandler(deps: {
|
||||
| 'statsCleanup'
|
||||
| 'statsCleanupVocab'
|
||||
| 'statsCleanupLifetime'
|
||||
| 'statsCleanupDuplicateLines'
|
||||
| 'statsCleanupDryRun'
|
||||
| 'statsCleanupLookbackDays'
|
||||
>,
|
||||
source: CliCommandSource,
|
||||
): Promise<void> => {
|
||||
@@ -126,6 +134,7 @@ export function createRunStatsCliCommandHandler(deps: {
|
||||
const cleanupModes = [
|
||||
args.statsCleanupVocab ? 'vocab' : null,
|
||||
args.statsCleanupLifetime ? 'lifetime' : null,
|
||||
args.statsCleanupDuplicateLines ? 'duplicate-lines' : null,
|
||||
].filter(Boolean);
|
||||
if (cleanupModes.length !== 1) {
|
||||
throw new Error('Choose exactly one stats cleanup mode.');
|
||||
@@ -142,6 +151,27 @@ export function createRunStatsCliCommandHandler(deps: {
|
||||
writeResponseSafe(args.statsResponsePath, { ok: true });
|
||||
return;
|
||||
}
|
||||
if (args.statsCleanupDuplicateLines && tracker.cleanupDuplicateSubtitleLines) {
|
||||
const result = await tracker.cleanupDuplicateSubtitleLines({
|
||||
dryRun: args.statsCleanupDryRun === true,
|
||||
lookbackDays: args.statsCleanupLookbackDays ?? null,
|
||||
});
|
||||
const window =
|
||||
result.lookbackDays === null ? 'all history' : `last ${result.lookbackDays}d`;
|
||||
deps.logInfo(
|
||||
`Stats duplicate-line cleanup ${result.dryRun ? 'preview' : 'complete'} (${window}): ` +
|
||||
`scanned=${result.scannedLines} bursts=${result.burstGroups} ` +
|
||||
`removedLines=${result.removedLines} removedWordCounts=${result.removedWordOccurrences} ` +
|
||||
`removedKanjiCounts=${result.removedKanjiOccurrences}`,
|
||||
);
|
||||
for (const sample of result.samples.slice(0, 5)) {
|
||||
deps.logInfo(
|
||||
` ${sample.videoTitle ?? `video ${sample.videoId}`}: "${sample.text}" x${sample.frames}`,
|
||||
);
|
||||
}
|
||||
writeResponseSafe(args.statsResponsePath, { ok: true });
|
||||
return;
|
||||
}
|
||||
if (!args.statsCleanupLifetime || !tracker.rebuildLifetimeSummaries) {
|
||||
throw new Error('Stats cleanup mode is not available.');
|
||||
}
|
||||
|
||||
@@ -4,13 +4,18 @@ import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import test from 'node:test';
|
||||
|
||||
import { buildAnimatedImageVideoFilter, MediaGenerator } from './media-generator';
|
||||
import {
|
||||
AUDIO_GENERATION_TIMEOUT_MS,
|
||||
buildAnimatedImageVideoFilter,
|
||||
MediaGenerator,
|
||||
type MediaGeneratorOptions,
|
||||
} from './media-generator';
|
||||
|
||||
async function withStubbedFfmpeg(
|
||||
run: (generator: MediaGenerator, argsPath: string) => Promise<void>,
|
||||
options: {
|
||||
logDebug?: (message: string) => void;
|
||||
now?: () => number;
|
||||
options: MediaGeneratorOptions = {},
|
||||
stubOptions: {
|
||||
skipOutput?: boolean;
|
||||
} = {},
|
||||
): Promise<void> {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-media-generator-test-'));
|
||||
@@ -31,7 +36,9 @@ async function withStubbedFfmpeg(
|
||||
'}',
|
||||
"fs.writeFileSync(process.env.SUBMINER_TEST_FFMPEG_ARGS, JSON.stringify(args), 'utf8');",
|
||||
'const outputPath = args.at(-1);',
|
||||
"fs.writeFileSync(outputPath, 'avif', 'utf8');",
|
||||
"if (process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT !== '1') {",
|
||||
" fs.writeFileSync(outputPath, 'avif', 'utf8');",
|
||||
'}',
|
||||
].join('\n'),
|
||||
'utf8',
|
||||
);
|
||||
@@ -46,8 +53,14 @@ async function withStubbedFfmpeg(
|
||||
|
||||
const originalPath = process.env.PATH;
|
||||
const originalArgsPath = process.env.SUBMINER_TEST_FFMPEG_ARGS;
|
||||
const originalSkipOutput = process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT;
|
||||
process.env.PATH = `${binDir}${path.delimiter}${originalPath ?? ''}`;
|
||||
process.env.SUBMINER_TEST_FFMPEG_ARGS = argsPath;
|
||||
if (stubOptions.skipOutput) {
|
||||
process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT = '1';
|
||||
} else {
|
||||
delete process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT;
|
||||
}
|
||||
const generator = new MediaGenerator(tempDir, options);
|
||||
|
||||
try {
|
||||
@@ -60,6 +73,11 @@ async function withStubbedFfmpeg(
|
||||
} else {
|
||||
process.env.SUBMINER_TEST_FFMPEG_ARGS = originalArgsPath;
|
||||
}
|
||||
if (originalSkipOutput === undefined) {
|
||||
delete process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT;
|
||||
} else {
|
||||
process.env.SUBMINER_TEST_FFMPEG_SKIP_OUTPUT = originalSkipOutput;
|
||||
}
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
@@ -316,6 +334,65 @@ test('generateAudio keeps explicit audio stream maps for normal media paths', as
|
||||
});
|
||||
});
|
||||
|
||||
test('generateAudio bounds probing when the selected local audio stream is known', async () => {
|
||||
await withStubbedFfmpeg(async (generator, argsPath) => {
|
||||
await generator.generateAudio('/video.mkv', 10, 12, 0, 2);
|
||||
|
||||
const args = readFfmpegArgs(argsPath);
|
||||
const inputIndex = args.indexOf('-i');
|
||||
assert.ok(args.indexOf('-probesize') > -1);
|
||||
assert.ok(args.indexOf('-probesize') < inputIndex);
|
||||
assert.equal(args[args.indexOf('-probesize') + 1], '32768');
|
||||
assert.ok(args.indexOf('-analyzeduration') < inputIndex);
|
||||
assert.equal(args[args.indexOf('-analyzeduration') + 1], '0');
|
||||
});
|
||||
});
|
||||
|
||||
test('generateAudio retains normal probing for non-Matroska local media', async () => {
|
||||
await withStubbedFfmpeg(async (generator, argsPath) => {
|
||||
await generator.generateAudio('/video.mp4', 10, 12, 0, 2);
|
||||
|
||||
const args = readFfmpegArgs(argsPath);
|
||||
assert.equal(args.includes('-probesize'), false);
|
||||
assert.equal(args.includes('-analyzeduration'), false);
|
||||
});
|
||||
});
|
||||
|
||||
test('generateAudio retains a two-minute extraction timeout', async () => {
|
||||
let observedTimeout: number | undefined;
|
||||
|
||||
await withStubbedFfmpeg(
|
||||
async (generator) => {
|
||||
await generator.generateAudio('/video.mp4', 10, 12);
|
||||
},
|
||||
{
|
||||
execFile: (_file, args, options, callback) => {
|
||||
observedTimeout = options.timeout;
|
||||
const outputPath = args.at(-1);
|
||||
assert.ok(outputPath);
|
||||
fs.writeFileSync(outputPath, 'mp3', 'utf8');
|
||||
queueMicrotask(() => callback(null));
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
assert.equal(AUDIO_GENERATION_TIMEOUT_MS, 120_000);
|
||||
assert.equal(observedTimeout, AUDIO_GENERATION_TIMEOUT_MS);
|
||||
});
|
||||
|
||||
test('generateAudio reports when ffmpeg exits without creating output', async () => {
|
||||
await withStubbedFfmpeg(
|
||||
async (generator) => {
|
||||
await assert.rejects(
|
||||
generator.generateAudio('/video.mp4', 10, 12),
|
||||
/FFmpeg audio generation failed: FFmpeg exited without creating an output file/,
|
||||
);
|
||||
},
|
||||
{},
|
||||
{ skipOutput: true },
|
||||
);
|
||||
});
|
||||
|
||||
test('generateAudio debug-logs cached input and completion timing', async () => {
|
||||
const logs: string[] = [];
|
||||
const times = [1000, 1052];
|
||||
|
||||
+32
-8
@@ -26,9 +26,17 @@ import { normalizeMediaInput, type MediaInput } from './media-input';
|
||||
const log = createLogger('media');
|
||||
const AUDIO_NORMALIZATION_FILTER = 'loudnorm=I=-23:TP=-2:LRA=11';
|
||||
const AUDIO_AMPLIFICATION_LIMITER_FILTER = 'alimiter=limit=0.891251:level=false';
|
||||
export const AUDIO_GENERATION_TIMEOUT_MS = 120_000;
|
||||
|
||||
export type { MediaInput, MediaInputOptions } from './media-input';
|
||||
|
||||
type MediaGeneratorExecFile = (
|
||||
file: string,
|
||||
args: readonly string[],
|
||||
options: { timeout: number },
|
||||
callback: (error: ExecFileException | null) => void,
|
||||
) => void;
|
||||
|
||||
function normalizeAnimatedImageFps(fps: number | undefined): number {
|
||||
const fallbackFps = 10;
|
||||
const safeFps = typeof fps === 'number' && Number.isFinite(fps) ? fps : fallbackFps;
|
||||
@@ -77,6 +85,7 @@ export function buildAnimatedImageVideoFilter(options: {
|
||||
export interface MediaGeneratorOptions {
|
||||
logDebug?: (message: string) => void;
|
||||
now?: () => number;
|
||||
execFile?: MediaGeneratorExecFile;
|
||||
}
|
||||
|
||||
function sanitizeDebugToken(value: string, fallback: string): string {
|
||||
@@ -274,6 +283,14 @@ export class MediaGenerator {
|
||||
const duration = endTime - start + safePadding;
|
||||
const mediaInput = normalizeMediaInput(videoPath);
|
||||
const inputDescription = describeMediaInputForDebugLog(videoPath);
|
||||
const hasSelectedAudioStream =
|
||||
!mediaInput.singleResolvedStream &&
|
||||
typeof audioStreamIndex === 'number' &&
|
||||
Number.isInteger(audioStreamIndex) &&
|
||||
audioStreamIndex >= 0;
|
||||
const isLocalMatroskaMedia =
|
||||
!/^[A-Za-z][A-Za-z\d+.-]*:\/\//.test(mediaInput.path) &&
|
||||
/\.(?:mkv|mka|mks|webm)$/i.test(mediaInput.path);
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const outputPath = this.createTempOutputPath('audio', 'mp3');
|
||||
@@ -284,16 +301,14 @@ export class MediaGenerator {
|
||||
'-t',
|
||||
duration.toString(),
|
||||
...mediaInput.inputArgs,
|
||||
...(hasSelectedAudioStream && isLocalMatroskaMedia
|
||||
? ['-probesize', '32768', '-analyzeduration', '0']
|
||||
: []),
|
||||
'-i',
|
||||
mediaInput.path,
|
||||
];
|
||||
|
||||
if (
|
||||
!mediaInput.singleResolvedStream &&
|
||||
typeof audioStreamIndex === 'number' &&
|
||||
Number.isInteger(audioStreamIndex) &&
|
||||
audioStreamIndex >= 0
|
||||
) {
|
||||
if (hasSelectedAudioStream) {
|
||||
args.push('-map', `0:${audioStreamIndex}`);
|
||||
}
|
||||
|
||||
@@ -321,7 +336,8 @@ export class MediaGenerator {
|
||||
this.logMediaDebug(
|
||||
`audio start ${inputDescription} start=${start} duration=${duration} padding=${safePadding}`,
|
||||
);
|
||||
execFile('ffmpeg', args, { timeout: 30000 }, (error) => {
|
||||
const runExecFile: MediaGeneratorExecFile = this.options.execFile ?? execFile;
|
||||
runExecFile('ffmpeg', args, { timeout: AUDIO_GENERATION_TIMEOUT_MS }, (error) => {
|
||||
if (error) {
|
||||
this.logMediaDebug(
|
||||
`audio failed ${inputDescription} elapsedMs=${this.elapsedMs(startedAt)} ${describeFfmpegFailureForDebugLog(error)}`,
|
||||
@@ -338,7 +354,15 @@ export class MediaGenerator {
|
||||
);
|
||||
resolve(data);
|
||||
} catch (err) {
|
||||
reject(err);
|
||||
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
|
||||
reject(
|
||||
new Error(
|
||||
'FFmpeg audio generation failed: FFmpeg exited without creating an output file.',
|
||||
),
|
||||
);
|
||||
} else {
|
||||
reject(err);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -18,6 +18,7 @@ import type {
|
||||
SessionTimelinePoint,
|
||||
StatsAnkiNoteInfo,
|
||||
StatsCoverImagesData,
|
||||
StatsDuplicateLineCleanupResult,
|
||||
StatsExcludedWord,
|
||||
StreakCalendarDay,
|
||||
TrendsDashboardData,
|
||||
@@ -31,6 +32,13 @@ export type StatsTrendRange = '7d' | '30d' | '90d' | '365d' | 'all';
|
||||
export type StatsTrendGroupBy = 'day' | 'month';
|
||||
export type StatsMineMode = 'word' | 'sentence' | 'audio';
|
||||
|
||||
/** Body of `POST /api/stats/maintenance/duplicate-lines`. */
|
||||
export interface StatsDuplicateLineCleanupRequest {
|
||||
dryRun?: boolean;
|
||||
/** Null means every recorded line, whatever its age. */
|
||||
lookbackDays?: number | null;
|
||||
}
|
||||
|
||||
export interface StatsSessionKnownWordsTimelinePoint {
|
||||
linesSeen: number;
|
||||
knownWordsSeen: number;
|
||||
@@ -100,39 +108,6 @@ export interface StatsAnkiNotesInfoRequest {
|
||||
noteIds: number[];
|
||||
}
|
||||
|
||||
export interface StatsMergeAnimeRequest {
|
||||
sourceAnimeIds: number[];
|
||||
}
|
||||
|
||||
export interface StatsMoveVideoRequest {
|
||||
animeId: number;
|
||||
}
|
||||
|
||||
export interface StatsAnimeMergeRecommendation {
|
||||
recommendationId: number;
|
||||
animeIds: [number, number];
|
||||
}
|
||||
|
||||
export interface StatsAnimeMergeRecommendationsResponse {
|
||||
recommendations: StatsAnimeMergeRecommendation[];
|
||||
}
|
||||
|
||||
export interface StatsMergeAnimeResponse {
|
||||
ok: true;
|
||||
/** Library entry that owns every merged episode afterwards. */
|
||||
animeId: number;
|
||||
mergedAnimeIds: number[];
|
||||
movedVideos: number;
|
||||
}
|
||||
|
||||
export interface StatsMoveVideoResponse {
|
||||
ok: true;
|
||||
animeId: number;
|
||||
previousAnimeId: number | null;
|
||||
/** True when the previous entry was emptied by the move and removed. */
|
||||
removedPreviousAnime: boolean;
|
||||
}
|
||||
|
||||
export interface StatsOkResponse {
|
||||
ok: true;
|
||||
}
|
||||
@@ -158,6 +133,7 @@ export interface StatsJsonResponseMap {
|
||||
vocabulary: VocabularyEntry[];
|
||||
excludedWords: StatsExcludedWord[];
|
||||
setExcludedWords: StatsOkResponse;
|
||||
duplicateLineCleanup: StatsDuplicateLineCleanupResult;
|
||||
wordOccurrences: VocabularyOccurrenceEntry[];
|
||||
sentenceSearch: SentenceSearchResult[];
|
||||
kanji: KanjiEntry[];
|
||||
@@ -167,7 +143,6 @@ export interface StatsJsonResponseMap {
|
||||
mediaLibrary: MediaLibraryItem[];
|
||||
mediaDetail: MediaDetailData;
|
||||
animeLibrary: AnimeLibraryItem[];
|
||||
animeMergeRecommendations: StatsAnimeMergeRecommendationsResponse;
|
||||
animeDetail: AnimeDetailData;
|
||||
animeWords: AnimeWord[];
|
||||
animeRollups: DailyRollup[];
|
||||
@@ -176,9 +151,6 @@ export interface StatsJsonResponseMap {
|
||||
deleteSession: StatsOkResponse;
|
||||
deleteVideo: StatsOkResponse;
|
||||
deleteAnime: StatsOkResponse;
|
||||
mergeAnime: StatsMergeAnimeResponse;
|
||||
moveVideoToAnime: StatsMoveVideoResponse;
|
||||
dismissAnimeMergeRecommendation: StatsOkResponse;
|
||||
anilistSearch: StatsAnilistSearchResult[];
|
||||
knownWords: string[];
|
||||
knownWordsSummary: StatsKnownWordsSummary;
|
||||
@@ -215,6 +187,9 @@ export interface StatsHttpClient {
|
||||
getVocabulary: (limit?: number) => Promise<VocabularyEntry[]>;
|
||||
getExcludedWords: () => Promise<StatsExcludedWord[]>;
|
||||
setExcludedWords: (words: StatsExcludedWord[]) => Promise<void>;
|
||||
cleanupDuplicateLines: (
|
||||
options?: StatsDuplicateLineCleanupRequest,
|
||||
) => Promise<StatsDuplicateLineCleanupResult>;
|
||||
getWordOccurrences: (
|
||||
headword: string,
|
||||
word: string,
|
||||
@@ -236,7 +211,6 @@ export interface StatsHttpClient {
|
||||
getMediaLibrary: () => Promise<MediaLibraryItem[]>;
|
||||
getMediaDetail: (videoId: number) => Promise<MediaDetailData>;
|
||||
getAnimeLibrary: () => Promise<AnimeLibraryItem[]>;
|
||||
getAnimeMergeRecommendations: () => Promise<StatsAnimeMergeRecommendationsResponse>;
|
||||
getAnimeDetail: (animeId: number) => Promise<AnimeDetailData>;
|
||||
getAnimeWords: (animeId: number, limit?: number) => Promise<AnimeWord[]>;
|
||||
getAnimeRollups: (animeId: number, limit?: number) => Promise<DailyRollup[]>;
|
||||
@@ -259,9 +233,6 @@ export interface StatsHttpClient {
|
||||
deleteSessions: (sessionIds: number[]) => Promise<void>;
|
||||
deleteVideo: (videoId: number) => Promise<void>;
|
||||
deleteAnime: (animeId: number) => Promise<void>;
|
||||
mergeAnime: (targetAnimeId: number, sourceAnimeIds: number[]) => Promise<StatsMergeAnimeResponse>;
|
||||
moveVideoToAnime: (videoId: number, animeId: number) => Promise<StatsMoveVideoResponse>;
|
||||
dismissAnimeMergeRecommendation: (recommendationId: number) => Promise<void>;
|
||||
getKnownWords: () => Promise<string[]>;
|
||||
getKnownWordsSummary: () => Promise<StatsKnownWordsSummary>;
|
||||
getAnimeKnownWordsSummary: (animeId: number) => Promise<StatsKnownWordsSummary>;
|
||||
|
||||
@@ -82,6 +82,29 @@ export interface StatsExcludedWord {
|
||||
reading: string;
|
||||
}
|
||||
|
||||
/** One animation burst the duplicate-line cleanup found in the stats database. */
|
||||
export interface StatsDuplicateLineSample {
|
||||
videoId: number;
|
||||
videoTitle: string | null;
|
||||
text: string;
|
||||
/** Events recorded for this run, including the one that is kept. */
|
||||
frames: number;
|
||||
removedLines: number;
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
export interface StatsDuplicateLineCleanupResult {
|
||||
dryRun: boolean;
|
||||
lookbackDays: number | null;
|
||||
scannedLines: number;
|
||||
burstGroups: number;
|
||||
removedLines: number;
|
||||
removedWordOccurrences: number;
|
||||
removedKanjiOccurrences: number;
|
||||
samples: StatsDuplicateLineSample[];
|
||||
}
|
||||
|
||||
export interface StatsCoverImage {
|
||||
contentType: string;
|
||||
dataUrl: string;
|
||||
|
||||
@@ -5,45 +5,22 @@ import type { AnimeLibraryItem } from '../../types/stats';
|
||||
interface AnimeCardProps {
|
||||
anime: AnimeLibraryItem;
|
||||
onClick: () => void;
|
||||
/** While selecting, clicking the card toggles it instead of opening it. */
|
||||
selectable?: boolean;
|
||||
selected?: boolean;
|
||||
}
|
||||
|
||||
export function AnimeCard({
|
||||
anime,
|
||||
onClick,
|
||||
selectable = false,
|
||||
selected = false,
|
||||
}: AnimeCardProps) {
|
||||
export function AnimeCard({ anime, onClick }: AnimeCardProps) {
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClick}
|
||||
aria-pressed={selectable ? selected : undefined}
|
||||
className={`group bg-ctp-surface0 border rounded-lg overflow-hidden hover:shadow-lg hover:shadow-ctp-blue/10 transition-all duration-200 hover:-translate-y-1 text-left w-full ${
|
||||
selected ? 'border-ctp-blue' : 'border-ctp-surface1 hover:border-ctp-blue/50'
|
||||
}`}
|
||||
className="group bg-ctp-surface0 border border-ctp-surface1 rounded-lg overflow-hidden hover:border-ctp-blue/50 hover:shadow-lg hover:shadow-ctp-blue/10 transition-all duration-200 hover:-translate-y-1 text-left w-full"
|
||||
>
|
||||
<div className="overflow-hidden relative">
|
||||
<div className="overflow-hidden">
|
||||
<AnimeCoverImage
|
||||
animeId={anime.animeId}
|
||||
title={anime.canonicalTitle}
|
||||
coverRetryToken={anime.anilistId ?? 0}
|
||||
className="w-full aspect-[3/4] rounded-t-lg transition-transform duration-200 group-hover:scale-105"
|
||||
/>
|
||||
{selectable && (
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className={`absolute top-2 left-2 w-5 h-5 rounded border flex items-center justify-center text-xs ${
|
||||
selected
|
||||
? 'bg-ctp-blue border-ctp-blue text-ctp-base'
|
||||
: 'bg-ctp-crust/70 border-ctp-surface2 text-transparent'
|
||||
}`}
|
||||
>
|
||||
{'✓'}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="p-3">
|
||||
<div className="text-sm font-medium text-ctp-text truncate">{anime.canonicalTitle}</div>
|
||||
|
||||
@@ -25,8 +25,6 @@ interface AnimeDetailViewProps {
|
||||
* keeps showing the previous title's art.
|
||||
*/
|
||||
onAnilistRelinked?: () => void;
|
||||
/** Called after an episode is reassigned to another entry. */
|
||||
onEpisodeMoved?: () => void;
|
||||
}
|
||||
|
||||
type Range = 14 | 30 | 90;
|
||||
@@ -152,7 +150,6 @@ export function AnimeDetailView({
|
||||
onOpenEpisodeDetail,
|
||||
onAnimeDeleted,
|
||||
onAnilistRelinked,
|
||||
onEpisodeMoved,
|
||||
}: AnimeDetailViewProps) {
|
||||
const { data, loading, error, reload } = useAnimeDetail(animeId);
|
||||
const [showAnilistSelector, setShowAnilistSelector] = useState(false);
|
||||
@@ -226,13 +223,6 @@ export function AnimeDetailView({
|
||||
<AnimeOverviewStats detail={detail} knownWordsSummary={knownWordsSummary} />
|
||||
<EpisodeList
|
||||
episodes={episodes}
|
||||
animeId={animeId}
|
||||
onEpisodeMoved={(removedPreviousAnime) => {
|
||||
onEpisodeMoved?.();
|
||||
// The last episode taking the entry with it leaves nothing to show.
|
||||
if (removedPreviousAnime) onBack();
|
||||
else reload();
|
||||
}}
|
||||
onOpenDetail={onOpenEpisodeDetail ? (videoId) => onOpenEpisodeDetail(videoId) : undefined}
|
||||
/>
|
||||
<AnimeWatchChart animeId={animeId} />
|
||||
|
||||
@@ -1,232 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { Window } from 'happy-dom';
|
||||
import { act, useState } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import type { AnimeLibraryItem } from '../../types/stats';
|
||||
import { AnimeMergeDialog } from './AnimeMergeDialog';
|
||||
import { LibraryEntryPicker } from './LibraryEntryPicker';
|
||||
|
||||
interface TestWindow extends Window {
|
||||
IS_REACT_ACT_ENVIRONMENT?: boolean;
|
||||
}
|
||||
|
||||
function installDom(): () => void {
|
||||
const previousWindow = globalThis.window;
|
||||
const previousDocument = globalThis.document;
|
||||
const previousHTMLElement = globalThis.HTMLElement;
|
||||
const previousIsReactActEnvironment = (
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT;
|
||||
const window = new Window() as TestWindow;
|
||||
|
||||
Object.defineProperty(globalThis, 'window', { value: window, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: window.document, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: window.HTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
return () => {
|
||||
Object.defineProperty(globalThis, 'window', { value: previousWindow, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: previousDocument, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: previousHTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = previousIsReactActEnvironment;
|
||||
};
|
||||
}
|
||||
|
||||
function libraryItem(animeId: number, title: string): AnimeLibraryItem {
|
||||
return {
|
||||
animeId,
|
||||
canonicalTitle: title,
|
||||
anilistId: null,
|
||||
totalSessions: 1,
|
||||
totalActiveMs: 1000,
|
||||
totalCards: 0,
|
||||
totalTokensSeen: 0,
|
||||
episodeCount: 1,
|
||||
episodesTotal: null,
|
||||
lastWatchedMs: 1,
|
||||
};
|
||||
}
|
||||
|
||||
test('AnimeMergeDialog focuses its close control, closes on Escape, and restores focus', async () => {
|
||||
const uninstallDom = installDom();
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
function Harness() {
|
||||
const [open, setOpen] = useState(false);
|
||||
return (
|
||||
<>
|
||||
<button type="button" onClick={() => setOpen(true)}>
|
||||
Review merge
|
||||
</button>
|
||||
{open ? (
|
||||
<AnimeMergeDialog
|
||||
entries={[libraryItem(1, 'Show'), libraryItem(2, 'Show Season 1')]}
|
||||
onClose={() => setOpen(false)}
|
||||
onMerged={() => undefined}
|
||||
/>
|
||||
) : null}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
await act(async () => root.render(<Harness />));
|
||||
const trigger = container.querySelector('button') as HTMLButtonElement;
|
||||
trigger.focus();
|
||||
await act(async () => trigger.click());
|
||||
|
||||
assert.equal(document.activeElement?.getAttribute('aria-label'), 'Close');
|
||||
await act(async () => {
|
||||
document.dispatchEvent(new window.KeyboardEvent('keydown', { key: 'Escape' }));
|
||||
});
|
||||
assert.equal(container.querySelector('[role="dialog"]'), null);
|
||||
assert.equal(document.activeElement, trigger);
|
||||
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeMergeDialog keeps keyboard focus inside the modal', async () => {
|
||||
const uninstallDom = installDom();
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<AnimeMergeDialog
|
||||
entries={[libraryItem(1, 'Show'), libraryItem(2, 'Show Season 1')]}
|
||||
onClose={() => undefined}
|
||||
onMerged={() => undefined}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
const dialog = container.querySelector('[role="dialog"]') as HTMLElement;
|
||||
const focusable = [...dialog.querySelectorAll('button:not([disabled])')] as HTMLButtonElement[];
|
||||
const first = focusable[0];
|
||||
const last = focusable.at(-1);
|
||||
assert.ok(first);
|
||||
assert.ok(last);
|
||||
|
||||
last.focus();
|
||||
await act(async () => {
|
||||
document.dispatchEvent(new window.KeyboardEvent('keydown', { key: 'Tab' }));
|
||||
});
|
||||
assert.equal(document.activeElement, first);
|
||||
|
||||
first.focus();
|
||||
await act(async () => {
|
||||
document.dispatchEvent(new window.KeyboardEvent('keydown', { key: 'Tab', shiftKey: true }));
|
||||
});
|
||||
assert.equal(document.activeElement, last);
|
||||
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('LibraryEntryPicker focuses search, closes on Escape, and restores focus', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show'),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
function Harness() {
|
||||
const [open, setOpen] = useState(false);
|
||||
return (
|
||||
<>
|
||||
<button type="button" onClick={() => setOpen(true)}>
|
||||
Move
|
||||
</button>
|
||||
{open ? (
|
||||
<LibraryEntryPicker
|
||||
heading="Move episode"
|
||||
onSelect={() => undefined}
|
||||
onClose={() => setOpen(false)}
|
||||
/>
|
||||
) : null}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
await act(async () => root.render(<Harness />));
|
||||
const trigger = container.querySelector('button') as HTMLButtonElement;
|
||||
trigger.focus();
|
||||
await act(async () => trigger.click());
|
||||
|
||||
assert.equal(document.activeElement?.getAttribute('placeholder'), 'Search library...');
|
||||
await act(async () => {
|
||||
document.dispatchEvent(new window.KeyboardEvent('keydown', { key: 'Escape' }));
|
||||
});
|
||||
assert.equal(container.querySelector('[role="dialog"]'), null);
|
||||
assert.equal(document.activeElement, trigger);
|
||||
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
apiClient.getAnimeLibrary = original;
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('LibraryEntryPicker cannot be dismissed while a move is in flight', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show'),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
let closeCalls = 0;
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<LibraryEntryPicker
|
||||
heading="Move episode"
|
||||
busyAnimeId={1}
|
||||
onSelect={() => undefined}
|
||||
onClose={() => {
|
||||
closeCalls += 1;
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
const closeButton = container.querySelector('button[aria-label="Close"]') as HTMLButtonElement;
|
||||
assert.equal(closeButton.disabled, true);
|
||||
await act(async () => {
|
||||
closeButton.click();
|
||||
container.firstElementChild?.dispatchEvent(new window.MouseEvent('click', { bubbles: true }));
|
||||
document.dispatchEvent(new window.KeyboardEvent('keydown', { key: 'Escape' }));
|
||||
});
|
||||
assert.equal(closeCalls, 0);
|
||||
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
apiClient.getAnimeLibrary = original;
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
@@ -1,164 +0,0 @@
|
||||
import { useId, useRef, useState } from 'react';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import { formatDuration, formatNumber } from '../../lib/formatters';
|
||||
import { useModalFocus } from '../../hooks/useModalFocus';
|
||||
import { AnimeCoverImage } from './AnimeCoverImage';
|
||||
import type { AnimeLibraryItem } from '../../types/stats';
|
||||
|
||||
interface AnimeMergeDialogProps {
|
||||
entries: AnimeLibraryItem[];
|
||||
onClose: () => void;
|
||||
onMerged: (survivingAnimeId: number) => void;
|
||||
}
|
||||
|
||||
/** Biggest entry first: the one most likely to carry the right title and art. */
|
||||
function pickDefaultKeeper(entries: AnimeLibraryItem[]): number {
|
||||
const best = [...entries].sort(
|
||||
(a, b) => b.episodeCount - a.episodeCount || b.totalActiveMs - a.totalActiveMs,
|
||||
)[0];
|
||||
return best?.animeId ?? 0;
|
||||
}
|
||||
|
||||
export function AnimeMergeDialog({ entries, onClose, onMerged }: AnimeMergeDialogProps) {
|
||||
const headingId = useId();
|
||||
const dialogRef = useRef<HTMLDivElement>(null);
|
||||
const closeButtonRef = useRef<HTMLButtonElement>(null);
|
||||
const [keeperId, setKeeperId] = useState(() => pickDefaultKeeper(entries));
|
||||
const [merging, setMerging] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const totalEpisodes = entries.reduce((sum, entry) => sum + entry.episodeCount, 0);
|
||||
const totalCards = entries.reduce((sum, entry) => sum + entry.totalCards, 0);
|
||||
const totalActiveMs = entries.reduce((sum, entry) => sum + entry.totalActiveMs, 0);
|
||||
|
||||
useModalFocus({
|
||||
dialogRef,
|
||||
initialFocusRef: closeButtonRef,
|
||||
dismissDisabled: merging,
|
||||
onDismiss: onClose,
|
||||
});
|
||||
|
||||
const handleMerge = async () => {
|
||||
const sourceAnimeIds = entries
|
||||
.map((entry) => entry.animeId)
|
||||
.filter((animeId) => animeId !== keeperId);
|
||||
if (sourceAnimeIds.length === 0) return;
|
||||
setMerging(true);
|
||||
setError(null);
|
||||
try {
|
||||
const result = await apiClient.mergeAnime(keeperId, sourceAnimeIds);
|
||||
onMerged(result.animeId);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Failed to merge these entries.');
|
||||
setMerging(false);
|
||||
}
|
||||
};
|
||||
|
||||
// Dismissing mid-request would leave the caller unaware of a merge that is
|
||||
// still going to land, so the backdrop and close button are inert until it
|
||||
// resolves.
|
||||
const handleDismiss = () => {
|
||||
if (!merging) onClose();
|
||||
};
|
||||
|
||||
return (
|
||||
<div
|
||||
className="fixed inset-0 z-50 flex items-start justify-center pt-[10vh]"
|
||||
onClick={handleDismiss}
|
||||
>
|
||||
<div className="absolute inset-0 bg-ctp-crust/70 backdrop-blur-[2px]" />
|
||||
<div
|
||||
ref={dialogRef}
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby={headingId}
|
||||
className="relative bg-ctp-base border border-ctp-surface1 rounded-xl shadow-2xl w-full max-w-lg max-h-[70vh] flex flex-col animate-fade-in"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div className="p-4 border-b border-ctp-surface1">
|
||||
<div className="flex items-center justify-between">
|
||||
<h3 id={headingId} className="text-sm font-semibold text-ctp-text">
|
||||
Merge {entries.length} Library Entries
|
||||
</h3>
|
||||
<button
|
||||
ref={closeButtonRef}
|
||||
type="button"
|
||||
onClick={handleDismiss}
|
||||
disabled={merging}
|
||||
aria-label="Close"
|
||||
className="text-ctp-overlay2 hover:text-ctp-text text-lg leading-none disabled:opacity-50"
|
||||
>
|
||||
{'✕'}
|
||||
</button>
|
||||
</div>
|
||||
<p className="text-xs text-ctp-overlay2 mt-2">
|
||||
Pick the entry to keep. Every episode moves onto it and the others are removed; no
|
||||
sessions or mined cards are deleted.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-2">
|
||||
{entries.map((entry) => (
|
||||
<button
|
||||
key={entry.animeId}
|
||||
type="button"
|
||||
disabled={merging}
|
||||
aria-pressed={keeperId === entry.animeId}
|
||||
onClick={() => setKeeperId(entry.animeId)}
|
||||
className={`w-full flex items-center gap-3 p-2.5 rounded-lg transition-colors text-left disabled:opacity-50 ${
|
||||
keeperId === entry.animeId ? 'bg-ctp-surface1' : 'hover:bg-ctp-surface0'
|
||||
}`}
|
||||
>
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className={`w-4 h-4 rounded-full border shrink-0 ${
|
||||
keeperId === entry.animeId
|
||||
? 'border-ctp-blue bg-ctp-blue'
|
||||
: 'border-ctp-surface2 bg-transparent'
|
||||
}`}
|
||||
/>
|
||||
<AnimeCoverImage
|
||||
animeId={entry.animeId}
|
||||
title={entry.canonicalTitle}
|
||||
coverRetryToken={entry.anilistId ?? 0}
|
||||
className="w-10 h-14 rounded shrink-0"
|
||||
/>
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="text-sm text-ctp-text truncate">{entry.canonicalTitle}</div>
|
||||
<div className="text-xs text-ctp-overlay2 mt-0.5">
|
||||
{entry.episodeCount} episode{entry.episodeCount !== 1 ? 's' : ''} ·{' '}
|
||||
{formatDuration(entry.totalActiveMs)} · {formatNumber(entry.totalCards)} cards
|
||||
</div>
|
||||
</div>
|
||||
{keeperId === entry.animeId ? (
|
||||
<span className="text-xs text-ctp-blue shrink-0">Keep</span>
|
||||
) : null}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="p-4 border-t border-ctp-surface1 space-y-2">
|
||||
{error ? (
|
||||
<div role="alert" className="text-xs text-ctp-red">
|
||||
{error}
|
||||
</div>
|
||||
) : null}
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div className="text-xs text-ctp-overlay2">
|
||||
Result: {totalEpisodes} episode{totalEpisodes !== 1 ? 's' : ''} ·{' '}
|
||||
{formatDuration(totalActiveMs)} · {formatNumber(totalCards)} cards
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
disabled={merging}
|
||||
onClick={() => void handleMerge()}
|
||||
className="px-3 py-1.5 rounded-lg bg-ctp-blue/15 border border-ctp-blue/40 text-xs text-ctp-blue hover:bg-ctp-blue/25 transition-colors disabled:opacity-50"
|
||||
>
|
||||
{merging ? 'Merging…' : 'Merge Entries'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,416 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { Window } from 'happy-dom';
|
||||
import { act } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import type { AnimeLibraryItem, StatsMergeAnimeResponse } from '../../types/stats';
|
||||
import { AnimeTab } from './AnimeTab';
|
||||
|
||||
interface TestWindow extends Window {
|
||||
IS_REACT_ACT_ENVIRONMENT?: boolean;
|
||||
}
|
||||
|
||||
function installDom(): () => void {
|
||||
const previousWindow = globalThis.window;
|
||||
const previousDocument = globalThis.document;
|
||||
const previousHTMLElement = globalThis.HTMLElement;
|
||||
const previousIsReactActEnvironment = (
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT;
|
||||
const window = new Window() as TestWindow;
|
||||
|
||||
Object.defineProperty(globalThis, 'window', { value: window, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: window.document, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: window.HTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
return () => {
|
||||
Object.defineProperty(globalThis, 'window', { value: previousWindow, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: previousDocument, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: previousHTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = previousIsReactActEnvironment;
|
||||
};
|
||||
}
|
||||
|
||||
function libraryItem(animeId: number, title: string, episodeCount: number): AnimeLibraryItem {
|
||||
return {
|
||||
animeId,
|
||||
canonicalTitle: title,
|
||||
anilistId: null,
|
||||
totalSessions: 1,
|
||||
totalActiveMs: 1000,
|
||||
totalCards: 1,
|
||||
totalTokensSeen: 0,
|
||||
episodeCount,
|
||||
episodesTotal: null,
|
||||
lastWatchedMs: animeId,
|
||||
};
|
||||
}
|
||||
|
||||
function findButton(container: Element, label: string): HTMLElement {
|
||||
const match = [...container.querySelectorAll('button')].find((button) =>
|
||||
(button.textContent ?? '').includes(label),
|
||||
);
|
||||
assert.ok(match, `expected a "${label}" button`);
|
||||
return match as unknown as HTMLElement;
|
||||
}
|
||||
|
||||
/** Library cards only expose aria-pressed while selection mode is on. */
|
||||
function cardButtons(container: Element): HTMLButtonElement[] {
|
||||
return [...container.querySelectorAll('button[aria-pressed]')] as unknown as HTMLButtonElement[];
|
||||
}
|
||||
|
||||
function mergeButton(container: Element): HTMLButtonElement {
|
||||
const match = [...container.querySelectorAll('button')].find(
|
||||
(button) => (button.textContent ?? '').trim() === 'Merge Selected',
|
||||
);
|
||||
assert.ok(match, 'expected a "Merge Selected" button');
|
||||
return match as unknown as HTMLButtonElement;
|
||||
}
|
||||
|
||||
test('AnimeTab merges the selected duplicate entries into the chosen keeper', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
mergeAnime: apiClient.mergeAnime,
|
||||
};
|
||||
|
||||
// Two cards for one show, the split this feature exists to undo.
|
||||
let entries = [libraryItem(1, 'Show', 2), libraryItem(2, 'Show Season 1', 1)];
|
||||
let libraryFetches = 0;
|
||||
let mergeCall: { targetAnimeId: number; sourceAnimeIds: number[] } | null = null;
|
||||
|
||||
apiClient.getAnimeLibrary = (async () => {
|
||||
libraryFetches += 1;
|
||||
return entries;
|
||||
}) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.mergeAnime = (async (targetAnimeId: number, sourceAnimeIds: number[]) => {
|
||||
mergeCall = { targetAnimeId, sourceAnimeIds };
|
||||
entries = [libraryItem(1, 'Show', 3)];
|
||||
return {
|
||||
ok: true,
|
||||
animeId: targetAnimeId,
|
||||
mergedAnimeIds: sourceAnimeIds,
|
||||
movedVideos: 1,
|
||||
} satisfies StatsMergeAnimeResponse;
|
||||
}) as typeof apiClient.mergeAnime;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(<AnimeTab />);
|
||||
});
|
||||
assert.equal(libraryFetches, 1);
|
||||
|
||||
await act(async () => {
|
||||
findButton(container, 'Select').click();
|
||||
});
|
||||
// Nothing to merge until at least two entries are picked.
|
||||
assert.equal(mergeButton(container).disabled, true);
|
||||
|
||||
// Sorted by last watched, so the season-tagged duplicate comes first.
|
||||
const cards = cardButtons(container);
|
||||
assert.equal(cards.length, 2);
|
||||
assert.match(cards[0]?.textContent ?? '', /Show Season 1/);
|
||||
|
||||
await act(async () => {
|
||||
cards[0]?.click();
|
||||
});
|
||||
assert.equal(mergeButton(container).disabled, true);
|
||||
|
||||
await act(async () => {
|
||||
cardButtons(container)[1]?.click();
|
||||
});
|
||||
assert.equal(mergeButton(container).disabled, false);
|
||||
|
||||
await act(async () => {
|
||||
mergeButton(container).click();
|
||||
});
|
||||
// The dialog defaults to the entry with the most episodes.
|
||||
assert.match(container.textContent ?? '', /Merge 2 Library Entries/);
|
||||
|
||||
await act(async () => {
|
||||
findButton(container, 'Merge Entries').click();
|
||||
});
|
||||
|
||||
assert.deepEqual(mergeCall, { targetAnimeId: 1, sourceAnimeIds: [2] });
|
||||
assert.equal(libraryFetches, 2);
|
||||
// Selection mode closes and the grid is back to a single card.
|
||||
assert.doesNotMatch(container.textContent ?? '', /Merge 2 Library Entries/);
|
||||
assert.doesNotMatch(container.textContent ?? '', /Show Season 1/);
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab keeps a suggested duplicate visible until it is reviewed and merged', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
dismissAnimeMergeRecommendation: apiClient.dismissAnimeMergeRecommendation,
|
||||
mergeAnime: apiClient.mergeAnime,
|
||||
};
|
||||
|
||||
let entries = [libraryItem(1, 'Show', 2), libraryItem(2, 'Show Season 1', 1)];
|
||||
let recommendations = [{ recommendationId: 41, animeIds: [1, 2] }];
|
||||
let mergeCall: { targetAnimeId: number; sourceAnimeIds: number[] } | null = null;
|
||||
|
||||
apiClient.getAnimeLibrary = (async () => entries) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => ({
|
||||
recommendations,
|
||||
})) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
apiClient.dismissAnimeMergeRecommendation = (async () =>
|
||||
undefined) as typeof apiClient.dismissAnimeMergeRecommendation;
|
||||
apiClient.mergeAnime = (async (targetAnimeId: number, sourceAnimeIds: number[]) => {
|
||||
mergeCall = { targetAnimeId, sourceAnimeIds };
|
||||
entries = [libraryItem(targetAnimeId, 'Show', 3)];
|
||||
recommendations = [];
|
||||
return {
|
||||
ok: true,
|
||||
animeId: targetAnimeId,
|
||||
mergedAnimeIds: sourceAnimeIds,
|
||||
movedVideos: 1,
|
||||
} satisfies StatsMergeAnimeResponse;
|
||||
}) as typeof apiClient.mergeAnime;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(<AnimeTab />);
|
||||
});
|
||||
|
||||
assert.match(container.textContent ?? '', /Possible duplicate/);
|
||||
assert.match(container.textContent ?? '', /Show/);
|
||||
assert.match(container.textContent ?? '', /Show Season 1/);
|
||||
|
||||
await act(async () => {
|
||||
findButton(container, 'Review merge').click();
|
||||
});
|
||||
assert.match(container.textContent ?? '', /Merge 2 Library Entries/);
|
||||
|
||||
const keeper = [...container.querySelectorAll('button[aria-pressed]')].find((button) =>
|
||||
(button.textContent ?? '').includes('Show Season 1'),
|
||||
) as HTMLButtonElement | undefined;
|
||||
assert.ok(keeper);
|
||||
await act(async () => {
|
||||
keeper.click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(container, 'Merge Entries').click();
|
||||
});
|
||||
|
||||
assert.deepEqual(mergeCall, { targetAnimeId: 2, sourceAnimeIds: [1] });
|
||||
assert.doesNotMatch(container.textContent ?? '', /Possible duplicate/);
|
||||
assert.doesNotMatch(container.textContent ?? '', /Show Season 1/);
|
||||
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab dismisses a false-positive duplicate recommendation', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
dismissAnimeMergeRecommendation: apiClient.dismissAnimeMergeRecommendation,
|
||||
};
|
||||
let dismissedId: number | null = null;
|
||||
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show', 2),
|
||||
libraryItem(2, 'Different Show', 1),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => ({
|
||||
recommendations: [{ recommendationId: 73, animeIds: [1, 2] }],
|
||||
})) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
apiClient.dismissAnimeMergeRecommendation = (async (recommendationId: number) => {
|
||||
dismissedId = recommendationId;
|
||||
}) as typeof apiClient.dismissAnimeMergeRecommendation;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => root.render(<AnimeTab />));
|
||||
|
||||
await act(async () => {
|
||||
findButton(container, 'Not duplicates').click();
|
||||
});
|
||||
|
||||
assert.equal(dismissedId, 73);
|
||||
assert.doesNotMatch(container.textContent ?? '', /Possible duplicate/);
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab refreshes the library and recommendations when the window regains focus', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
};
|
||||
let entries = [libraryItem(1, 'Show', 2), libraryItem(2, 'Show Season 1', 1)];
|
||||
let libraryFetches = 0;
|
||||
let recommendationFetches = 0;
|
||||
apiClient.getAnimeLibrary = (async () => {
|
||||
libraryFetches += 1;
|
||||
return entries;
|
||||
}) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => {
|
||||
recommendationFetches += 1;
|
||||
return { recommendations: [] };
|
||||
}) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => root.render(<AnimeTab />));
|
||||
|
||||
entries = [libraryItem(1, 'Show', 3)];
|
||||
await act(async () => {
|
||||
window.dispatchEvent(new window.Event('focus'));
|
||||
});
|
||||
|
||||
assert.equal(libraryFetches, 2);
|
||||
assert.equal(recommendationFetches, 2);
|
||||
assert.doesNotMatch(container.textContent ?? '', /Show Season 1/);
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab keeps a recommendation visible through a transient refresh failure', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
};
|
||||
let failRecommendations = false;
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show', 2),
|
||||
libraryItem(2, 'Show Season 1', 1),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => {
|
||||
if (failRecommendations) throw new Error('temporary failure');
|
||||
return { recommendations: [{ recommendationId: 41, animeIds: [1, 2] }] };
|
||||
}) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => root.render(<AnimeTab />));
|
||||
assert.match(container.textContent ?? '', /Possible duplicate/);
|
||||
|
||||
failRecommendations = true;
|
||||
await act(async () => window.dispatchEvent(new window.Event('focus')));
|
||||
|
||||
assert.match(container.textContent ?? '', /Possible duplicate/);
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab retains a recommendation and reports a failed dismissal', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
dismissAnimeMergeRecommendation: apiClient.dismissAnimeMergeRecommendation,
|
||||
};
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show', 2),
|
||||
libraryItem(2, 'Show Season 1', 1),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => ({
|
||||
recommendations: [{ recommendationId: 41, animeIds: [1, 2] }],
|
||||
})) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
apiClient.dismissAnimeMergeRecommendation = (async () => {
|
||||
throw new Error('offline');
|
||||
}) as typeof apiClient.dismissAnimeMergeRecommendation;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => root.render(<AnimeTab />));
|
||||
|
||||
await act(async () => findButton(container, 'Not duplicates').click());
|
||||
|
||||
assert.match(container.textContent ?? '', /Possible duplicate/);
|
||||
assert.match(container.textContent ?? '', /Could not dismiss this suggestion/);
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
|
||||
test('AnimeTab keeps an open recommendation review stable during background refresh', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
getAnimeMergeRecommendations: apiClient.getAnimeMergeRecommendations,
|
||||
};
|
||||
let recommendations = [{ recommendationId: 41, animeIds: [1, 2] as [number, number] }];
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Show', 2),
|
||||
libraryItem(2, 'Show Season 1', 1),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.getAnimeMergeRecommendations = (async () => ({
|
||||
recommendations,
|
||||
})) as typeof apiClient.getAnimeMergeRecommendations;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
await act(async () => root.render(<AnimeTab />));
|
||||
await act(async () => findButton(container, 'Review merge').click());
|
||||
assert.match(container.textContent ?? '', /Merge 2 Library Entries/);
|
||||
|
||||
recommendations = [];
|
||||
await act(async () => window.dispatchEvent(new window.Event('focus')));
|
||||
|
||||
assert.match(container.textContent ?? '', /Merge 2 Library Entries/);
|
||||
assert.match(container.textContent ?? '', /Show Season 1/);
|
||||
await act(async () => root.unmount());
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
@@ -9,8 +9,6 @@ import {
|
||||
} from '../../lib/library-card-size';
|
||||
import { AnimeCard } from './AnimeCard';
|
||||
import { AnimeDetailView } from './AnimeDetailView';
|
||||
import { AnimeMergeDialog } from './AnimeMergeDialog';
|
||||
import { DuplicateReviewStrip } from './DuplicateReviewStrip';
|
||||
|
||||
type SortKey = 'lastWatched' | 'watchTime' | 'cards' | 'episodes';
|
||||
|
||||
@@ -55,17 +53,7 @@ export function AnimeTab({
|
||||
onNavigateToWord,
|
||||
onOpenEpisodeDetail,
|
||||
}: AnimeTabProps) {
|
||||
const {
|
||||
anime,
|
||||
loading,
|
||||
error,
|
||||
reload,
|
||||
recommendations,
|
||||
dismissRecommendation,
|
||||
dismissingRecommendationId,
|
||||
recommendationActionError,
|
||||
clearRecommendation,
|
||||
} = useAnimeLibrary();
|
||||
const { anime, loading, error, reload } = useAnimeLibrary();
|
||||
const [search, setSearch] = useState('');
|
||||
const [sortKey, setSortKey] = useState<SortKey>('lastWatched');
|
||||
const [cardSize, setCardSize] = useState<LibraryCardSize>(() =>
|
||||
@@ -74,23 +62,6 @@ export function AnimeTab({
|
||||
),
|
||||
);
|
||||
const [selectedAnimeId, setSelectedAnimeId] = useState<number | null>(null);
|
||||
const [selectionMode, setSelectionMode] = useState(false);
|
||||
const [checkedAnimeIds, setCheckedAnimeIds] = useState<number[]>([]);
|
||||
const [showMergeDialog, setShowMergeDialog] = useState(false);
|
||||
const [reviewRecommendationId, setReviewRecommendationId] = useState<number | null>(null);
|
||||
const [reviewAnimeIds, setReviewAnimeIds] = useState<[number, number] | null>(null);
|
||||
|
||||
function toggleChecked(animeId: number): void {
|
||||
setCheckedAnimeIds((ids) =>
|
||||
ids.includes(animeId) ? ids.filter((id) => id !== animeId) : [...ids, animeId],
|
||||
);
|
||||
}
|
||||
|
||||
function exitSelectionMode(): void {
|
||||
setSelectionMode(false);
|
||||
setCheckedAnimeIds([]);
|
||||
setShowMergeDialog(false);
|
||||
}
|
||||
|
||||
function handleCardSizeChange(size: LibraryCardSize): void {
|
||||
setCardSize(size);
|
||||
@@ -115,22 +86,6 @@ export function AnimeTab({
|
||||
}, [anime, search, sortKey]);
|
||||
|
||||
const totalMs = anime.reduce((sum, a) => sum + a.totalActiveMs, 0);
|
||||
const checkedEntries = checkedAnimeIds
|
||||
.map((animeId) => anime.find((entry) => entry.animeId === animeId))
|
||||
.filter((entry): entry is (typeof anime)[number] => entry !== undefined);
|
||||
const hydratedRecommendations = recommendations
|
||||
.map((recommendation) => ({
|
||||
...recommendation,
|
||||
entries: recommendation.animeIds
|
||||
.map((animeId) => anime.find((entry) => entry.animeId === animeId))
|
||||
.filter((entry): entry is (typeof anime)[number] => entry !== undefined),
|
||||
}))
|
||||
.filter((recommendation) => recommendation.entries.length >= 2);
|
||||
const activeRecommendation = hydratedRecommendations[0] ?? null;
|
||||
const reviewEntries = (reviewAnimeIds ?? [])
|
||||
.map((animeId) => anime.find((entry) => entry.animeId === animeId))
|
||||
.filter((entry): entry is (typeof anime)[number] => entry !== undefined);
|
||||
const mergeEntries = reviewRecommendationId !== null ? reviewEntries : checkedEntries;
|
||||
|
||||
if (selectedAnimeId !== null) {
|
||||
return (
|
||||
@@ -145,7 +100,6 @@ export function AnimeTab({
|
||||
}
|
||||
onAnimeDeleted={reload}
|
||||
onAnilistRelinked={reload}
|
||||
onEpisodeMoved={reload}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -189,57 +143,11 @@ export function AnimeTab({
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => (selectionMode ? exitSelectionMode() : setSelectionMode(true))}
|
||||
title="Select several entries to merge them into one"
|
||||
className={`px-2 py-2 rounded-lg border text-xs shrink-0 transition-colors ${
|
||||
selectionMode
|
||||
? 'bg-ctp-blue/15 border-ctp-blue/40 text-ctp-blue'
|
||||
: 'bg-ctp-surface0 border-ctp-surface1 text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
{selectionMode ? 'Cancel' : 'Select'}
|
||||
</button>
|
||||
<div className="text-xs text-ctp-overlay2 shrink-0">
|
||||
{filtered.length} titles · {formatDuration(totalMs)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{activeRecommendation ? (
|
||||
<DuplicateReviewStrip
|
||||
entries={activeRecommendation.entries}
|
||||
current={1}
|
||||
total={hydratedRecommendations.length}
|
||||
dismissing={dismissingRecommendationId === activeRecommendation.recommendationId}
|
||||
error={recommendationActionError}
|
||||
onReview={() => {
|
||||
setReviewRecommendationId(activeRecommendation.recommendationId);
|
||||
setReviewAnimeIds(activeRecommendation.animeIds);
|
||||
setShowMergeDialog(true);
|
||||
}}
|
||||
onDismiss={() => void dismissRecommendation(activeRecommendation.recommendationId)}
|
||||
/>
|
||||
) : null}
|
||||
|
||||
{selectionMode && (
|
||||
<div className="flex items-center justify-between gap-3 bg-ctp-surface0 border border-ctp-surface1 rounded-lg px-3 py-2">
|
||||
<div className="text-xs text-ctp-overlay2">
|
||||
{checkedEntries.length === 0
|
||||
? 'Pick the duplicate entries to combine'
|
||||
: `${checkedEntries.length} selected`}
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
disabled={checkedEntries.length < 2}
|
||||
onClick={() => setShowMergeDialog(true)}
|
||||
className="px-3 py-1.5 rounded-lg bg-ctp-blue/15 border border-ctp-blue/40 text-xs text-ctp-blue hover:bg-ctp-blue/25 transition-colors disabled:opacity-40 disabled:cursor-not-allowed"
|
||||
>
|
||||
Merge Selected
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{filtered.length === 0 ? (
|
||||
<div className="text-sm text-ctp-overlay2 p-4">No titles found</div>
|
||||
) : (
|
||||
@@ -248,35 +156,11 @@ export function AnimeTab({
|
||||
<AnimeCard
|
||||
key={item.animeId}
|
||||
anime={item}
|
||||
selectable={selectionMode}
|
||||
selected={checkedAnimeIds.includes(item.animeId)}
|
||||
onClick={() =>
|
||||
selectionMode ? toggleChecked(item.animeId) : setSelectedAnimeId(item.animeId)
|
||||
}
|
||||
onClick={() => setSelectedAnimeId(item.animeId)}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{showMergeDialog && mergeEntries.length >= 2 && (
|
||||
<AnimeMergeDialog
|
||||
entries={mergeEntries}
|
||||
onClose={() => {
|
||||
setShowMergeDialog(false);
|
||||
setReviewRecommendationId(null);
|
||||
setReviewAnimeIds(null);
|
||||
}}
|
||||
onMerged={() => {
|
||||
if (reviewRecommendationId !== null) {
|
||||
clearRecommendation(reviewRecommendationId);
|
||||
}
|
||||
exitSelectionMode();
|
||||
setReviewRecommendationId(null);
|
||||
setReviewAnimeIds(null);
|
||||
reload();
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
import type { AnimeLibraryItem } from '../../types/stats';
|
||||
|
||||
interface DuplicateReviewStripProps {
|
||||
entries: AnimeLibraryItem[];
|
||||
current: number;
|
||||
total: number;
|
||||
dismissing: boolean;
|
||||
error?: string | null;
|
||||
onReview: () => void;
|
||||
onDismiss: () => void;
|
||||
}
|
||||
|
||||
export function DuplicateReviewStrip({
|
||||
entries,
|
||||
current,
|
||||
total,
|
||||
dismissing,
|
||||
error = null,
|
||||
onReview,
|
||||
onDismiss,
|
||||
}: DuplicateReviewStripProps) {
|
||||
return (
|
||||
<aside
|
||||
aria-label="Possible duplicate library entries"
|
||||
className="relative overflow-hidden rounded-lg border border-ctp-yellow/25 bg-ctp-yellow/[0.06] px-3 py-2.5"
|
||||
>
|
||||
<div className="absolute inset-y-0 left-0 w-0.5 bg-ctp-yellow/70" aria-hidden="true" />
|
||||
<div className="flex items-center gap-3">
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-xs font-medium text-ctp-yellow">Possible duplicate</span>
|
||||
{total > 1 ? (
|
||||
<span className="text-[10px] tabular-nums text-ctp-overlay1">
|
||||
{current} of {total}
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
<p className="mt-0.5 truncate text-xs text-ctp-subtext0">
|
||||
{entries.map((entry) => entry.canonicalTitle).join(' · ')}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
disabled={dismissing}
|
||||
onClick={onDismiss}
|
||||
className="shrink-0 rounded-md px-2.5 py-1.5 text-xs text-ctp-overlay2 transition-colors hover:bg-ctp-surface0 hover:text-ctp-text disabled:opacity-50"
|
||||
>
|
||||
{dismissing ? 'Dismissing…' : 'Not duplicates'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={dismissing}
|
||||
onClick={onReview}
|
||||
className="shrink-0 rounded-md border border-ctp-yellow/35 bg-ctp-yellow/10 px-2.5 py-1.5 text-xs font-medium text-ctp-yellow transition-colors hover:bg-ctp-yellow/20 disabled:opacity-50"
|
||||
>
|
||||
Review merge
|
||||
</button>
|
||||
</div>
|
||||
{error ? (
|
||||
<p role="alert" className="mt-1.5 text-xs text-ctp-red">
|
||||
{error}
|
||||
</p>
|
||||
) : null}
|
||||
</aside>
|
||||
);
|
||||
}
|
||||
@@ -4,39 +4,21 @@ import { apiClient } from '../../lib/api-client';
|
||||
import { confirmEpisodeDelete } from '../../lib/delete-confirm';
|
||||
import { buildLookupRateDisplay } from '../../lib/yomitan-lookup';
|
||||
import { EpisodeDetail } from './EpisodeDetail';
|
||||
import { LibraryEntryPicker } from './LibraryEntryPicker';
|
||||
import type { AnimeEpisode } from '../../types/stats';
|
||||
|
||||
/**
|
||||
* Row actions that only appear on hover. Keyboard focus and pointers with no
|
||||
* hover (touch) reveal them too, otherwise those users cannot reach the button
|
||||
* at all.
|
||||
*/
|
||||
const HOVER_REVEALED =
|
||||
'opacity-0 group-hover:opacity-100 focus-visible:opacity-100 [@media(hover:none)]:opacity-100';
|
||||
|
||||
interface EpisodeListProps {
|
||||
episodes: AnimeEpisode[];
|
||||
/** Entry these episodes currently belong to; excluded from the move picker. */
|
||||
animeId?: number;
|
||||
onEpisodeDeleted?: () => void;
|
||||
/** Fires after an episode is reassigned, so the caller can refetch. */
|
||||
onEpisodeMoved?: (removedPreviousAnime: boolean) => void;
|
||||
onOpenDetail?: (videoId: number) => void;
|
||||
}
|
||||
|
||||
export function EpisodeList({
|
||||
episodes: initialEpisodes,
|
||||
animeId,
|
||||
onEpisodeDeleted,
|
||||
onEpisodeMoved,
|
||||
onOpenDetail,
|
||||
}: EpisodeListProps) {
|
||||
const [expandedVideoId, setExpandedVideoId] = useState<number | null>(null);
|
||||
const [episodes, setEpisodes] = useState(initialEpisodes);
|
||||
const [movingEpisode, setMovingEpisode] = useState<AnimeEpisode | null>(null);
|
||||
const [moveTargetId, setMoveTargetId] = useState<number | null>(null);
|
||||
const [moveError, setMoveError] = useState<string | null>(null);
|
||||
|
||||
if (episodes.length === 0) return null;
|
||||
|
||||
@@ -69,22 +51,6 @@ export function EpisodeList({
|
||||
onEpisodeDeleted?.();
|
||||
};
|
||||
|
||||
const handleMoveEpisode = async (videoId: number, targetAnimeId: number) => {
|
||||
setMoveTargetId(targetAnimeId);
|
||||
setMoveError(null);
|
||||
try {
|
||||
const result = await apiClient.moveVideoToAnime(videoId, targetAnimeId);
|
||||
setEpisodes((prev) => prev.filter((ep) => ep.videoId !== videoId));
|
||||
if (expandedVideoId === videoId) setExpandedVideoId(null);
|
||||
setMovingEpisode(null);
|
||||
onEpisodeMoved?.(result.removedPreviousAnime);
|
||||
} catch (err) {
|
||||
setMoveError(err instanceof Error ? err.message : 'Failed to move this episode.');
|
||||
} finally {
|
||||
setMoveTargetId(null);
|
||||
}
|
||||
};
|
||||
|
||||
const watchedCount = episodes.filter((ep) => ep.watched).length;
|
||||
|
||||
return (
|
||||
@@ -198,28 +164,14 @@ export function EpisodeList({
|
||||
>
|
||||
{'\u2713'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
setMoveError(null);
|
||||
setMovingEpisode(ep);
|
||||
}}
|
||||
className={`w-5 h-5 rounded border border-ctp-surface2 text-transparent hover:border-ctp-blue/50 hover:text-ctp-blue focus-visible:text-ctp-blue hover:bg-ctp-blue/10 transition-colors text-xs flex items-center justify-center ${HOVER_REVEALED}`}
|
||||
title="Move to another library entry"
|
||||
aria-label="Move to another library entry"
|
||||
>
|
||||
{'\u2192'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
void handleDeleteEpisode(ep.videoId, ep.canonicalTitle);
|
||||
}}
|
||||
className={`w-5 h-5 rounded border border-ctp-surface2 text-transparent hover:border-ctp-red/50 hover:text-ctp-red focus-visible:text-ctp-red hover:bg-ctp-red/10 transition-colors text-xs flex items-center justify-center ${HOVER_REVEALED}`}
|
||||
className="w-5 h-5 rounded border border-ctp-surface2 text-transparent hover:border-ctp-red/50 hover:text-ctp-red hover:bg-ctp-red/10 transition-colors opacity-0 group-hover:opacity-100 text-xs flex items-center justify-center"
|
||||
title="Delete episode"
|
||||
aria-label="Delete episode"
|
||||
>
|
||||
{'\u2715'}
|
||||
</button>
|
||||
@@ -239,19 +191,6 @@ export function EpisodeList({
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{movingEpisode && (
|
||||
<LibraryEntryPicker
|
||||
heading={`Move "${movingEpisode.canonicalTitle}" To`}
|
||||
excludeAnimeIds={animeId != null ? [animeId] : []}
|
||||
busyAnimeId={moveTargetId}
|
||||
error={moveError}
|
||||
onSelect={(entry) => void handleMoveEpisode(movingEpisode.videoId, entry.animeId)}
|
||||
onClose={() => {
|
||||
setMovingEpisode(null);
|
||||
setMoveError(null);
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,159 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { Window } from 'happy-dom';
|
||||
import { act } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import type { AnimeEpisode, AnimeLibraryItem, StatsMoveVideoResponse } from '../../types/stats';
|
||||
import { EpisodeList } from './EpisodeList';
|
||||
|
||||
interface TestWindow extends Window {
|
||||
IS_REACT_ACT_ENVIRONMENT?: boolean;
|
||||
}
|
||||
|
||||
function installDom(): () => void {
|
||||
const previousWindow = globalThis.window;
|
||||
const previousDocument = globalThis.document;
|
||||
const previousHTMLElement = globalThis.HTMLElement;
|
||||
const previousIsReactActEnvironment = (
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT;
|
||||
const window = new Window() as TestWindow;
|
||||
|
||||
Object.defineProperty(globalThis, 'window', { value: window, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: window.document, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: window.HTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
return () => {
|
||||
Object.defineProperty(globalThis, 'window', { value: previousWindow, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: previousDocument, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: previousHTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = previousIsReactActEnvironment;
|
||||
};
|
||||
}
|
||||
|
||||
function episode(videoId: number, title: string): AnimeEpisode {
|
||||
return {
|
||||
videoId,
|
||||
episode: videoId,
|
||||
season: null,
|
||||
durationMs: 1_440_000,
|
||||
endedMediaMs: null,
|
||||
watched: 0,
|
||||
canonicalTitle: title,
|
||||
totalSessions: 1,
|
||||
totalActiveMs: 1000,
|
||||
totalCards: 0,
|
||||
totalTokensSeen: 0,
|
||||
totalYomitanLookupCount: 0,
|
||||
lastWatchedMs: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function libraryItem(animeId: number, title: string): AnimeLibraryItem {
|
||||
return {
|
||||
animeId,
|
||||
canonicalTitle: title,
|
||||
anilistId: null,
|
||||
totalSessions: 1,
|
||||
totalActiveMs: 1000,
|
||||
totalCards: 0,
|
||||
totalTokensSeen: 0,
|
||||
episodeCount: 1,
|
||||
episodesTotal: null,
|
||||
lastWatchedMs: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function findButtonByTitle(container: Element, title: string): HTMLElement {
|
||||
const match = [...container.querySelectorAll('button')].find(
|
||||
(button) => button.getAttribute('title') === title,
|
||||
);
|
||||
assert.ok(match, `expected a button titled "${title}"`);
|
||||
return match as unknown as HTMLElement;
|
||||
}
|
||||
|
||||
function findButtonByText(container: Element, text: string): HTMLElement {
|
||||
const match = [...container.querySelectorAll('button')].find((button) =>
|
||||
(button.textContent ?? '').includes(text),
|
||||
);
|
||||
assert.ok(match, `expected a "${text}" button`);
|
||||
return match as unknown as HTMLElement;
|
||||
}
|
||||
|
||||
test('EpisodeList moves an episode to the library entry picked in the dialog', async () => {
|
||||
const uninstallDom = installDom();
|
||||
const original = {
|
||||
getAnimeLibrary: apiClient.getAnimeLibrary,
|
||||
moveVideoToAnime: apiClient.moveVideoToAnime,
|
||||
};
|
||||
|
||||
let moveCall: { videoId: number; animeId: number } | null = null;
|
||||
let movedResult: boolean | null = null;
|
||||
|
||||
apiClient.getAnimeLibrary = (async () => [
|
||||
libraryItem(1, 'Current Entry'),
|
||||
libraryItem(2, 'Real Series'),
|
||||
]) as typeof apiClient.getAnimeLibrary;
|
||||
apiClient.moveVideoToAnime = (async (videoId: number, animeId: number) => {
|
||||
moveCall = { videoId, animeId };
|
||||
return {
|
||||
ok: true,
|
||||
animeId,
|
||||
previousAnimeId: 1,
|
||||
removedPreviousAnime: true,
|
||||
} satisfies StatsMoveVideoResponse;
|
||||
}) as typeof apiClient.moveVideoToAnime;
|
||||
|
||||
try {
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<EpisodeList
|
||||
episodes={[episode(5, 'Stray Episode')]}
|
||||
animeId={1}
|
||||
onEpisodeMoved={(removedPreviousAnime) => {
|
||||
movedResult = removedPreviousAnime;
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
await act(async () => {
|
||||
findButtonByTitle(container, 'Move to another library entry').click();
|
||||
});
|
||||
assert.match(container.textContent ?? '', /Move "Stray Episode" To/);
|
||||
// The entry the episode already belongs to is not offered as a target.
|
||||
assert.doesNotMatch(container.textContent ?? '', /Current Entry/);
|
||||
|
||||
await act(async () => {
|
||||
findButtonByText(container, 'Real Series').click();
|
||||
});
|
||||
|
||||
assert.deepEqual(moveCall, { videoId: 5, animeId: 2 });
|
||||
assert.equal(movedResult, true);
|
||||
// The row leaves this entry's list and the picker closes.
|
||||
assert.doesNotMatch(container.textContent ?? '', /Stray Episode/);
|
||||
|
||||
await act(async () => {
|
||||
root.unmount();
|
||||
});
|
||||
} finally {
|
||||
Object.assign(apiClient, original);
|
||||
uninstallDom();
|
||||
}
|
||||
});
|
||||
@@ -1,168 +0,0 @@
|
||||
import { useEffect, useId, useMemo, useRef, useState } from 'react';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import { formatDuration } from '../../lib/formatters';
|
||||
import { useModalFocus } from '../../hooks/useModalFocus';
|
||||
import { AnimeCoverImage } from './AnimeCoverImage';
|
||||
import type { AnimeLibraryItem } from '../../types/stats';
|
||||
|
||||
interface LibraryEntryPickerProps {
|
||||
heading: string;
|
||||
/** Entries that cannot be picked, typically the one being moved away from. */
|
||||
excludeAnimeIds?: number[];
|
||||
initialQuery?: string;
|
||||
busyAnimeId?: number | null;
|
||||
error?: string | null;
|
||||
onSelect: (entry: AnimeLibraryItem) => void;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
export function LibraryEntryPicker({
|
||||
heading,
|
||||
excludeAnimeIds = [],
|
||||
initialQuery = '',
|
||||
busyAnimeId = null,
|
||||
error = null,
|
||||
onSelect,
|
||||
onClose,
|
||||
}: LibraryEntryPickerProps) {
|
||||
const [entries, setEntries] = useState<AnimeLibraryItem[] | null>(null);
|
||||
const [loadFailed, setLoadFailed] = useState(false);
|
||||
const [query, setQuery] = useState(initialQuery);
|
||||
const inputRef = useRef<HTMLInputElement>(null);
|
||||
const dialogRef = useRef<HTMLDivElement>(null);
|
||||
const headingId = useId();
|
||||
const searchId = useId();
|
||||
const busy = busyAnimeId !== null;
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
apiClient
|
||||
.getAnimeLibrary()
|
||||
.then((data) => {
|
||||
if (!cancelled) setEntries(data);
|
||||
})
|
||||
.catch(() => {
|
||||
// Distinct from an empty library: telling the user "no other titles"
|
||||
// when the request failed hides a retryable error.
|
||||
if (cancelled) return;
|
||||
setEntries([]);
|
||||
setLoadFailed(true);
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, []);
|
||||
|
||||
useModalFocus({
|
||||
dialogRef,
|
||||
initialFocusRef: inputRef,
|
||||
dismissDisabled: busy,
|
||||
onDismiss: onClose,
|
||||
});
|
||||
|
||||
const handleDismiss = () => {
|
||||
if (!busy) onClose();
|
||||
};
|
||||
|
||||
const excluded = useMemo(() => new Set(excludeAnimeIds), [excludeAnimeIds]);
|
||||
const visible = useMemo(() => {
|
||||
const term = query.trim().toLowerCase();
|
||||
return (entries ?? [])
|
||||
.filter((entry) => !excluded.has(entry.animeId))
|
||||
.filter((entry) => !term || entry.canonicalTitle.toLowerCase().includes(term))
|
||||
.sort((a, b) => b.lastWatchedMs - a.lastWatchedMs);
|
||||
}, [entries, excluded, query]);
|
||||
|
||||
return (
|
||||
<div
|
||||
className="fixed inset-0 z-50 flex items-start justify-center pt-[10vh]"
|
||||
onClick={handleDismiss}
|
||||
>
|
||||
<div className="absolute inset-0 bg-ctp-crust/70 backdrop-blur-[2px]" />
|
||||
<div
|
||||
ref={dialogRef}
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby={headingId}
|
||||
className="relative bg-ctp-base border border-ctp-surface1 rounded-xl shadow-2xl w-full max-w-lg max-h-[70vh] flex flex-col animate-fade-in"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div className="p-4 border-b border-ctp-surface1">
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
<h3 id={headingId} className="text-sm font-semibold text-ctp-text">
|
||||
{heading}
|
||||
</h3>
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleDismiss}
|
||||
disabled={busy}
|
||||
aria-label="Close"
|
||||
className="text-ctp-overlay2 hover:text-ctp-text text-lg leading-none disabled:opacity-50"
|
||||
>
|
||||
{'✕'}
|
||||
</button>
|
||||
</div>
|
||||
<label htmlFor={searchId} className="sr-only">
|
||||
Search library
|
||||
</label>
|
||||
<input
|
||||
ref={inputRef}
|
||||
id={searchId}
|
||||
type="text"
|
||||
value={query}
|
||||
onChange={(e) => setQuery(e.target.value)}
|
||||
placeholder="Search library..."
|
||||
className="w-full bg-ctp-surface0 border border-ctp-surface1 rounded-lg px-3 py-2 text-sm text-ctp-text placeholder:text-ctp-overlay2 focus:outline-none focus:border-ctp-blue"
|
||||
/>
|
||||
{error ? (
|
||||
<div role="alert" className="text-xs text-ctp-red mt-2">
|
||||
{error}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-2">
|
||||
{entries === null && <div className="text-xs text-ctp-overlay2 p-3">Loading...</div>}
|
||||
{loadFailed && (
|
||||
<div role="alert" className="text-xs text-ctp-red p-3">
|
||||
Could not load the library. Close this dialog and try again.
|
||||
</div>
|
||||
)}
|
||||
{!loadFailed && entries !== null && visible.length === 0 && (
|
||||
<div className="text-xs text-ctp-overlay2 p-3">
|
||||
{query.trim() ? 'No matches' : 'No other titles'}
|
||||
</div>
|
||||
)}
|
||||
{visible.map((entry) => (
|
||||
<button
|
||||
key={entry.animeId}
|
||||
type="button"
|
||||
disabled={busy}
|
||||
onClick={() => onSelect(entry)}
|
||||
className="w-full flex items-center gap-3 p-2.5 rounded-lg hover:bg-ctp-surface0 transition-colors text-left disabled:opacity-50"
|
||||
>
|
||||
<AnimeCoverImage
|
||||
animeId={entry.animeId}
|
||||
title={entry.canonicalTitle}
|
||||
coverRetryToken={entry.anilistId ?? 0}
|
||||
className="w-10 h-14 rounded shrink-0"
|
||||
/>
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="text-sm text-ctp-text truncate">{entry.canonicalTitle}</div>
|
||||
<div className="text-xs text-ctp-overlay2 mt-0.5">
|
||||
{entry.episodeCount} episode{entry.episodeCount !== 1 ? 's' : ''} ·{' '}
|
||||
{formatDuration(entry.totalActiveMs)}
|
||||
</div>
|
||||
</div>
|
||||
{busyAnimeId === entry.animeId ? (
|
||||
<span className="text-xs text-ctp-blue shrink-0">Moving...</span>
|
||||
) : (
|
||||
<span className="text-xs text-ctp-overlay2 shrink-0">Select</span>
|
||||
)}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { Window } from 'happy-dom';
|
||||
import { act } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
import { apiClient } from '../../lib/api-client';
|
||||
import type { StatsDuplicateLineCleanupResult } from '../../types/stats';
|
||||
import { DuplicateLineCleanup } from './DuplicateLineCleanup';
|
||||
|
||||
interface TestWindow extends Window {
|
||||
IS_REACT_ACT_ENVIRONMENT?: boolean;
|
||||
}
|
||||
|
||||
function installDom(): () => void {
|
||||
const previousWindow = globalThis.window;
|
||||
const previousDocument = globalThis.document;
|
||||
const previousHTMLElement = globalThis.HTMLElement;
|
||||
const previousISReactActEnvironment = (
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT;
|
||||
const window = new Window() as TestWindow;
|
||||
|
||||
Object.defineProperty(globalThis, 'window', { value: window, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: window.document, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: window.HTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = true;
|
||||
|
||||
return () => {
|
||||
Object.defineProperty(globalThis, 'window', { value: previousWindow, configurable: true });
|
||||
Object.defineProperty(globalThis, 'document', { value: previousDocument, configurable: true });
|
||||
Object.defineProperty(globalThis, 'HTMLElement', {
|
||||
value: previousHTMLElement,
|
||||
configurable: true,
|
||||
});
|
||||
(
|
||||
globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT?: boolean }
|
||||
).IS_REACT_ACT_ENVIRONMENT = previousISReactActEnvironment;
|
||||
};
|
||||
}
|
||||
|
||||
function findButton(container: Element, label: string): HTMLButtonElement {
|
||||
const match = [...container.querySelectorAll('button')].find(
|
||||
(button) => (button.textContent ?? '').trim() === label,
|
||||
);
|
||||
assert.ok(match, `expected a "${label}" button`);
|
||||
return match as unknown as HTMLButtonElement;
|
||||
}
|
||||
|
||||
/** The backdrop stays clickable during an apply, so it reaches the guard in `close`. */
|
||||
function findBackdrop(container: Element): HTMLButtonElement {
|
||||
const match = container.querySelector('button[aria-label="Close duplicate line cleanup"]');
|
||||
assert.ok(match, 'expected the backdrop close button');
|
||||
return match as unknown as HTMLButtonElement;
|
||||
}
|
||||
|
||||
function deferred<T>(): { promise: Promise<T>; resolve: (value: T) => void } {
|
||||
let resolve!: (value: T) => void;
|
||||
const promise = new Promise<T>((done) => {
|
||||
resolve = done;
|
||||
});
|
||||
return { promise, resolve };
|
||||
}
|
||||
|
||||
function summary(
|
||||
overrides: Partial<StatsDuplicateLineCleanupResult> = {},
|
||||
): StatsDuplicateLineCleanupResult {
|
||||
return {
|
||||
dryRun: false,
|
||||
lookbackDays: 30,
|
||||
scannedLines: 900,
|
||||
burstGroups: 2,
|
||||
removedLines: 180,
|
||||
removedWordOccurrences: 540,
|
||||
removedKanjiOccurrences: 120,
|
||||
samples: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
interface Harness {
|
||||
container: Element;
|
||||
cleanedCalls: () => number;
|
||||
closedCalls: () => number;
|
||||
teardown: () => void;
|
||||
}
|
||||
|
||||
async function mount(cleanup: (typeof apiClient)['cleanupDuplicateLines']): Promise<Harness> {
|
||||
const uninstallDom = installDom();
|
||||
const originalCleanup = apiClient.cleanupDuplicateLines;
|
||||
apiClient.cleanupDuplicateLines = cleanup;
|
||||
|
||||
let cleaned = 0;
|
||||
let closed = 0;
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const root = createRoot(container);
|
||||
|
||||
await act(async () => {
|
||||
root.render(
|
||||
<DuplicateLineCleanup
|
||||
onClose={() => {
|
||||
closed += 1;
|
||||
}}
|
||||
onCleaned={() => {
|
||||
cleaned += 1;
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
});
|
||||
|
||||
return {
|
||||
container,
|
||||
cleanedCalls: () => cleaned,
|
||||
closedCalls: () => closed,
|
||||
teardown: () => {
|
||||
apiClient.cleanupDuplicateLines = originalCleanup;
|
||||
uninstallDom();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
test('a reload is still owed after a later scan replaces the applied result', async () => {
|
||||
const harness = await mount(async ({ dryRun } = {}) => summary({ dryRun: dryRun === true }));
|
||||
|
||||
try {
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Clean Up').click();
|
||||
});
|
||||
assert.equal(harness.cleanedCalls(), 0, 'reload must wait for the result to be read');
|
||||
|
||||
// The follow-up scan clears the applied summary, but the rows are already gone.
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Close').click();
|
||||
});
|
||||
|
||||
assert.equal(harness.cleanedCalls(), 1);
|
||||
assert.equal(harness.closedCalls(), 1);
|
||||
} finally {
|
||||
harness.teardown();
|
||||
}
|
||||
});
|
||||
|
||||
test('a reload is still owed after the lookback window changes', async () => {
|
||||
const harness = await mount(async ({ dryRun } = {}) => summary({ dryRun: dryRun === true }));
|
||||
|
||||
try {
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Clean Up').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, '7 days').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Close').click();
|
||||
});
|
||||
|
||||
assert.equal(harness.cleanedCalls(), 1);
|
||||
} finally {
|
||||
harness.teardown();
|
||||
}
|
||||
});
|
||||
|
||||
test('closing is refused while an apply is in flight', async () => {
|
||||
const pending = deferred<StatsDuplicateLineCleanupResult>();
|
||||
const harness = await mount(async ({ dryRun } = {}) =>
|
||||
dryRun === true ? summary({ dryRun: true }) : pending.promise,
|
||||
);
|
||||
|
||||
try {
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Clean Up').click();
|
||||
});
|
||||
|
||||
assert.equal(findButton(harness.container, 'Close').disabled, true);
|
||||
// The backdrop is never disabled, so this is the path that has to be refused.
|
||||
await act(async () => {
|
||||
findBackdrop(harness.container).click();
|
||||
});
|
||||
assert.equal(harness.closedCalls(), 0, 'the modal must stay open mid-apply');
|
||||
assert.equal(harness.cleanedCalls(), 0);
|
||||
|
||||
await act(async () => {
|
||||
pending.resolve(summary({ removedLines: 12 }));
|
||||
await pending.promise;
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Close').click();
|
||||
});
|
||||
|
||||
assert.equal(harness.closedCalls(), 1);
|
||||
assert.equal(harness.cleanedCalls(), 1);
|
||||
} finally {
|
||||
harness.teardown();
|
||||
}
|
||||
});
|
||||
|
||||
test('a scan on its own owes no reload', async () => {
|
||||
const harness = await mount(async ({ dryRun } = {}) => summary({ dryRun: dryRun === true }));
|
||||
|
||||
try {
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Close').click();
|
||||
});
|
||||
|
||||
assert.equal(harness.cleanedCalls(), 0);
|
||||
assert.equal(harness.closedCalls(), 1);
|
||||
} finally {
|
||||
harness.teardown();
|
||||
}
|
||||
});
|
||||
|
||||
test('an apply that removes nothing owes no reload', async () => {
|
||||
// The scan saw work to do, but by the time it ran another cleanup had taken it.
|
||||
const harness = await mount(async ({ dryRun } = {}) =>
|
||||
dryRun === true ? summary({ dryRun: true }) : summary({ burstGroups: 0, removedLines: 0 }),
|
||||
);
|
||||
|
||||
try {
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Scan').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Clean Up').click();
|
||||
});
|
||||
await act(async () => {
|
||||
findButton(harness.container, 'Close').click();
|
||||
});
|
||||
|
||||
assert.equal(harness.cleanedCalls(), 0);
|
||||
assert.equal(harness.closedCalls(), 1);
|
||||
} finally {
|
||||
harness.teardown();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,202 @@
|
||||
import { useCallback, useState } from 'react';
|
||||
import { getStatsClient } from '../../hooks/useStatsApi';
|
||||
import { formatNumber } from '../../lib/formatters';
|
||||
import type { StatsDuplicateLineCleanupResult } from '../../types/stats';
|
||||
|
||||
interface DuplicateLineCleanupProps {
|
||||
onClose: () => void;
|
||||
/** Called after rows are actually removed, so the charts can reload. */
|
||||
onCleaned: () => void;
|
||||
}
|
||||
|
||||
const LOOKBACK_CHOICES: Array<{ label: string; days: number | null }> = [
|
||||
{ label: '7 days', days: 7 },
|
||||
{ label: '30 days', days: 30 },
|
||||
{ label: '90 days', days: 90 },
|
||||
{ label: '1 year', days: 365 },
|
||||
{ label: 'All time', days: null },
|
||||
];
|
||||
|
||||
function formatTimecode(ms: number): string {
|
||||
const totalSeconds = Math.max(0, Math.floor(ms / 1000));
|
||||
const minutes = Math.floor(totalSeconds / 60);
|
||||
const seconds = totalSeconds % 60;
|
||||
return `${minutes}:${String(seconds).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
export function DuplicateLineCleanup({ onClose, onCleaned }: DuplicateLineCleanupProps) {
|
||||
const [lookbackDays, setLookbackDays] = useState<number | null>(30);
|
||||
const [preview, setPreview] = useState<StatsDuplicateLineCleanupResult | null>(null);
|
||||
const [applied, setApplied] = useState<StatsDuplicateLineCleanupResult | null>(null);
|
||||
const [busy, setBusy] = useState<'scan' | 'apply' | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
// Survives everything the displayed result does not: another scan, a different window.
|
||||
// Rows are gone from the moment an apply succeeds, so the reload is owed until it runs.
|
||||
const [needsReload, setNeedsReload] = useState(false);
|
||||
|
||||
const run = useCallback(
|
||||
async (dryRun: boolean) => {
|
||||
setBusy(dryRun ? 'scan' : 'apply');
|
||||
setError(null);
|
||||
try {
|
||||
const result = await getStatsClient().cleanupDuplicateLines({ dryRun, lookbackDays });
|
||||
if (dryRun) {
|
||||
setPreview(result);
|
||||
setApplied(null);
|
||||
} else {
|
||||
setApplied(result);
|
||||
setPreview(null);
|
||||
if (result.removedLines > 0) {
|
||||
setNeedsReload(true);
|
||||
}
|
||||
}
|
||||
} catch (cause) {
|
||||
setError(cause instanceof Error ? cause.message : String(cause));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
},
|
||||
[lookbackDays],
|
||||
);
|
||||
|
||||
// Reloading the vocabulary tables unmounts this modal along with the rest of the tab,
|
||||
// so it waits for the user to close: they get to read what was removed first. Closing
|
||||
// is refused mid-apply, which would drop the reload on the floor along with the report.
|
||||
const close = useCallback(() => {
|
||||
if (busy === 'apply') {
|
||||
return;
|
||||
}
|
||||
if (needsReload) {
|
||||
onCleaned();
|
||||
}
|
||||
onClose();
|
||||
}, [busy, needsReload, onCleaned, onClose]);
|
||||
|
||||
const result = applied ?? preview;
|
||||
const nothingToDo = preview !== null && preview.removedLines === 0;
|
||||
|
||||
return (
|
||||
<div className="fixed inset-0 z-50">
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Close duplicate line cleanup"
|
||||
className="absolute inset-0 bg-ctp-crust/70 backdrop-blur-[2px]"
|
||||
onClick={close}
|
||||
/>
|
||||
<div className="absolute inset-x-0 top-1/2 mx-auto max-w-xl -translate-y-1/2 rounded-xl border border-ctp-surface1 bg-ctp-mantle shadow-2xl">
|
||||
<div className="flex items-center justify-between border-b border-ctp-surface1 px-5 py-4">
|
||||
<h2 className="text-sm font-semibold text-ctp-text">Duplicate Lines</h2>
|
||||
<button
|
||||
type="button"
|
||||
disabled={busy === 'apply'}
|
||||
className="rounded-md border border-ctp-surface2 px-3 py-1.5 text-xs font-medium text-ctp-subtext0 transition hover:border-ctp-blue hover:text-ctp-blue disabled:opacity-50"
|
||||
onClick={close}
|
||||
>
|
||||
Close
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="space-y-4 px-5 py-4">
|
||||
<p className="text-xs leading-relaxed text-ctp-subtext0">
|
||||
Typeset subtitles — karaoke openings, animated signs — are authored as one event per
|
||||
animation frame, and older versions counted every frame as its own line. This finds
|
||||
those runs and collapses each one back to a single line, giving back the word and kanji
|
||||
counts they inflated. Ordinary repeated dialogue is left alone.
|
||||
</p>
|
||||
|
||||
<div>
|
||||
<div className="mb-2 text-xs font-medium text-ctp-subtext1">Look back over</div>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{LOOKBACK_CHOICES.map((choice) => (
|
||||
<button
|
||||
key={choice.label}
|
||||
type="button"
|
||||
disabled={busy !== null}
|
||||
onClick={() => {
|
||||
setLookbackDays(choice.days);
|
||||
setPreview(null);
|
||||
setApplied(null);
|
||||
}}
|
||||
className={`rounded-lg border px-3 py-1.5 text-xs transition disabled:opacity-50 ${
|
||||
lookbackDays === choice.days
|
||||
? 'border-ctp-blue/50 bg-ctp-surface2 text-ctp-text'
|
||||
: 'border-ctp-surface1 bg-ctp-surface0 text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
{choice.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="rounded-lg border border-ctp-red/30 bg-ctp-red/10 px-3 py-2 text-xs text-ctp-red">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{result && (
|
||||
<div className="rounded-lg bg-ctp-surface0 px-4 py-3">
|
||||
<div className="text-sm text-ctp-text">
|
||||
{applied
|
||||
? `Removed ${formatNumber(applied.removedLines)} repeated lines`
|
||||
: nothingToDo
|
||||
? 'No animation bursts found in this window'
|
||||
: `Found ${formatNumber(preview!.burstGroups)} bursts covering ${formatNumber(preview!.removedLines)} extra lines`}
|
||||
</div>
|
||||
<div className="mt-1 text-xs text-ctp-overlay2">
|
||||
{formatNumber(result.scannedLines)} lines scanned ·{' '}
|
||||
{formatNumber(result.removedWordOccurrences)} word counts ·{' '}
|
||||
{formatNumber(result.removedKanjiOccurrences)} kanji counts
|
||||
{applied ? ' removed' : ' would be removed'}
|
||||
</div>
|
||||
|
||||
{result.samples.length > 0 && (
|
||||
<div className="mt-3 max-h-52 space-y-1.5 overflow-y-auto">
|
||||
{result.samples.map((sample) => (
|
||||
<div
|
||||
key={`${sample.videoId}:${sample.startMs}:${sample.text}`}
|
||||
className="flex items-center justify-between gap-3 rounded-md bg-ctp-mantle px-3 py-1.5"
|
||||
>
|
||||
<div className="min-w-0">
|
||||
<div className="truncate text-xs text-ctp-text">{sample.text}</div>
|
||||
<div className="truncate text-[11px] text-ctp-overlay1">
|
||||
{sample.videoTitle ?? `Video ${sample.videoId}`} ·{' '}
|
||||
{formatTimecode(sample.startMs)}
|
||||
</div>
|
||||
</div>
|
||||
<span className="shrink-0 text-xs text-ctp-peach">×{sample.frames}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex items-center justify-end gap-2">
|
||||
<button
|
||||
type="button"
|
||||
disabled={busy !== null}
|
||||
onClick={() => void run(true)}
|
||||
className="rounded-md border border-ctp-surface2 px-3 py-1.5 text-xs font-medium text-ctp-subtext0 transition hover:border-ctp-blue hover:text-ctp-blue disabled:opacity-50"
|
||||
>
|
||||
{busy === 'scan' ? 'Scanning…' : 'Scan'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={busy !== null || preview === null || nothingToDo}
|
||||
onClick={() => void run(false)}
|
||||
className="rounded-md border border-ctp-red/30 px-3 py-1.5 text-xs font-medium text-ctp-red transition hover:bg-ctp-red/10 disabled:opacity-40"
|
||||
>
|
||||
{busy === 'apply' ? 'Cleaning…' : 'Clean Up'}
|
||||
</button>
|
||||
</div>
|
||||
<p className="text-[11px] text-ctp-overlay1">
|
||||
Scan first: cleanup removes rows and cannot be undone. Session watch time and lines-seen
|
||||
totals are left untouched.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user