mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-11-10 04:46:07 -08:00
feat(stats): overhaul the Trends tab (#325)
This commit is contained in:
1 parent
f89f78b861
commit
269c0708cb
73 files changed
+6085
-3354
No files matched your search
@@ -0,0 +1,15 @@
|
||||
type: changed
|
||||
area: stats
|
||||
|
||||
- Rebuilt the Trends tab: summary totals with period comparison, a Learning section (new words per 1,000 seen, new words by frequency rank), a day and hour heatmap, a title timeline, and a single running-total chart with a metric switcher that climbs from zero. These replace the seven cumulative charts, the Sessions chart, the day-of-week and hour charts, the four overlapping per-title charts, and the top titles chart.
|
||||
- Added week grouping, custom date ranges, a title filter for the whole page, and click-through from charts to sessions and from titles to their library page. Range and grouping are remembered, and changing them no longer blanks the page or loses your scroll position.
|
||||
- Fixed the Library section including days outside the selected range, so its totals now match Activity.
|
||||
- Fixed episode counts: Trends now counts finished episodes, each once, instead of counting a video again for every day it was opened.
|
||||
- New word counts across Trends and Vocabulary now count dictionary words, so a word's conjugations count once. Totals are lower than before.
|
||||
- Rate charts no longer plot idle days as zero or spike on very short days: they show gaps when there is too little data, and daily views use a rolling 7 days.
|
||||
- Fixed clipped chart axis labels and month labels that read like dates, and the All range ignoring Show empty days.
|
||||
- Rebuilt the Vocabulary tab: one word table with Worth mining, All words, Across titles, and Newest views, search across every word, form, and reading, sortable columns, and inline Hide. It replaces three lists that could only see your 500 most seen word forms. Also added known words by frequency band, a vocabulary growth chart, and a kanji view covering every kanji, with a filter for kanji outside your known words.
|
||||
- Fixed word counts and readings: lists and the word detail panel now total every form of a word and show the dictionary form's reading, so 見る no longer shows みてる. The detail panel loads example lines straight away and lists a word's forms.
|
||||
- Fixed names slipping past Hide Names and the Trends new-word counts when tagged with more than one part of speech.
|
||||
- Stats now says when a word or line can't be mined because it came from a stream, in Search and the word panel, instead of showing Unavailable buttons. Lines you can mine are listed first. The word panel counts affected lines, and the word table marks words with only streamed lines as Stream only.
|
||||
- Fixed stats treating the day as one earlier in time zones ahead of UTC, which affected today's totals, the custom date picker, and first-seen dates.
|
||||
@@ -64,7 +64,17 @@ The same show can end up on several cards when release names disagree. To fix th
|
||||
|
||||
### Trends
|
||||
|
||||
Charts for watch time, cards, words, and sessions per day or month, running totals, efficiency (words per minute, cards per hour), and viewing patterns by weekday and hour. Each chart has its own date range and grouping.
|
||||
Pick a date range (or set **Custom** dates), group by day, week, or month, and the whole page follows. Your choices are remembered.
|
||||
|
||||
- **Summary**: totals for the range, compared with the period just before it.
|
||||
- **Activity**: watch time, cards, words, and finished episodes. Click a bar to open that period's sessions.
|
||||
- **Learning**: new words per 1,000 words seen, and how common those new words are. Conjugations of a word count once, and names are left out.
|
||||
- **Efficiency**: words per minute, cards per hour, and lookups per 100 words. Daily views use a rolling 7 days, and gaps mean too little data to rate.
|
||||
- **Running total**: one chart with a switcher for each metric, climbing from zero at the start of the range.
|
||||
- **Patterns**: a day and hour heatmap of when you watch.
|
||||
- **Titles**: a timeline of what you watched when, and a sortable summary table. Click a title, or use the filter box, to narrow the page to it. **↗** opens the title.
|
||||
|
||||
An episode counts as finished once, on the last day you watched it.
|
||||
|
||||

|
||||
|
||||
@@ -76,10 +86,15 @@ Session history. Expand a session for its pace, lines, known-word rate, lookups,
|
||||
|
||||
### Vocabulary
|
||||
|
||||
Unique words and kanji you have seen, new words per day, frequency rank tables with Hide Known and Hide Kana filters, and a kanji breakdown. Click a word to see every line it appeared in.
|
||||
Words are counted as dictionary words, so a word's conjugations count once. Names are left out unless you turn off **Hide names**.
|
||||
|
||||
- **Exclusions** hides words from every vocabulary view. You can restore them from the same dialog.
|
||||
- **Duplicates** cleans up lines repeated by karaoke openings and animated signs (see [Repeated lines](#repeated-lines)).
|
||||
- **Summary**: words seen, how many are in Anki, new words this week, and kanji seen.
|
||||
- **Known Words by Frequency**: how much of each frequency band you already know.
|
||||
- **Vocabulary Growth**: your running total of words seen.
|
||||
- **Words**: one table for every word you have seen. **Worth mining** lists common words you haven't mined, most seen first. **All words**, **Across titles**, and **Newest** are the other views. Sort by any column, or filter by known words, names, kana, and part of speech. Searching looks through everything, including conjugated forms and readings. Click a word for its forms, where it appeared, and example lines you can mine from. **Hide** excludes a word.
|
||||
- **Kanji**: every kanji you have seen, shaded by how often. Dashed kanji are not in any word you know.
|
||||
|
||||
**Exclusions** at the bottom of the page lists hidden words so you can restore them. **Clean up duplicates** removes lines repeated by karaoke openings and animated signs (see [Repeated lines](#repeated-lines)).
|
||||
|
||||

|
||||
|
||||
@@ -89,7 +104,7 @@ Searches the primary subtitle lines and titles in your history. **Search by head
|
||||
|
||||
## Mining from the dashboard
|
||||
|
||||
Search results and the Vocabulary word panel can create cards from past lines, as long as the source video file is still available:
|
||||
Search results and the Vocabulary word panel can create cards from past lines, as long as the source video file is still available. Lines from a stream, such as Jellyfin or YouTube, have no file to cut clips from, so they say so instead of offering mining. Lines you can mine are listed first. The word panel counts how many of a word's lines this affects, and the word table marks words with only streamed lines as **Stream only**.
|
||||
|
||||
- **Mine Word**: looks up the word with the selected dictionary backend (Yomitan or Hachidori), plus sentence, audio, and image. The history line you picked is the card's sentence, even while mpv plays another line. Hachidori uses its own Anki template, dictionary aliases, and frequency data.
|
||||
- **Mine Sentence**: a sentence card with `IsSentenceCard` set, for Lapis and Kiku note types.
|
||||
@@ -103,7 +118,7 @@ Dashboard cards also get the `SubMiner::Stats` tag. With the Anki proxy off, Sub
|
||||
|
||||
Karaoke openings and animated signs repeat the same text once per frame. SubMiner collapses these as it records, so one lyric is stored once. Stats recorded before that can hold hundreds of copies and skew Top Repeated Words.
|
||||
|
||||
To clean them, use **Duplicates** in the Vocabulary tab: pick a time window, **Scan** to preview, then **Clean Up**. Or from the terminal:
|
||||
To clean them, use **Clean up duplicates** in the Vocabulary tab: pick a time window, **Scan** to preview, then **Clean Up**. Or from the terminal:
|
||||
|
||||
```bash
|
||||
subminer stats cleanup --duplicate-lines --dry-run --lookback-days 30
|
||||
|
||||
@@ -12,21 +12,42 @@ Trend charts now consume one chart-oriented backend payload from `/api/stats/tre
|
||||
- collapse multiple network round-trips into one request
|
||||
- keep heavy chart shaping close to tracker/query logic
|
||||
|
||||
## Date Ranges And Buckets
|
||||
|
||||
- A range resolves to calendar days (`resolveTrendWindow`). Rollups are selected by day span, never by "last N rows", so sparse activity cannot pull older days into a range. Custom ranges are `{ startDay, endDay }` from `range=custom&fromDay&toDay`.
|
||||
- `all` spans the recorded data, not today.
|
||||
- Day and week buckets read daily rollups. Month buckets read monthly rollups for months wholly in range, because the `minimal` and `deep-history` retention presets prune daily rollups first, and daily rollups for a partly covered month. Weeks start on Monday.
|
||||
- `buckets` is the one axis for `activity`, `progress`, and `ratios`, index for index. Charts use it to map a clicked bar back to its days.
|
||||
- Each `progress` point is the total at the end of its period, and the chart adds the zero starting point.
|
||||
|
||||
## Data Sources
|
||||
|
||||
- rollup-backed:
|
||||
- activity charts
|
||||
- cumulative watch/cards/tokens/sessions trends
|
||||
- per-anime watch/cards/tokens/episodes series (`animeDaily`: sparse per-title values for active days only; `TrendsTab` builds the running totals with `buildAnimeCumulative`, because a dense title x day payload grew to several MB)
|
||||
- activity charts and running totals
|
||||
- `titleTimeline`: sparse per-title minutes for active days only, because a dense title x day payload grew to several MB
|
||||
- session-metric-backed:
|
||||
- lookup trends
|
||||
- lookup rate trends
|
||||
- watch-time by day-of-week/hour
|
||||
- lookups and lookup rate
|
||||
- `watchTimeByDayHour`, attributed to the hour each session started
|
||||
- rollup word counts and session metrics share one session query per request (`getTrendSessionRows`), so the per-session display-word count runs once instead of once per rollup query
|
||||
- `imm_videos.watched`-backed:
|
||||
- finished episodes count each watched video once, on the last day it has a rollup, since the flag has no timestamp
|
||||
- vocabulary-backed:
|
||||
- new-words trend reads permanent daily lexical rollups
|
||||
- rollup rows count only vocabulary-visible tokens and normalize mixed legacy timestamp units
|
||||
- a persisted rollup version invalidates stale materializations and triggers an atomic background rebuild
|
||||
- `getNewWordFirstSeenRows` folds `imm_words` rows into dictionary words first, because the table is unique on (headword, word, reading) and stores each conjugation separately
|
||||
- the aggregate scans every word, so it is cached per connection until `total_changes()` or `PRAGMA data_version` moves
|
||||
- Trends new-word metrics exclude names; the Vocabulary tab offers both
|
||||
- `imm_lexical_daily_rollups` is no longer read for new-word counts (it counts surface forms). Its version state still gates the `vocabulary_visible` backfill.
|
||||
|
||||
## Vocabulary Word List
|
||||
|
||||
`/api/stats/vocabulary/words` (`query-vocabulary-words.ts`) folds `imm_words` into dictionary words on the server and filters, sorts, and pages there, because a client-side list can only hold a slice of the table. The fold is cached per connection until a write, the same way as the new-word aggregate. `titles` is computed for the returned rows; sorting or filtering by it needs every word's count (about 900ms), so that map tolerates five minutes of staleness. Names are detected with `isProperNounPos2`, since merged tokens store tags like `固有名詞|一般`.
|
||||
|
||||
## Rates
|
||||
|
||||
Rates are `null` when the underlying data is under 30 active minutes or 2,000 words, and charts draw that as a gap. Daily views rate a trailing 7 calendar days, volume-weighted, which is why the query loads days before the range. Week and month views rate each bucket.
|
||||
|
||||
## Title Filter
|
||||
|
||||
`title` narrows totals, charts, and patterns. `librarySummary` and `titleTimeline` stay complete so the page can switch titles. New words have no per-title record, so `learning` and `summary.current.newWords` are `null` while filtered.
|
||||
|
||||
## Metric Semantics
|
||||
|
||||
@@ -40,6 +61,6 @@ Stats data crosses the process boundary over HTTP only. Shared JSON models live
|
||||
|
||||
The stats preload bridge remains limited to native window behavior such as confirmation-dialog layering. Do not add stats data request channels back to Electron IPC.
|
||||
|
||||
The stats UI should treat the trends payload as chart-ready data. Presentation-only work in the client is fine (including the per-anime running totals, memoized per payload), but rebuilding the main trend datasets from raw sessions should stay out of the render path.
|
||||
The stats UI should treat the trends payload as chart-ready data. Presentation-only work in the client is fine (including the title timeline layout), but rebuilding the main trend datasets from raw sessions should stay out of the render path.
|
||||
|
||||
For session detail timelines, omitting `limit` now means "return the full retained session telemetry/history". Explicit `limit` remains available for bounded callers, but the default stats UI path should not trim long sessions to the newest 200 samples.
|
||||
@@ -155,15 +155,20 @@ const WATCH_TIME_PER_ANIME = [
|
||||
];
|
||||
|
||||
const TRENDS_DASHBOARD = {
|
||||
window: { startDay: 20_000, endDay: 20_000 },
|
||||
buckets: [{ label: 'Mar 1', startDay: 20_000, endDay: 20_000 }],
|
||||
summary: {
|
||||
current: { watchTimeMin: 25, words: 300, newWords: 12, cards: 5, episodes: 2, activeDays: 1 },
|
||||
previous: null,
|
||||
},
|
||||
activity: {
|
||||
watchTime: [{ label: 'Mar 1', value: 25 }],
|
||||
cards: [{ label: 'Mar 1', value: 5 }],
|
||||
words: [{ label: 'Mar 1', value: 300 }],
|
||||
sessions: [{ label: 'Mar 1', value: 3 }],
|
||||
episodes: [{ label: 'Mar 1', value: 2 }],
|
||||
},
|
||||
progress: {
|
||||
watchTime: [{ label: 'Mar 1', value: 25 }],
|
||||
sessions: [{ label: 'Mar 1', value: 3 }],
|
||||
words: [{ label: 'Mar 1', value: 300 }],
|
||||
newWords: [{ label: 'Mar 1', value: 12 }],
|
||||
cards: [{ label: 'Mar 1', value: 5 }],
|
||||
@@ -175,11 +180,19 @@ const TRENDS_DASHBOARD = {
|
||||
cardsPerHour: [{ label: 'Mar 1', value: 12 }],
|
||||
readingSpeed: [{ label: 'Mar 1', value: 180 }],
|
||||
},
|
||||
learning: {
|
||||
newWordsPerThousand: [{ label: 'Mar 1', value: null }],
|
||||
frequencyBands: [],
|
||||
},
|
||||
watchTimeByDayHour: [],
|
||||
librarySummary: [
|
||||
{
|
||||
title: 'Little Witch Academia',
|
||||
animeId: 1,
|
||||
videoId: 1,
|
||||
watchTimeMin: 25,
|
||||
videos: 1,
|
||||
episodesFinished: 1,
|
||||
sessions: 1,
|
||||
cards: 5,
|
||||
words: 300,
|
||||
@@ -189,16 +202,7 @@ const TRENDS_DASHBOARD = {
|
||||
lastWatched: 20_000,
|
||||
},
|
||||
],
|
||||
animeDaily: {
|
||||
watchTime: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 25 }],
|
||||
episodes: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 1 }],
|
||||
cards: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 5 }],
|
||||
words: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 300 }],
|
||||
},
|
||||
patterns: {
|
||||
watchTimeByDayOfWeek: [{ label: 'Sun', value: 25 }],
|
||||
watchTimeByHour: [{ label: '12:00', value: 25 }],
|
||||
},
|
||||
titleTimeline: [{ epochDay: 20_000, animeTitle: 'Little Witch Academia', value: 25 }],
|
||||
};
|
||||
|
||||
const ANIME_EPISODES = [
|
||||
@@ -227,6 +231,12 @@ const WORD_DETAIL = {
|
||||
pos2: '一般',
|
||||
pos3: null,
|
||||
frequency: 42,
|
||||
frequencyRank: 100,
|
||||
dictionaryReading: 'ねこ',
|
||||
formCount: 1,
|
||||
forms: [{ wordId: 1, word: '猫', reading: 'ねこ', seen: 42 }],
|
||||
lineCount: 40,
|
||||
streamLineCount: 0,
|
||||
firstSeen: Date.now() - 100_000,
|
||||
lastSeen: Date.now(),
|
||||
};
|
||||
@@ -313,11 +323,11 @@ function createMockTracker(
|
||||
}),
|
||||
getVocabularyChartData: async () => ({
|
||||
ready: true,
|
||||
topWords: [{ wordId: 1, headword: 'する', frequency: 50 }],
|
||||
topWordsWithoutNames: [{ wordId: 1, headword: 'する', frequency: 50 }],
|
||||
newWordsTimeline: [{ epochDay: 20_000, wordCount: 3 }],
|
||||
newWordsTimelineWithoutNames: [{ epochDay: 20_000, wordCount: 3 }],
|
||||
}),
|
||||
getDictionaryWords: async () => ({ rows: [], total: 0, maxSeen: 0 }),
|
||||
getVocabularyCoverage: async () => [],
|
||||
getStatsExcludedWords: async () => [],
|
||||
replaceStatsExcludedWords: async () => {},
|
||||
getKanjiStats: async () => KANJI_STATS,
|
||||
@@ -674,9 +684,16 @@ const CLAMPED_QUERIES: Array<{
|
||||
];
|
||||
|
||||
const TRENDS_DASHBOARD_QUERIES = [
|
||||
{ query: 'range=90d&groupBy=month', expectedArgs: ['90d', 'month', true] },
|
||||
{ query: 'range=weird&groupBy=year', expectedArgs: ['30d', 'day', true] },
|
||||
{ query: 'range=30d&groupBy=day&fillEmpty=false', expectedArgs: ['30d', 'day', false] },
|
||||
{ query: 'range=90d&groupBy=month', expectedArgs: ['90d', 'month', true, null] },
|
||||
{ query: 'range=90d&groupBy=week', expectedArgs: ['90d', 'week', true, null] },
|
||||
{ query: 'range=weird&groupBy=year', expectedArgs: ['30d', 'day', true, null] },
|
||||
{ query: 'range=30d&groupBy=day&fillEmpty=false', expectedArgs: ['30d', 'day', false, null] },
|
||||
{
|
||||
query: 'range=custom&fromDay=20000&toDay=20010',
|
||||
expectedArgs: [{ startDay: 20000, endDay: 20010 }, 'day', true, null],
|
||||
},
|
||||
{ query: 'range=custom&fromDay=20010&toDay=20000', expectedArgs: ['30d', 'day', true, null] },
|
||||
{ query: 'range=30d&title=Bocchi%20%26%20Co', expectedArgs: ['30d', 'day', true, 'Bocchi & Co'] },
|
||||
];
|
||||
|
||||
const MALFORMED_IDS = [
|
||||
@@ -991,7 +1008,7 @@ describe('stats server API routes', () => {
|
||||
});
|
||||
|
||||
for (const { query, expectedArgs } of TRENDS_DASHBOARD_QUERIES) {
|
||||
it(`GET /api/stats/trends/dashboard?${query} requests ${expectedArgs.join('/')}`, async () => {
|
||||
it(`GET /api/stats/trends/dashboard?${query} requests ${JSON.stringify(expectedArgs)}`, async () => {
|
||||
let seenArgs: unknown[] = [];
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
@@ -1008,6 +1025,69 @@ describe('stats server API routes', () => {
|
||||
});
|
||||
}
|
||||
|
||||
it('GET /api/stats/vocabulary/words passes a parsed query and ignores unknown values', async () => {
|
||||
const seenQueries: unknown[] = [];
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getDictionaryWords: async (_knownWords: unknown, query: unknown) => {
|
||||
seenQueries.push(query);
|
||||
return { rows: [], total: 0, maxSeen: 0 };
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
await app.request(
|
||||
'/api/stats/vocabulary/words?search=%20%E7%8C%AB%20&sort=rank&order=asc&hideKnown=true&hideNames=true&pos=verb&maxRank=10000&minTitles=2&limit=25&offset=50',
|
||||
);
|
||||
await app.request('/api/stats/vocabulary/words?sort=bogus&pos=bogus&maxRank=-1&limit=99999');
|
||||
|
||||
assert.deepEqual(seenQueries, [
|
||||
{
|
||||
search: '猫',
|
||||
sort: 'rank',
|
||||
descending: false,
|
||||
hideKnown: true,
|
||||
hideNames: true,
|
||||
hideKana: false,
|
||||
partOfSpeech: 'verb',
|
||||
maxRank: 10000,
|
||||
minTitles: 2,
|
||||
limit: 25,
|
||||
offset: 50,
|
||||
},
|
||||
{
|
||||
search: undefined,
|
||||
sort: 'seen',
|
||||
descending: true,
|
||||
hideKnown: false,
|
||||
hideNames: false,
|
||||
hideKana: false,
|
||||
partOfSpeech: undefined,
|
||||
maxRank: undefined,
|
||||
minTitles: undefined,
|
||||
limit: 200,
|
||||
offset: 0,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('GET /api/stats/sessions narrows to a valid day range and ignores a reversed one', async () => {
|
||||
const seenRanges: unknown[] = [];
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getSessionSummaries: async (_limit?: number, dayRange?: unknown) => {
|
||||
seenRanges.push(dayRange);
|
||||
return SESSION_SUMMARIES;
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
await app.request('/api/stats/sessions?limit=500&fromDay=20000&toDay=20006');
|
||||
await app.request('/api/stats/sessions?fromDay=20006&toDay=20000');
|
||||
|
||||
assert.deepEqual(seenRanges, [{ startDay: 20000, endDay: 20006 }, undefined]);
|
||||
});
|
||||
|
||||
it('GET /api/stats/vocabulary/occurrences returns recent occurrence rows for a word', async () => {
|
||||
let seenArgs: unknown[] = [];
|
||||
const app = createStatsApp(
|
||||
@@ -1026,7 +1106,7 @@ describe('stats server API routes', () => {
|
||||
const body = await res.json();
|
||||
assert.ok(Array.isArray(body));
|
||||
assert.equal(body[0].animeTitle, 'Little Witch Academia');
|
||||
assert.deepEqual(seenArgs, ['猫', '猫', 'ねこ', 500, 25]);
|
||||
assert.deepEqual(seenArgs, ['猫', '猫', 'ねこ', 500, 25, false]);
|
||||
});
|
||||
|
||||
it('GET /api/stats/kanji/occurrences returns recent occurrence rows for a kanji', async () => {
|
||||
@@ -1056,6 +1136,33 @@ describe('stats server API routes', () => {
|
||||
assert.equal(res.status, 400);
|
||||
});
|
||||
|
||||
it('GET /api/stats/vocabulary/occurrences finds all forms of legacy words missing text', async () => {
|
||||
const seenHeadwords: unknown[] = [];
|
||||
const app = createStatsApp(
|
||||
createMockTracker({
|
||||
getWordOccurrences: async (...args: unknown[]) => {
|
||||
seenHeadwords.push(args[0]);
|
||||
return OCCURRENCES;
|
||||
},
|
||||
}),
|
||||
);
|
||||
const request = (params: string) =>
|
||||
app.request(`/api/stats/vocabulary/occurrences?allForms=true&${params}`);
|
||||
|
||||
assert.equal((await request('headword=&word=%E7%8C%AB&reading=')).status, 200);
|
||||
assert.equal((await request('headword=%E7%8A%AC&word=&reading=')).status, 200);
|
||||
assert.equal((await request('headword=&word=&reading=%E3%81%AD%E3%81%93')).status, 200);
|
||||
assert.deepEqual(seenHeadwords, ['猫', '犬', 'ねこ']);
|
||||
|
||||
assert.equal((await request('headword=%20&word=&reading=')).status, 400);
|
||||
});
|
||||
|
||||
it('GET /api/stats/vocabulary/occurrences needs a headword and form for an exact lookup', async () => {
|
||||
const app = createStatsApp(createMockTracker());
|
||||
const res = await app.request('/api/stats/vocabulary/occurrences?headword=&word=%E7%8C%AB');
|
||||
assert.equal(res.status, 400);
|
||||
});
|
||||
|
||||
it('PUT /api/stats/excluded-words replaces tracker exclusion rows', async () => {
|
||||
let seenWords: unknown = null;
|
||||
const app = createStatsApp(
|
||||
|
||||
@@ -51,6 +51,16 @@ import {
|
||||
getSessionWordsByLine,
|
||||
} from './immersion-tracker/query-sessions';
|
||||
import { getTrendsDashboard } from './immersion-tracker/query-trends';
|
||||
import {
|
||||
getDictionaryWordPage,
|
||||
getVocabularyCoverage,
|
||||
} from './immersion-tracker/query-vocabulary-words';
|
||||
import type {
|
||||
StatsDayRange,
|
||||
StatsDictionaryWordQuery,
|
||||
StatsTrendGroupBy,
|
||||
StatsTrendRange,
|
||||
} from '../../types/stats-http-contract';
|
||||
import {
|
||||
getKanjiAnimeAppearances,
|
||||
getKanjiDetail,
|
||||
@@ -618,8 +628,11 @@ export class ImmersionTrackerService {
|
||||
this.db.close();
|
||||
}
|
||||
|
||||
async getSessionSummaries(limit = 50): Promise<SessionSummaryQueryRow[]> {
|
||||
return getSessionSummaries(this.db, limit);
|
||||
async getSessionSummaries(
|
||||
limit = 50,
|
||||
dayRange?: StatsDayRange,
|
||||
): Promise<SessionSummaryQueryRow[]> {
|
||||
return getSessionSummaries(this.db, limit, dayRange);
|
||||
}
|
||||
|
||||
async getSessionTimeline(sessionId: number, limit?: number): Promise<SessionTimelineRow[]> {
|
||||
@@ -673,11 +686,12 @@ export class ImmersionTrackerService {
|
||||
}
|
||||
|
||||
async getTrendsDashboard(
|
||||
range: '7d' | '30d' | '90d' | '365d' | 'all' = '30d',
|
||||
groupBy: 'day' | 'month' = 'day',
|
||||
range: StatsTrendRange = '30d',
|
||||
groupBy: StatsTrendGroupBy = 'day',
|
||||
fillEmptyBuckets = true,
|
||||
title: string | null = null,
|
||||
) {
|
||||
return getTrendsDashboard(this.db, range, groupBy, fillEmptyBuckets);
|
||||
return getTrendsDashboard(this.db, range, groupBy, fillEmptyBuckets, title);
|
||||
}
|
||||
|
||||
async getVocabularyStats(limit = 100, excludePos?: string[]): Promise<VocabularyStatsRow[]> {
|
||||
@@ -703,6 +717,17 @@ export class ImmersionTrackerService {
|
||||
return getVocabularyChartData(this.db);
|
||||
}
|
||||
|
||||
async getDictionaryWords(
|
||||
knownWords: ReadonlySet<string> | null,
|
||||
query: StatsDictionaryWordQuery,
|
||||
) {
|
||||
return getDictionaryWordPage(this.db, knownWords, query);
|
||||
}
|
||||
|
||||
async getVocabularyCoverage(knownWords: ReadonlySet<string>) {
|
||||
return getVocabularyCoverage(this.db, knownWords);
|
||||
}
|
||||
|
||||
async getStatsExcludedWords(): Promise<StatsExcludedWordRow[]> {
|
||||
return getStatsExcludedWords(this.db);
|
||||
}
|
||||
@@ -754,8 +779,9 @@ export class ImmersionTrackerService {
|
||||
reading: string,
|
||||
limit = 100,
|
||||
offset = 0,
|
||||
allForms = false,
|
||||
): Promise<WordOccurrenceRow[]> {
|
||||
return getWordOccurrences(this.db, headword, word, reading, limit, offset);
|
||||
return getWordOccurrences(this.db, headword, word, reading, limit, offset, allForms);
|
||||
}
|
||||
|
||||
async getKanjiOccurrences(kanji: string, limit = 100, offset = 0): Promise<KanjiOccurrenceRow[]> {
|
||||
|
||||
@@ -704,7 +704,7 @@ test('getTrendsDashboard returns chart-ready aggregated series', () => {
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`,
|
||||
).run(Math.floor(dayOneStart / 86_400_000), videoId, 1, 30, 10, 120, 2);
|
||||
).run(getLocalEpochDay(db, dayOneStart), videoId, 1, 30, 10, 120, 2);
|
||||
|
||||
db.prepare(
|
||||
`
|
||||
@@ -713,7 +713,7 @@ test('getTrendsDashboard returns chart-ready aggregated series', () => {
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`,
|
||||
).run(Math.floor(dayTwoStart / 86_400_000), videoId, 1, 45, 10, 140, 3);
|
||||
).run(getLocalEpochDay(db, dayTwoStart), videoId, 1, 45, 10, 140, 3);
|
||||
|
||||
db.prepare(
|
||||
`
|
||||
@@ -739,18 +739,17 @@ test('getTrendsDashboard returns chart-ready aggregated series', () => {
|
||||
assert.equal(dashboard.activity.watchTime[0]?.value, 30);
|
||||
assert.equal(dashboard.progress.watchTime[1]?.value, 75);
|
||||
assert.equal(dashboard.progress.lookups[1]?.value, 18);
|
||||
assert.equal(dashboard.ratios.lookupsPerHundred[0]?.value, +((8 / 120) * 100).toFixed(1));
|
||||
assert.equal(dashboard.ratios.cardsPerHour[0]?.value, +(2 / (30 / 60)).toFixed(1));
|
||||
assert.equal(dashboard.ratios.cardsPerHour[1]?.value, +(3 / (45 / 60)).toFixed(1));
|
||||
assert.equal(dashboard.ratios.cardsPerHour[1]?.value, +(5 / (75 / 60)).toFixed(1));
|
||||
assert.equal(dashboard.ratios.readingSpeed[0]?.value, +(120 / 30).toFixed(1));
|
||||
assert.equal(dashboard.ratios.readingSpeed[1]?.value, +(140 / 45).toFixed(1));
|
||||
assert.equal(dashboard.ratios.readingSpeed[1]?.value, +(260 / 75).toFixed(1));
|
||||
assert.equal(dashboard.librarySummary[0]?.title, 'Trend Dashboard Anime');
|
||||
assert.deepEqual(
|
||||
dashboard.animeDaily.watchTime.map((point) => point.value).sort((a, b) => a - b),
|
||||
dashboard.titleTimeline.map((point) => point.value).sort((a, b) => a - b),
|
||||
[30, 45],
|
||||
);
|
||||
assert.equal(
|
||||
dashboard.patterns.watchTimeByDayOfWeek.reduce((sum, point) => sum + point.value, 0),
|
||||
dashboard.watchTimeByDayHour.flat().reduce((sum, minutes) => sum + minutes, 0),
|
||||
75,
|
||||
);
|
||||
} finally {
|
||||
@@ -822,7 +821,7 @@ test('getTrendsDashboard redacts legacy Jellyfin stream titles', () => {
|
||||
|
||||
const dashboard = getTrendsDashboard(db, 'all', 'day');
|
||||
const titles = [
|
||||
...dashboard.animeDaily.watchTime.map((point) => point.animeTitle),
|
||||
...dashboard.titleTimeline.map((point) => point.animeTitle),
|
||||
...dashboard.librarySummary.map((row) => row.title),
|
||||
];
|
||||
|
||||
@@ -4751,7 +4750,7 @@ test('deleteSession removes zero-session media from library and trends', () => {
|
||||
|
||||
const trends = getTrendsDashboard(db, 'all', 'day');
|
||||
assert.deepEqual(trends.activity.watchTime, []);
|
||||
assert.deepEqual(trends.activity.sessions, []);
|
||||
assert.deepEqual(trends.activity.cards, []);
|
||||
|
||||
const dailyRollups = getDailyRollups(db, 30);
|
||||
const monthlyRollups = getMonthlyRollups(db, 30);
|
||||
@@ -5073,7 +5072,7 @@ test('getTrendsDashboard rollup word metrics keep persisted totals over partial
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`,
|
||||
).run(Math.floor(row.start / 86_400_000), videoId, 1, 10, 1, row.rawWords, 0);
|
||||
).run(getLocalEpochDay(db, row.start), videoId, 1, 10, 1, row.rawWords, 0);
|
||||
}
|
||||
|
||||
const dashboard = getTrendsDashboard(db, 'all', 'day');
|
||||
@@ -5085,13 +5084,8 @@ test('getTrendsDashboard rollup word metrics keep persisted totals over partial
|
||||
dashboard.progress.words.map((point) => point.value),
|
||||
[10, 30],
|
||||
);
|
||||
assert.equal(dashboard.ratios.lookupsPerHundred[0]?.value, 200);
|
||||
assert.equal(dashboard.librarySummary[0]?.words, 30);
|
||||
assert.equal(dashboard.librarySummary[0]?.lookupsPerHundred, 33.3);
|
||||
assert.equal(
|
||||
dashboard.animeDaily.words.reduce((sum, point) => sum + point.value, 0),
|
||||
30,
|
||||
);
|
||||
} finally {
|
||||
db.close();
|
||||
cleanupDbPath(dbPath);
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
getLexicalDailyRollups,
|
||||
rebuildLexicalDailyRollups,
|
||||
} from './lexical-rollups';
|
||||
import { getTrendsDashboard } from './query-trends';
|
||||
import {
|
||||
getVocabularyChartData,
|
||||
getVocabularySummary,
|
||||
@@ -262,7 +261,7 @@ test('lexical rollup rebuild tolerates nullable legacy vocabulary text', () => {
|
||||
assert.equal(areLexicalDailyRollupsReady(db), true);
|
||||
assert.equal(getVocabularySummary(db, null).uniqueWords, 1);
|
||||
assert.equal(getVocabularySummary(db, new Set(['猫'])).knownWordCount, 1);
|
||||
assert.equal(getVocabularyChartData(db).topWords[0]?.headword, '猫');
|
||||
assert.equal(getVocabularyChartData(db).newWordsTimeline[0]?.wordCount, 1);
|
||||
} finally {
|
||||
db.close();
|
||||
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
|
||||
@@ -593,7 +592,7 @@ test('imm_words persists vocabulary visibility for rollup maintenance', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('vocabulary charts use complete top-word and lexical rollup data', () => {
|
||||
test('vocabulary charts count every word in new-word history', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
@@ -611,8 +610,6 @@ test('vocabulary charts use complete top-word and lexical rollup data', () => {
|
||||
|
||||
const charts = getVocabularyChartData(db);
|
||||
|
||||
assert.equal(charts.topWords[0]?.headword, '語500');
|
||||
assert.equal(charts.topWords[0]?.frequency, 10_000);
|
||||
assert.equal(charts.newWordsTimeline[0]?.wordCount, 501);
|
||||
} finally {
|
||||
db.close();
|
||||
@@ -620,41 +617,6 @@ test('vocabulary charts use complete top-word and lexical rollup data', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('vocabulary charts find full top-word sets beyond excluded and name rows', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
const insertWord = db.prepare(
|
||||
`INSERT INTO imm_words(headword, word, reading, pos2, first_seen, last_seen, frequency)
|
||||
VALUES (?, ?, '', ?, 1700000000, 1700000000, ?)`,
|
||||
);
|
||||
const exclusions = [];
|
||||
for (let index = 0; index < 100; index += 1) {
|
||||
const headword = `語${index}`;
|
||||
insertWord.run(
|
||||
headword,
|
||||
headword,
|
||||
index < 80 && index >= 60 ? '固有名詞' : '一般',
|
||||
100 - index,
|
||||
);
|
||||
if (index < 60) exclusions.push({ headword, word: headword, reading: '' });
|
||||
}
|
||||
replaceStatsExcludedWords(db, exclusions);
|
||||
|
||||
const charts = getVocabularyChartData(db);
|
||||
|
||||
assert.equal(charts.topWords.length, 12);
|
||||
assert.equal(charts.topWords[0]?.headword, '語60');
|
||||
assert.equal(charts.topWordsWithoutNames.length, 12);
|
||||
assert.equal(charts.topWordsWithoutNames[0]?.headword, '語80');
|
||||
} finally {
|
||||
db.close();
|
||||
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('vocabulary charts handle exclusion lists above one SQLite variable batch', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
@@ -674,7 +636,6 @@ test('vocabulary charts handle exclusion lists above one SQLite variable batch',
|
||||
|
||||
const charts = getVocabularyChartData(db);
|
||||
|
||||
assert.deepEqual(charts.topWords, []);
|
||||
assert.deepEqual(charts.newWordsTimeline, []);
|
||||
} finally {
|
||||
db.close();
|
||||
@@ -697,24 +658,3 @@ test('lexical rollup rebuild preserves the original error when rollback also fai
|
||||
|
||||
assert.throws(() => rebuildLexicalDailyRollups(db), originalError);
|
||||
});
|
||||
|
||||
test('trends read historical new-word buckets from lexical rollups', () => {
|
||||
const dbPath = makeDbPath();
|
||||
const db = new Database(dbPath);
|
||||
|
||||
try {
|
||||
ensureSchema(db);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_words(headword, word, reading, first_seen, last_seen, frequency)
|
||||
VALUES ('海', '海', 'うみ', 1700000000, 1700000000, 1)`,
|
||||
).run();
|
||||
db.prepare(`UPDATE imm_lexical_daily_rollups SET word_count = 9`).run();
|
||||
|
||||
const dashboard = getTrendsDashboard(db, 'all', 'day', false);
|
||||
|
||||
assert.equal(dashboard.progress.newWords[0]?.value, 9);
|
||||
} finally {
|
||||
db.close();
|
||||
fs.rmSync(path.dirname(dbPath), { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { TrendFrequencyBand } from '../../../types/stats-wire';
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import type {
|
||||
KanjiAnimeAppearanceRow,
|
||||
@@ -19,17 +20,44 @@ import type {
|
||||
import { forEachIdChunk, fromDbTimestamp, makePlaceholders, toDbTimestamp } from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
import { narrowFullwidthKana, widenHalfwidthKana } from '../../utils/halfwidth-kana';
|
||||
import {
|
||||
areLexicalDailyRollupsReady,
|
||||
getLexicalDailyRollups,
|
||||
localEpochDaySql,
|
||||
} from './lexical-rollups';
|
||||
import { areLexicalDailyRollupsReady, localEpochDaySql } from './lexical-rollups';
|
||||
import { isVocabularyStatsRowVisible } from './vocabulary-visibility';
|
||||
|
||||
// Merged tokens join MeCab tags, so a name can be stored as 固有名詞|一般 or 固有名詞|接尾.
|
||||
const PROPER_NOUN_TAG = '固有名詞';
|
||||
const NOT_A_NAME_SQL = `COALESCE(pos2, '') NOT LIKE '%${PROPER_NOUN_TAG}%'`;
|
||||
|
||||
/** The dictionary word an `imm_words` row belongs to. Legacy rows can lack a headword or form. */
|
||||
export const DICTIONARY_WORD_KEY_SQL = `COALESCE(NULLIF(TRIM(headword), ''), NULLIF(TRIM(word), ''), TRIM(reading))`;
|
||||
|
||||
/** True for lines from a stream: only local videos have a path to cut clips from. */
|
||||
export const NO_LOCAL_FILE_SQL = `TRIM(COALESCE(v.source_path, '')) = ''`;
|
||||
|
||||
// Sorts lines that can be mined ahead of those that can't, because newest-first alone
|
||||
// lets a run of recent streams bury every line with a usable clip.
|
||||
// Mirrors getStatsMineCardUnavailableReason in the stats client.
|
||||
const MINEABLE_LINES_FIRST_SQL = `
|
||||
CASE
|
||||
WHEN NOT (${NO_LOCAL_FILE_SQL})
|
||||
AND l.segment_start_ms IS NOT NULL
|
||||
AND l.segment_end_ms > l.segment_start_ms
|
||||
THEN 0
|
||||
ELSE 1
|
||||
END`;
|
||||
|
||||
export function isProperNounPos2(pos2: string | null | undefined): boolean {
|
||||
return pos2?.includes(PROPER_NOUN_TAG) ?? false;
|
||||
}
|
||||
|
||||
export interface NewWordFirstSeenRow {
|
||||
firstSeenDay: number;
|
||||
firstSeenDayWithoutNames: number | null;
|
||||
band: TrendFrequencyBand;
|
||||
wordCount: number;
|
||||
}
|
||||
|
||||
const VOCABULARY_STATS_FILTER_OVERSAMPLE_FACTOR = 4;
|
||||
const VOCABULARY_STATS_FILTER_OVERSAMPLE_MIN = 100;
|
||||
const VOCABULARY_CHART_LIMIT = 12;
|
||||
const VOCABULARY_CHART_PAGE_SIZE = 100;
|
||||
const EXCLUSION_ALIAS_BATCH_SIZE = 300;
|
||||
const VOCABULARY_SUMMARY_SCAN_BATCH_SIZE = 5_000;
|
||||
const SENTENCE_SEARCH_DEFAULT_LIMIT = 50;
|
||||
@@ -38,8 +66,6 @@ const KANJI_PATTERN = /\p{Script=Han}/gu;
|
||||
|
||||
export interface VocabularyChartData {
|
||||
ready: boolean;
|
||||
topWords: Array<{ wordId: number; headword: string; frequency: number }>;
|
||||
topWordsWithoutNames: Array<{ wordId: number; headword: string; frequency: number }>;
|
||||
newWordsTimeline: Array<{ epochDay: number; wordCount: number }>;
|
||||
newWordsTimelineWithoutNames: Array<{ epochDay: number; wordCount: number }>;
|
||||
}
|
||||
@@ -161,111 +187,137 @@ function getWordAnimeCounts(db: DatabaseSync, wordIds: number[]): Map<number, nu
|
||||
}
|
||||
|
||||
/**
|
||||
* Chart data is intentionally independent of the paginated vocabulary tables.
|
||||
* Top words use the frequency index; new-word history reads permanent daily
|
||||
* lexical rollups rather than loading every vocabulary row into the dashboard.
|
||||
* New-word history is a SQL aggregate over first-seen dates rather than loading
|
||||
* every vocabulary row into the dashboard.
|
||||
*/
|
||||
export function getVocabularyChartData(db: DatabaseSync): VocabularyChartData {
|
||||
const ready = areLexicalDailyRollupsReady(db);
|
||||
const excludedAliases = new Set(
|
||||
getStatsExcludedWords(db).flatMap((word) => excludedVocabularyAliases(word)),
|
||||
);
|
||||
const isExcluded = (word: Pick<VocabularyStatsRow, 'headword' | 'word' | 'reading'>): boolean =>
|
||||
excludedVocabularyAliases(word).some((alias) => excludedAliases.has(alias));
|
||||
const topWords = getTopVocabularyChartWords(db, isExcluded);
|
||||
const rollups = ready ? getLexicalDailyRollups(db) : [];
|
||||
const timeline = new Map(rollups.map((row) => [row.epochDay, { ...row }]));
|
||||
if (excludedAliases.size > 0 && ready) {
|
||||
const aliases = [...excludedAliases];
|
||||
const excludedRows = new Map<
|
||||
number,
|
||||
Pick<VocabularyStatsRow, 'headword' | 'word' | 'reading' | 'pos2'> & {
|
||||
wordId: number;
|
||||
epochDay: number;
|
||||
}
|
||||
>();
|
||||
for (let offset = 0; offset < aliases.length; offset += EXCLUSION_ALIAS_BATCH_SIZE) {
|
||||
const batch = aliases.slice(offset, offset + EXCLUSION_ALIAS_BATCH_SIZE);
|
||||
const placeholders = batch.map(() => '?').join(', ');
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id AS wordId, headword, word, reading, pos2,
|
||||
${localEpochDaySql('first_seen')} AS epochDay
|
||||
FROM imm_words
|
||||
WHERE vocabulary_visible = 1
|
||||
AND (headword IN (${placeholders}) OR word IN (${placeholders}) OR reading IN (${placeholders}))
|
||||
`,
|
||||
)
|
||||
.all(...batch, ...batch, ...batch) as Array<
|
||||
Pick<VocabularyStatsRow, 'headword' | 'word' | 'reading' | 'pos2'> & {
|
||||
wordId: number;
|
||||
epochDay: number;
|
||||
}
|
||||
>;
|
||||
for (const row of rows) excludedRows.set(row.wordId, row);
|
||||
}
|
||||
for (const word of excludedRows.values()) {
|
||||
if (!isExcluded(word)) continue;
|
||||
const rollup = timeline.get(word.epochDay);
|
||||
if (!rollup) continue;
|
||||
rollup.wordCount -= 1;
|
||||
if (word.pos2 !== '固有名詞') rollup.wordCountWithoutNames -= 1;
|
||||
const firstSeenRows = ready ? getNewWordFirstSeenRows(db) : [];
|
||||
const timeline = new Map<number, number>();
|
||||
const timelineWithoutNames = new Map<number, number>();
|
||||
for (const row of firstSeenRows) {
|
||||
timeline.set(row.firstSeenDay, (timeline.get(row.firstSeenDay) ?? 0) + row.wordCount);
|
||||
if (row.firstSeenDayWithoutNames !== null) {
|
||||
timelineWithoutNames.set(
|
||||
row.firstSeenDayWithoutNames,
|
||||
(timelineWithoutNames.get(row.firstSeenDayWithoutNames) ?? 0) + row.wordCount,
|
||||
);
|
||||
}
|
||||
}
|
||||
return {
|
||||
ready,
|
||||
topWords: topWords.all.map((word) => ({
|
||||
wordId: word.wordId,
|
||||
headword: vocabularyDisplayHeadword(word),
|
||||
frequency: word.frequency,
|
||||
})),
|
||||
topWordsWithoutNames: topWords.withoutNames.map((word) => ({
|
||||
wordId: word.wordId,
|
||||
headword: vocabularyDisplayHeadword(word),
|
||||
frequency: word.frequency,
|
||||
})),
|
||||
newWordsTimeline: [...timeline.values()]
|
||||
.filter((row) => row.wordCount > 0)
|
||||
.map((row) => ({ epochDay: row.epochDay, wordCount: row.wordCount })),
|
||||
newWordsTimelineWithoutNames: [...timeline.values()]
|
||||
.filter((row) => row.wordCountWithoutNames > 0)
|
||||
.map((row) => ({ epochDay: row.epochDay, wordCount: row.wordCountWithoutNames })),
|
||||
newWordsTimeline: toNewWordTimeline(timeline),
|
||||
newWordsTimelineWithoutNames: toNewWordTimeline(timelineWithoutNames),
|
||||
};
|
||||
}
|
||||
|
||||
function getTopVocabularyChartWords(
|
||||
db: DatabaseSync,
|
||||
isExcluded: (word: Pick<VocabularyStatsRow, 'headword' | 'word' | 'reading'>) => boolean,
|
||||
): { all: VocabularyStatsRow[]; withoutNames: VocabularyStatsRow[] } {
|
||||
const stmt = db.prepare(`
|
||||
SELECT id AS wordId, headword, word, reading,
|
||||
part_of_speech AS partOfSpeech, pos1, pos2, pos3,
|
||||
frequency, frequency_rank AS frequencyRank,
|
||||
first_seen AS firstSeen, last_seen AS lastSeen,
|
||||
0 AS animeCount
|
||||
FROM imm_words
|
||||
ORDER BY frequency DESC, id
|
||||
LIMIT ? OFFSET ?
|
||||
`);
|
||||
const all: VocabularyStatsRow[] = [];
|
||||
const withoutNames: VocabularyStatsRow[] = [];
|
||||
let offset = 0;
|
||||
function toNewWordTimeline(byDay: Map<number, number>): VocabularyChartData['newWordsTimeline'] {
|
||||
return [...byDay.entries()]
|
||||
.sort(([left], [right]) => left - right)
|
||||
.map(([epochDay, wordCount]) => ({ epochDay, wordCount }));
|
||||
}
|
||||
|
||||
while (all.length < VOCABULARY_CHART_LIMIT || withoutNames.length < VOCABULARY_CHART_LIMIT) {
|
||||
const page = stmt.all(VOCABULARY_CHART_PAGE_SIZE, offset) as VocabularyStatsRow[];
|
||||
if (page.length === 0) break;
|
||||
for (const word of page) {
|
||||
if (!isVocabularyStatsRowVisible(word) || isExcluded(word)) continue;
|
||||
if (all.length < VOCABULARY_CHART_LIMIT) all.push(word);
|
||||
if (word.pos2 !== '固有名詞' && withoutNames.length < VOCABULARY_CHART_LIMIT) {
|
||||
withoutNames.push(word);
|
||||
/** Word rows matching the user's stats exclusions, using the same aliases as the summary. */
|
||||
export function getExcludedWordIds(db: DatabaseSync): number[] {
|
||||
const excludedAliases = new Set(
|
||||
getStatsExcludedWords(db).flatMap((word) => excludedVocabularyAliases(word)),
|
||||
);
|
||||
const aliases = [...excludedAliases];
|
||||
const ids = new Set<number>();
|
||||
for (let offset = 0; offset < aliases.length; offset += EXCLUSION_ALIAS_BATCH_SIZE) {
|
||||
const batch = aliases.slice(offset, offset + EXCLUSION_ALIAS_BATCH_SIZE);
|
||||
const placeholders = batch.map(() => '?').join(', ');
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id AS wordId, headword, word, reading
|
||||
FROM imm_words
|
||||
WHERE vocabulary_visible = 1
|
||||
AND (headword IN (${placeholders}) OR word IN (${placeholders}) OR reading IN (${placeholders}))
|
||||
`,
|
||||
)
|
||||
.all(...batch, ...batch, ...batch) as Array<
|
||||
Pick<VocabularyStatsRow, 'headword' | 'word' | 'reading'> & { wordId: number }
|
||||
>;
|
||||
for (const row of rows) {
|
||||
if (excludedVocabularyAliases(row).some((alias) => excludedAliases.has(alias))) {
|
||||
ids.add(row.wordId);
|
||||
}
|
||||
}
|
||||
offset += page.length;
|
||||
}
|
||||
return [...ids];
|
||||
}
|
||||
|
||||
return { all, withoutNames };
|
||||
// The aggregate below scans every word (about 200ms on 68k rows), and the Trends tab
|
||||
// runs it on each control change. Results stay valid until something writes:
|
||||
// total_changes covers this connection, data_version covers other connections.
|
||||
const newWordFirstSeenCache = new WeakMap<
|
||||
DatabaseSync,
|
||||
{ version: string; rows: NewWordFirstSeenRow[] }
|
||||
>();
|
||||
|
||||
export function databaseWriteVersion(db: DatabaseSync): string {
|
||||
const { changes } = db.prepare('SELECT total_changes() AS changes').get() as { changes: number };
|
||||
const { data_version: dataVersion } = db.prepare('PRAGMA data_version').get() as {
|
||||
data_version: number;
|
||||
};
|
||||
return `${changes}:${dataVersion}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* When each dictionary word was first seen, folded across its surface forms:
|
||||
* `imm_words` stores 言う and 言った as separate rows, so counting rows would count
|
||||
* conjugations as new words. Rows group identical (day, day without names, band)
|
||||
* results so the caller gets a small aggregate rather than one row per word.
|
||||
* `firstSeenDayWithoutNames` and `band` ignore proper-noun forms, and the day is
|
||||
* null when every form of the word is a proper noun.
|
||||
*/
|
||||
export function getNewWordFirstSeenRows(db: DatabaseSync): NewWordFirstSeenRow[] {
|
||||
const version = databaseWriteVersion(db);
|
||||
const cached = newWordFirstSeenCache.get(db);
|
||||
if (cached?.version === version) return cached.rows;
|
||||
|
||||
const excludedIds = getExcludedWordIds(db);
|
||||
const exclusion =
|
||||
excludedIds.length > 0 ? `AND id NOT IN (${makePlaceholders(excludedIds)})` : '';
|
||||
const seconds = `CASE
|
||||
WHEN ABS(CAST(first_seen AS REAL)) >= 10000000000 THEN CAST(first_seen AS REAL) / 1000
|
||||
ELSE CAST(first_seen AS REAL)
|
||||
END`;
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
WITH dictionary_words AS (
|
||||
SELECT
|
||||
MIN(${seconds}) AS firstSeen,
|
||||
MIN(CASE WHEN ${NOT_A_NAME_SQL} THEN ${seconds} END) AS firstSeenWithoutNames,
|
||||
MIN(CASE WHEN ${NOT_A_NAME_SQL} THEN frequency_rank END) AS rank
|
||||
FROM imm_words
|
||||
WHERE vocabulary_visible = 1 AND first_seen IS NOT NULL ${exclusion}
|
||||
GROUP BY ${DICTIONARY_WORD_KEY_SQL}
|
||||
)
|
||||
SELECT
|
||||
${localEpochDaySql('firstSeen')} AS firstSeenDay,
|
||||
CASE
|
||||
WHEN firstSeenWithoutNames IS NULL THEN NULL
|
||||
ELSE ${localEpochDaySql('firstSeenWithoutNames')}
|
||||
END AS firstSeenDayWithoutNames,
|
||||
CASE
|
||||
WHEN rank IS NULL THEN 'unranked'
|
||||
WHEN rank <= 1000 THEN 'top1k'
|
||||
WHEN rank <= 5000 THEN 'top5k'
|
||||
WHEN rank <= 10000 THEN 'top10k'
|
||||
WHEN rank <= 20000 THEN 'top20k'
|
||||
ELSE 'beyond20k'
|
||||
END AS band,
|
||||
COUNT(*) AS wordCount
|
||||
FROM dictionary_words
|
||||
GROUP BY 1, 2, 3
|
||||
`,
|
||||
)
|
||||
.all(...excludedIds) as unknown as NewWordFirstSeenRow[];
|
||||
newWordFirstSeenCache.set(db, { version, rows });
|
||||
return rows;
|
||||
}
|
||||
|
||||
function excludedVocabularyAliases(
|
||||
@@ -324,6 +376,12 @@ export function getVocabularySummary(
|
||||
knownWordCount: knownWords ? 0 : null,
|
||||
knownWordCountWithoutNames: knownWords ? 0 : null,
|
||||
};
|
||||
// Keyed by dictionary form, so a word's conjugations count once and it is new
|
||||
// this week only if no form of it was seen earlier.
|
||||
const dictionaryWords = new Map<
|
||||
string,
|
||||
{ firstSeenSec: number; firstSeenSecWithoutNames: number | null; known: boolean }
|
||||
>();
|
||||
|
||||
let lastId = Number.MIN_SAFE_INTEGER;
|
||||
for (;;) {
|
||||
@@ -337,23 +395,40 @@ export function getVocabularySummary(
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const isName = word.pos2 === '固有名詞';
|
||||
const isNewThisWeek = timestampSeconds(fromDbTimestamp(word.firstSeen) ?? 0) >= weekAgoSec;
|
||||
const isKnown = knownWords?.has(vocabularyDisplayHeadword(word)) ?? false;
|
||||
summary.uniqueWords += 1;
|
||||
if (!isName) summary.uniqueWordsWithoutNames += 1;
|
||||
if (isNewThisWeek) {
|
||||
summary.newThisWeek += 1;
|
||||
if (!isName) summary.newThisWeekWithoutNames += 1;
|
||||
}
|
||||
if (isKnown) {
|
||||
summary.knownWordCount! += 1;
|
||||
if (!isName) summary.knownWordCountWithoutNames! += 1;
|
||||
const firstSeenSec = timestampSeconds(fromDbTimestamp(word.firstSeen) ?? 0);
|
||||
const key = vocabularyDisplayHeadword(word);
|
||||
const entry = dictionaryWords.get(key) ?? {
|
||||
firstSeenSec,
|
||||
firstSeenSecWithoutNames: null,
|
||||
known: false,
|
||||
};
|
||||
entry.known ||= (knownWords?.has(key) ?? false) || (knownWords?.has(word.word) ?? false);
|
||||
entry.firstSeenSec = Math.min(entry.firstSeenSec, firstSeenSec);
|
||||
if (!isProperNounPos2(word.pos2)) {
|
||||
entry.firstSeenSecWithoutNames = Math.min(
|
||||
entry.firstSeenSecWithoutNames ?? firstSeenSec,
|
||||
firstSeenSec,
|
||||
);
|
||||
}
|
||||
dictionaryWords.set(key, entry);
|
||||
}
|
||||
if (words.length < scanBatchSize) break;
|
||||
}
|
||||
|
||||
for (const { firstSeenSec, firstSeenSecWithoutNames, known } of dictionaryWords.values()) {
|
||||
const hasNonNameForm = firstSeenSecWithoutNames !== null;
|
||||
summary.uniqueWords += 1;
|
||||
if (hasNonNameForm) summary.uniqueWordsWithoutNames += 1;
|
||||
if (firstSeenSec >= weekAgoSec) summary.newThisWeek += 1;
|
||||
if (hasNonNameForm && firstSeenSecWithoutNames >= weekAgoSec) {
|
||||
summary.newThisWeekWithoutNames += 1;
|
||||
}
|
||||
if (known) {
|
||||
summary.knownWordCount! += 1;
|
||||
if (hasNonNameForm) summary.knownWordCountWithoutNames! += 1;
|
||||
}
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
@@ -418,7 +493,12 @@ export function getWordOccurrences(
|
||||
reading: string,
|
||||
limit = 100,
|
||||
offset = 0,
|
||||
allForms = false,
|
||||
): WordOccurrenceRow[] {
|
||||
const wordFilter = allForms
|
||||
? `w.id IN (SELECT id FROM imm_words WHERE ${DICTIONARY_WORD_KEY_SQL} = ?)`
|
||||
: 'w.headword = ? AND w.word = ? AND w.reading = ?';
|
||||
const wordParams = allForms ? [headword] : [headword, word, reading];
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
@@ -440,13 +520,13 @@ export function getWordOccurrences(
|
||||
JOIN imm_subtitle_lines l ON l.line_id = o.line_id
|
||||
JOIN imm_videos v ON v.video_id = l.video_id
|
||||
LEFT JOIN imm_anime a ON a.anime_id = l.anime_id
|
||||
WHERE w.headword = ? AND w.word = ? AND w.reading = ?
|
||||
ORDER BY l.CREATED_DATE DESC, l.line_id DESC
|
||||
WHERE ${wordFilter}
|
||||
ORDER BY ${MINEABLE_LINES_FIRST_SQL}, l.CREATED_DATE DESC, l.line_id DESC
|
||||
LIMIT ?
|
||||
OFFSET ?
|
||||
`,
|
||||
)
|
||||
.all(headword, word, reading, limit, offset) as unknown as WordOccurrenceRow[];
|
||||
.all(...wordParams, limit, offset) as unknown as WordOccurrenceRow[];
|
||||
return rows.map(widenLineText);
|
||||
}
|
||||
|
||||
@@ -553,7 +633,7 @@ export function searchSubtitleSentences(
|
||||
JOIN imm_videos v ON v.video_id = l.video_id
|
||||
LEFT JOIN imm_anime a ON a.anime_id = l.anime_id
|
||||
WHERE ${clauses.join(' AND ')}
|
||||
ORDER BY l.CREATED_DATE DESC, l.line_id DESC
|
||||
ORDER BY ${MINEABLE_LINES_FIRST_SQL}, l.CREATED_DATE DESC, l.line_id DESC
|
||||
LIMIT ?
|
||||
`,
|
||||
)
|
||||
@@ -598,17 +678,99 @@ export function getSessionEvents(
|
||||
}));
|
||||
}
|
||||
|
||||
const WORD_DETAIL_FORM_LIMIT = 30;
|
||||
|
||||
/**
|
||||
* One entry per written form. A form can be stored more than once when its reading
|
||||
* was recorded differently, as 見て is under both みて and み. Returns most seen first.
|
||||
*/
|
||||
export function mergeFormsByWord<T extends { word: string; seen: number }>(forms: T[]): T[] {
|
||||
const byWord = new Map<string, T>();
|
||||
for (const form of forms) {
|
||||
const existing = byWord.get(form.word);
|
||||
if (existing) {
|
||||
existing.seen += form.seen;
|
||||
} else {
|
||||
byWord.set(form.word, { ...form });
|
||||
}
|
||||
}
|
||||
return [...byWord.values()].sort((left, right) => right.seen - left.seen);
|
||||
}
|
||||
|
||||
/** A word's form plus totals across every form of its dictionary word. */
|
||||
export function getWordDetail(db: DatabaseSync, wordId: number): WordDetailRow | null {
|
||||
return db
|
||||
const word = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id AS wordId, headword, word, reading,
|
||||
part_of_speech AS partOfSpeech, pos1, pos2, pos3,
|
||||
frequency, first_seen AS firstSeen, last_seen AS lastSeen
|
||||
SELECT id AS wordId, COALESCE(headword, '') AS headword, COALESCE(word, '') AS word,
|
||||
COALESCE(reading, '') AS reading,
|
||||
part_of_speech AS partOfSpeech, pos1, pos2, pos3
|
||||
FROM imm_words WHERE id = ?
|
||||
`,
|
||||
)
|
||||
.get(wordId) as WordDetailRow | null;
|
||||
.get(wordId) as Pick<
|
||||
WordDetailRow,
|
||||
'wordId' | 'headword' | 'word' | 'reading' | 'partOfSpeech' | 'pos1' | 'pos2' | 'pos3'
|
||||
> | null;
|
||||
if (!word) return null;
|
||||
|
||||
const hasHeadword = word.headword.trim() !== '';
|
||||
const forms = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id AS wordId, COALESCE(word, '') AS word, COALESCE(reading, '') AS reading,
|
||||
frequency AS seen, frequency_rank AS rank,
|
||||
first_seen AS firstSeen, last_seen AS lastSeen
|
||||
FROM imm_words
|
||||
WHERE ${hasHeadword ? '(headword = ? AND vocabulary_visible = 1) OR id = ?' : 'id = ?'}
|
||||
ORDER BY frequency DESC, id
|
||||
`,
|
||||
)
|
||||
.all(...(hasHeadword ? [word.headword, wordId] : [wordId])) as Array<{
|
||||
wordId: number;
|
||||
word: string;
|
||||
reading: string;
|
||||
seen: number | null;
|
||||
rank: number | null;
|
||||
firstSeen: number;
|
||||
lastSeen: number;
|
||||
}>;
|
||||
|
||||
const lines = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT COUNT(DISTINCT l.line_id) AS lineCount,
|
||||
COUNT(DISTINCT CASE WHEN ${NO_LOCAL_FILE_SQL} THEN l.line_id END) AS streamLineCount
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_words w ON w.id = o.word_id
|
||||
JOIN imm_subtitle_lines l ON l.line_id = o.line_id
|
||||
JOIN imm_videos v ON v.video_id = l.video_id
|
||||
WHERE ${hasHeadword ? 'w.headword = ?' : 'w.id = ?'}
|
||||
`,
|
||||
)
|
||||
.get(hasHeadword ? word.headword : wordId) as { lineCount: number; streamLineCount: number };
|
||||
const merged = mergeFormsByWord(forms.map((form) => ({ ...form, seen: form.seen ?? 0 })));
|
||||
const toSeconds = (value: number): number => timestampSeconds(fromDbTimestamp(value) ?? 0);
|
||||
const ranks = forms.map((form) => form.rank).filter((rank): rank is number => (rank ?? 0) > 0);
|
||||
return {
|
||||
...word,
|
||||
frequency: forms.reduce((sum, form) => sum + (form.seen ?? 0), 0),
|
||||
frequencyRank: ranks.length > 0 ? Math.min(...ranks) : null,
|
||||
dictionaryReading:
|
||||
forms.find((form) => form.word === word.headword && form.reading.trim() !== '')?.reading ??
|
||||
'',
|
||||
formCount: merged.length,
|
||||
forms: merged.slice(0, WORD_DETAIL_FORM_LIMIT).map((form) => ({
|
||||
wordId: form.wordId,
|
||||
word: form.word,
|
||||
reading: form.reading,
|
||||
seen: form.seen,
|
||||
})),
|
||||
lineCount: lines.lineCount,
|
||||
streamLineCount: lines.streamLineCount,
|
||||
firstSeen: Math.min(...forms.map((form) => toSeconds(form.firstSeen))),
|
||||
lastSeen: Math.max(...forms.map((form) => toSeconds(form.lastSeen))),
|
||||
};
|
||||
}
|
||||
|
||||
export function getWordAnimeAppearances(
|
||||
@@ -621,21 +783,26 @@ export function getWordAnimeAppearances(
|
||||
SELECT a.anime_id AS animeId, a.canonical_title AS animeTitle,
|
||||
SUM(o.occurrence_count) AS occurrenceCount
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_words w ON w.id = o.word_id
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
JOIN imm_anime a ON a.anime_id = sl.anime_id
|
||||
WHERE o.word_id = ? AND sl.anime_id IS NOT NULL
|
||||
WHERE (w.id = ? OR w.headword = (SELECT NULLIF(TRIM(headword), '') FROM imm_words WHERE id = ?))
|
||||
AND sl.anime_id IS NOT NULL
|
||||
GROUP BY a.anime_id
|
||||
ORDER BY occurrenceCount DESC
|
||||
`,
|
||||
)
|
||||
.all(wordId) as WordAnimeAppearanceRow[];
|
||||
.all(wordId, wordId) as WordAnimeAppearanceRow[];
|
||||
}
|
||||
|
||||
/** Other dictionary words sharing a reading or kanji; a word's own forms don't count. */
|
||||
export function getSimilarWords(db: DatabaseSync, wordId: number, limit = 10): SimilarWordRow[] {
|
||||
const word = db.prepare('SELECT headword, reading FROM imm_words WHERE id = ?').get(wordId) as {
|
||||
headword: string;
|
||||
reading: string;
|
||||
} | null;
|
||||
const word = db
|
||||
.prepare(
|
||||
`SELECT COALESCE(headword, '') AS headword, COALESCE(reading, '') AS reading
|
||||
FROM imm_words WHERE id = ?`,
|
||||
)
|
||||
.get(wordId) as { headword: string; reading: string } | null;
|
||||
if (!word || word.headword.trim() === '') return [];
|
||||
|
||||
const clauses: string[] = [];
|
||||
@@ -653,22 +820,41 @@ export function getSimilarWords(db: DatabaseSync, wordId: number, limit = 10): S
|
||||
|
||||
if (clauses.length === 0) return [];
|
||||
|
||||
const orderBy =
|
||||
reading !== '' ? 'CASE WHEN reading = ? THEN 0 ELSE 1 END, frequency DESC' : 'frequency DESC';
|
||||
const orderParams = reading !== '' ? [word.reading] : [];
|
||||
const readingMatch =
|
||||
reading !== ''
|
||||
? 'MAX(CASE WHEN reading = ? THEN 1 ELSE 0 END) OVER (PARTITION BY headword)'
|
||||
: '0';
|
||||
const readingMatchParams = reading !== '' ? [word.reading] : [];
|
||||
|
||||
// One form matching is enough to relate a word, but its total and the form it shows
|
||||
// come from all its visible forms, so a rare form with the same reading can't stand in.
|
||||
// The blank check lists its characters because SQL TRIM alone strips only ASCII spaces.
|
||||
return db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id AS wordId, headword, word, reading, frequency
|
||||
FROM imm_words
|
||||
WHERE id != ?
|
||||
AND (${clauses.join(' OR ')})
|
||||
ORDER BY ${orderBy}
|
||||
WITH related AS (
|
||||
SELECT DISTINCT headword
|
||||
FROM imm_words
|
||||
WHERE headword != ? AND vocabulary_visible = 1
|
||||
AND TRIM(headword, ' ' || char(9, 10, 11, 12, 13, 12288)) != ''
|
||||
AND (${clauses.join(' OR ')})
|
||||
),
|
||||
forms AS (
|
||||
SELECT id, headword, word, reading,
|
||||
SUM(frequency) OVER (PARTITION BY headword) AS totalSeen,
|
||||
${readingMatch} AS readingMatch,
|
||||
ROW_NUMBER() OVER (PARTITION BY headword ORDER BY frequency DESC, id) AS formOrder
|
||||
FROM imm_words
|
||||
WHERE vocabulary_visible = 1 AND headword IN (SELECT headword FROM related)
|
||||
)
|
||||
SELECT id AS wordId, headword, word, reading, totalSeen AS frequency
|
||||
FROM forms
|
||||
WHERE formOrder = 1
|
||||
ORDER BY readingMatch DESC, totalSeen DESC, headword
|
||||
LIMIT ?
|
||||
`,
|
||||
)
|
||||
.all(wordId, ...params, ...orderParams, limit) as SimilarWordRow[];
|
||||
.all(word.headword, ...params, ...readingMatchParams, limit) as SimilarWordRow[];
|
||||
}
|
||||
|
||||
export function getKanjiDetail(db: DatabaseSync, kanjiId: number): KanjiDetailRow | null {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { StatsDayRange } from '../../../types/stats-http-contract';
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import type {
|
||||
ImmersionSessionRollupRow,
|
||||
@@ -15,7 +16,12 @@ import {
|
||||
visibleWordSql,
|
||||
} from './query-shared';
|
||||
|
||||
export function getSessionSummaries(db: DatabaseSync, limit = 50): SessionSummaryQueryRow[] {
|
||||
export function getSessionSummaries(
|
||||
db: DatabaseSync,
|
||||
limit = 50,
|
||||
dayRange?: StatsDayRange,
|
||||
): SessionSummaryQueryRow[] {
|
||||
const dayFilter = dayRange ? `WHERE ${sessionRollupDaySql('s')} BETWEEN ? AND ?` : '';
|
||||
const prepared = db.prepare(`
|
||||
${ACTIVE_SESSION_METRICS_CTE}
|
||||
SELECT
|
||||
@@ -38,10 +44,12 @@ export function getSessionSummaries(db: DatabaseSync, limit = 50): SessionSummar
|
||||
LEFT JOIN active_session_metrics asm ON asm.sessionId = s.session_id
|
||||
LEFT JOIN imm_videos v ON v.video_id = s.video_id
|
||||
LEFT JOIN imm_anime a ON a.anime_id = v.anime_id
|
||||
${dayFilter}
|
||||
ORDER BY s.started_at_ms DESC
|
||||
LIMIT ?
|
||||
`);
|
||||
const rows = prepared.all(limit) as Array<
|
||||
const params = dayRange ? [dayRange.startDay, dayRange.endDay, limit] : [limit];
|
||||
const rows = prepared.all(...params) as Array<
|
||||
SessionSummaryQueryRow & {
|
||||
startedAtMs: number | string;
|
||||
endedAtMs: number | string | null;
|
||||
@@ -393,6 +401,63 @@ export function selectDailyRollupRows(db: DatabaseSync, limit: number): RawRollu
|
||||
.all(limit) as unknown as RawRollupRow[];
|
||||
}
|
||||
|
||||
/** Raw daily rollup rows for an inclusive day span; a null start means no lower bound. */
|
||||
export function selectDailyRollupRowsBetween(
|
||||
db: DatabaseSync,
|
||||
startDay: number | null,
|
||||
endDay: number,
|
||||
): RawRollupRow[] {
|
||||
return db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
r.rollup_day AS rollupDayOrMonth,
|
||||
r.video_id AS videoId,
|
||||
r.total_sessions AS totalSessions,
|
||||
r.total_active_min AS totalActiveMin,
|
||||
r.total_lines_seen AS totalLinesSeen,
|
||||
r.total_tokens_seen AS totalTokensSeen,
|
||||
r.total_cards AS totalCards,
|
||||
r.cards_per_hour AS cardsPerHour,
|
||||
r.lookup_hit_rate AS lookupHitRate
|
||||
FROM imm_daily_rollups r
|
||||
WHERE r.rollup_day BETWEEN ? AND ?
|
||||
ORDER BY r.rollup_day DESC, r.video_id DESC
|
||||
`,
|
||||
)
|
||||
.all(startDay ?? Number.MIN_SAFE_INTEGER, endDay) as unknown as RawRollupRow[];
|
||||
}
|
||||
|
||||
/** Raw monthly rollup rows for an inclusive `YYYYMM` span; a null start means no lower bound. */
|
||||
export function selectMonthlyRollupRowsBetween(
|
||||
db: DatabaseSync,
|
||||
startMonth: number | null,
|
||||
endMonth: number,
|
||||
): RawRollupRow[] {
|
||||
return db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
r.rollup_month AS rollupDayOrMonth,
|
||||
r.video_id AS videoId,
|
||||
r.total_sessions AS totalSessions,
|
||||
r.total_active_min AS totalActiveMin,
|
||||
r.total_lines_seen AS totalLinesSeen,
|
||||
r.total_tokens_seen AS totalTokensSeen,
|
||||
r.total_cards AS totalCards,
|
||||
CASE
|
||||
WHEN r.total_active_min > 0 THEN (r.total_cards * 60.0) / r.total_active_min
|
||||
ELSE NULL
|
||||
END AS cardsPerHour,
|
||||
NULL AS lookupHitRate
|
||||
FROM imm_monthly_rollups r
|
||||
WHERE r.rollup_month BETWEEN ? AND ?
|
||||
ORDER BY r.rollup_month DESC, r.video_id DESC
|
||||
`,
|
||||
)
|
||||
.all(startMonth ?? Number.MIN_SAFE_INTEGER, endMonth) as unknown as RawRollupRow[];
|
||||
}
|
||||
|
||||
/** Monthly counterpart of `selectDailyRollupRows`. */
|
||||
export function selectMonthlyRollupRows(db: DatabaseSync, limit: number): RawRollupRow[] {
|
||||
return db
|
||||
|
||||
@@ -0,0 +1,541 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { Database, type DatabaseSync } from './sqlite';
|
||||
import { getOrCreateAnimeRecord, getOrCreateVideoRecord, linkVideoToAnimeRecord } from './storage';
|
||||
import { ensureSchema } from './storage';
|
||||
import { SOURCE_TYPE_LOCAL } from './types';
|
||||
import { getTrendsDashboard } from './query-trends';
|
||||
import { getSessionSummaries } from './query-sessions';
|
||||
import { getVocabularyChartData, getVocabularySummary } from './query-lexical';
|
||||
import { markLexicalDailyRollupsReady } from './lexical-rollups';
|
||||
|
||||
/** Noon, local time, on a calendar day, so the local epoch day holds in every time zone. */
|
||||
function localNoonMs(epochDay: number): number {
|
||||
const date = new Date(epochDay * 86_400_000);
|
||||
return new Date(date.getUTCFullYear(), date.getUTCMonth(), date.getUTCDate(), 12).getTime();
|
||||
}
|
||||
|
||||
// Mocked "now" is noon on 2026-03-01, a Sunday.
|
||||
const TODAY = Date.UTC(2026, 2, 1) / 86_400_000;
|
||||
const NOW_MS = String(localNoonMs(TODAY));
|
||||
|
||||
function withDashboardDb(run: (db: DatabaseSync) => void): void {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-trends-'));
|
||||
const db = new Database(path.join(dir, 'immersion.sqlite'));
|
||||
const previousNow = globalThis.__subminerTestNowMs;
|
||||
globalThis.__subminerTestNowMs = NOW_MS;
|
||||
try {
|
||||
ensureSchema(db);
|
||||
run(db);
|
||||
} finally {
|
||||
globalThis.__subminerTestNowMs = previousNow;
|
||||
db.close();
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function addVideo(db: DatabaseSync, title: string, episode: number, watched = false): number {
|
||||
const key = `${title}-${episode}`;
|
||||
const videoId = getOrCreateVideoRecord(db, `local:/tmp/${key}.mkv`, {
|
||||
canonicalTitle: key,
|
||||
sourcePath: `/tmp/${key}.mkv`,
|
||||
sourceUrl: null,
|
||||
sourceType: SOURCE_TYPE_LOCAL,
|
||||
});
|
||||
const animeId = getOrCreateAnimeRecord(db, {
|
||||
parsedTitle: title,
|
||||
canonicalTitle: title,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: null,
|
||||
});
|
||||
linkVideoToAnimeRecord(db, videoId, {
|
||||
animeId,
|
||||
parsedBasename: `${key}.mkv`,
|
||||
parsedTitle: title,
|
||||
parsedSeason: 1,
|
||||
parsedEpisode: episode,
|
||||
parserSource: 'test',
|
||||
parserConfidence: 1,
|
||||
parseMetadataJson: null,
|
||||
});
|
||||
if (watched) db.prepare('UPDATE imm_videos SET watched = 1 WHERE video_id = ?').run(videoId);
|
||||
return videoId;
|
||||
}
|
||||
|
||||
function addRollup(
|
||||
db: DatabaseSync,
|
||||
day: number,
|
||||
videoId: number,
|
||||
values: { minutes?: number; words?: number; cards?: number } = {},
|
||||
): void {
|
||||
db.prepare(
|
||||
`INSERT INTO imm_daily_rollups (
|
||||
rollup_day, video_id, total_sessions, total_active_min, total_lines_seen,
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, 1, ?, 1, ?, ?)`,
|
||||
).run(day, videoId, values.minutes ?? 30, values.words ?? 100, values.cards ?? 0);
|
||||
}
|
||||
|
||||
function addSession(
|
||||
db: DatabaseSync,
|
||||
day: number,
|
||||
videoId: number,
|
||||
values: { minutes?: number; lookups?: number } = {},
|
||||
): void {
|
||||
const startedAtMs = localNoonMs(day).toString();
|
||||
const activeMs = (values.minutes ?? 30) * 60_000;
|
||||
db.prepare(
|
||||
`INSERT INTO imm_sessions (
|
||||
session_uuid, video_id, started_at_ms, ended_at_ms, status, active_watched_ms,
|
||||
total_watched_ms, yomitan_lookup_count
|
||||
) VALUES (?, ?, ?, ?, 2, ?, ?, ?)`,
|
||||
).run(
|
||||
`session-${day}-${videoId}-${Math.random()}`,
|
||||
videoId,
|
||||
startedAtMs,
|
||||
(BigInt(startedAtMs) + BigInt(activeMs)).toString(),
|
||||
activeMs,
|
||||
activeMs,
|
||||
values.lookups ?? 0,
|
||||
);
|
||||
}
|
||||
|
||||
function addWord(
|
||||
db: DatabaseSync,
|
||||
values: {
|
||||
headword: string;
|
||||
word?: string;
|
||||
day: number;
|
||||
pos2?: string | null;
|
||||
rank?: number | null;
|
||||
},
|
||||
): void {
|
||||
const firstSeen = localNoonMs(values.day) / 1000;
|
||||
db.prepare(
|
||||
`INSERT INTO imm_words (
|
||||
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen,
|
||||
frequency, frequency_rank
|
||||
) VALUES (?, ?, ?, 'noun', '名詞', ?, '', ?, ?, 1, ?)`,
|
||||
).run(
|
||||
values.headword,
|
||||
values.word ?? values.headword,
|
||||
values.word ?? values.headword,
|
||||
values.pos2 ?? null,
|
||||
firstSeen,
|
||||
firstSeen,
|
||||
values.rank ?? null,
|
||||
);
|
||||
}
|
||||
|
||||
test('the title lists stay inside the calendar range when activity is sparse', () => {
|
||||
withDashboardDb((db) => {
|
||||
const recent = addVideo(db, 'Recent Show', 1);
|
||||
const old = addVideo(db, 'Old Show', 1);
|
||||
addRollup(db, TODAY, recent, { minutes: 20 });
|
||||
// Outside a 7-day calendar window, though well within the last 7 active days.
|
||||
addRollup(db, TODAY - 40, old, { minutes: 90 });
|
||||
addSession(db, TODAY - 40, old);
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day');
|
||||
|
||||
assert.deepEqual(
|
||||
dashboard.librarySummary.map((row) => row.title),
|
||||
['Recent Show'],
|
||||
);
|
||||
assert.deepEqual(
|
||||
dashboard.titleTimeline.map((point) => point.animeTitle),
|
||||
['Recent Show'],
|
||||
);
|
||||
assert.equal(dashboard.summary.current.watchTimeMin, 20);
|
||||
assert.equal(
|
||||
dashboard.librarySummary.reduce((sum, row) => sum + row.watchTimeMin, 0),
|
||||
dashboard.activity.watchTime.reduce((sum, point) => sum + point.value, 0),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('every grouping reports the same totals for a range', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
for (const day of [TODAY, TODAY - 3, TODAY - 20, TODAY - 29, TODAY - 45]) {
|
||||
addRollup(db, day, videoId, { minutes: 10, cards: 1 });
|
||||
}
|
||||
// January is only partly in range, and its monthly total includes days that are not.
|
||||
const insertMonthly = db.prepare(
|
||||
`INSERT INTO imm_monthly_rollups (
|
||||
rollup_month, video_id, total_sessions, total_active_min, total_lines_seen,
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, 1, ?, 1, 100, 1)`,
|
||||
);
|
||||
insertMonthly.run(202601, videoId, 20);
|
||||
insertMonthly.run(202602, videoId, 20);
|
||||
insertMonthly.run(202603, videoId, 10);
|
||||
|
||||
const sum = (groupBy: 'day' | 'week' | 'month') =>
|
||||
getTrendsDashboard(db, '30d', groupBy).activity.watchTime.reduce(
|
||||
(total, point) => total + point.value,
|
||||
0,
|
||||
);
|
||||
|
||||
assert.equal(sum('day'), 40);
|
||||
assert.equal(sum('week'), 40);
|
||||
assert.equal(sum('month'), 40);
|
||||
});
|
||||
});
|
||||
|
||||
test('all-time month buckets use monthly rollups that outlive daily ones', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
const insertMonthly = db.prepare(
|
||||
`INSERT INTO imm_monthly_rollups (
|
||||
rollup_month, video_id, total_sessions, total_active_min, total_lines_seen,
|
||||
total_tokens_seen, total_cards
|
||||
) VALUES (?, ?, 3, ?, 1, 100, 50)`,
|
||||
);
|
||||
insertMonthly.run(202501, videoId, 500);
|
||||
insertMonthly.run(202503, videoId, 200);
|
||||
|
||||
assert.deepEqual(getTrendsDashboard(db, 'all', 'month').activity.watchTime, [
|
||||
{ label: 'Jan 2025', value: 500 },
|
||||
{ label: 'Feb 2025', value: 0 },
|
||||
{ label: 'Mar 2025', value: 200 },
|
||||
]);
|
||||
assert.deepEqual(getTrendsDashboard(db, 'all', 'month', false).activity.watchTime, [
|
||||
{ label: 'Jan 2025', value: 500 },
|
||||
{ label: 'Mar 2025', value: 200 },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
test('a finished video counts once however many days it was watched', () => {
|
||||
withDashboardDb((db) => {
|
||||
const finished = addVideo(db, 'Show', 1, true);
|
||||
const unfinished = addVideo(db, 'Show', 2, false);
|
||||
addRollup(db, TODAY - 2, finished);
|
||||
addRollup(db, TODAY - 1, finished);
|
||||
addRollup(db, TODAY, unfinished);
|
||||
|
||||
for (const groupBy of ['day', 'week', 'month'] as const) {
|
||||
const dashboard = getTrendsDashboard(db, '30d', groupBy);
|
||||
assert.equal(dashboard.progress.episodes.at(-1)?.value, 1, groupBy);
|
||||
assert.equal(dashboard.summary.current.episodes, 1, groupBy);
|
||||
}
|
||||
|
||||
const daily = getTrendsDashboard(db, '7d', 'day').activity.episodes;
|
||||
// Attributed to the last day it was watched.
|
||||
assert.equal(daily.at(-2)?.value, 1);
|
||||
assert.equal(daily.at(-3)?.value, 0);
|
||||
assert.equal(getTrendsDashboard(db, '30d', 'day').librarySummary[0]?.episodesFinished, 1);
|
||||
assert.equal(getTrendsDashboard(db, '30d', 'day').librarySummary[0]?.videos, 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('rates are gaps without enough data, not zeros', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
// A two-minute day with one card: 30 cards per hour if rated on its own.
|
||||
addRollup(db, TODAY - 20, videoId, { minutes: 2, words: 50, cards: 1 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '30d', 'day');
|
||||
|
||||
assert.equal(dashboard.ratios.cardsPerHour.at(-1)?.value, null);
|
||||
assert.equal(
|
||||
dashboard.ratios.cardsPerHour.every((point) => point.value === null),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
dashboard.ratios.readingSpeed.every((point) => point.value === null),
|
||||
true,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('daily rates are volume-weighted over a trailing week, reaching before the range', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
// Eight days before today: outside the 7d range, inside the first day's lookback.
|
||||
addRollup(db, TODAY - 8, videoId, { minutes: 60, words: 3_000, cards: 6 });
|
||||
addRollup(db, TODAY - 6, videoId, { minutes: 60, words: 1_000, cards: 2 });
|
||||
|
||||
const speed = getTrendsDashboard(db, '7d', 'day').ratios.readingSpeed;
|
||||
|
||||
// (3000 + 1000) words / (60 + 60) min, not an average of 50 and 16.7 ...
|
||||
assert.equal(speed[0]?.value, +(4_000 / 120).toFixed(1));
|
||||
// ... and once the older day leaves the trailing week only the second remains.
|
||||
assert.equal(speed[5]?.value, +(1_000 / 60).toFixed(1));
|
||||
});
|
||||
});
|
||||
|
||||
test('week grouping starts on Monday and clips the first bucket to the range', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY, videoId);
|
||||
|
||||
const { buckets } = getTrendsDashboard(db, '30d', 'week');
|
||||
|
||||
assert.equal(buckets[0]?.startDay, TODAY - 29);
|
||||
assert.equal(buckets.at(-1)?.endDay, TODAY);
|
||||
// Today is a Sunday, so the final bucket is a full Monday-to-Sunday week.
|
||||
assert.equal(buckets.at(-1)?.startDay, TODAY - 6);
|
||||
for (let index = 1; index < buckets.length; index += 1) {
|
||||
assert.equal(buckets[index]!.startDay, buckets[index - 1]!.endDay + 1);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test('showing empty periods on all-time fills the calendar between active days', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY - 10, videoId);
|
||||
addRollup(db, TODAY, videoId);
|
||||
|
||||
assert.equal(getTrendsDashboard(db, 'all', 'day', true).activity.watchTime.length, 11);
|
||||
assert.equal(getTrendsDashboard(db, 'all', 'day', false).activity.watchTime.length, 2);
|
||||
assert.deepEqual(getTrendsDashboard(db, 'all', 'day', true).window, {
|
||||
startDay: TODAY - 10,
|
||||
endDay: TODAY,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
test('month labels name the year, and day labels do when the range crosses one', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY, videoId);
|
||||
|
||||
assert.equal(getTrendsDashboard(db, '30d', 'month').buckets.at(-1)?.label, 'Mar 2026');
|
||||
assert.equal(getTrendsDashboard(db, '7d', 'day').buckets.at(-1)?.label, 'Mar 1');
|
||||
assert.equal(getTrendsDashboard(db, '90d', 'day').buckets.at(-1)?.label, 'Mar 1, 2026');
|
||||
});
|
||||
});
|
||||
|
||||
test('a custom range clips to its days and compares against the same-length span before it', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY - 5, videoId, { minutes: 10 });
|
||||
addRollup(db, TODAY - 15, videoId, { minutes: 20 });
|
||||
addRollup(db, TODAY - 25, videoId, { minutes: 40 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, { startDay: TODAY - 19, endDay: TODAY - 10 });
|
||||
|
||||
assert.equal(dashboard.activity.watchTime.length, 10);
|
||||
assert.equal(dashboard.summary.current.watchTimeMin, 20);
|
||||
assert.equal(dashboard.summary.previous?.watchTimeMin, 40);
|
||||
assert.deepEqual(dashboard.window, { startDay: TODAY - 19, endDay: TODAY - 10 });
|
||||
});
|
||||
});
|
||||
|
||||
test('summary compares with the previous period and all-time has no comparison', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY, videoId, { minutes: 10, cards: 1 });
|
||||
addRollup(db, TODAY - 10, videoId, { minutes: 25, cards: 4 });
|
||||
|
||||
const { current, previous } = getTrendsDashboard(db, '7d', 'day').summary;
|
||||
|
||||
assert.equal(current.watchTimeMin, 10);
|
||||
assert.equal(current.activeDays, 1);
|
||||
assert.equal(previous?.watchTimeMin, 25);
|
||||
assert.equal(previous?.cards, 4);
|
||||
assert.equal(getTrendsDashboard(db, 'all', 'day').summary.previous, null);
|
||||
});
|
||||
});
|
||||
|
||||
test('a title filter narrows the page but keeps the title lists complete', () => {
|
||||
withDashboardDb((db) => {
|
||||
const first = addVideo(db, 'First', 1);
|
||||
const second = addVideo(db, 'Second', 1);
|
||||
addRollup(db, TODAY, first, { minutes: 10 });
|
||||
addRollup(db, TODAY, second, { minutes: 50 });
|
||||
addSession(db, TODAY, first, { minutes: 10, lookups: 3 });
|
||||
addSession(db, TODAY, second, { minutes: 50, lookups: 9 });
|
||||
addWord(db, { headword: '海', day: TODAY });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day', true, 'First');
|
||||
|
||||
assert.equal(dashboard.summary.current.watchTimeMin, 10);
|
||||
assert.equal(dashboard.progress.lookups.at(-1)?.value, 3);
|
||||
assert.equal(
|
||||
dashboard.watchTimeByDayHour.flat().reduce((sum, value) => sum + value, 0),
|
||||
10,
|
||||
);
|
||||
assert.equal(dashboard.librarySummary.length, 2);
|
||||
assert.equal(dashboard.titleTimeline.length, 2);
|
||||
// New words are not tracked per title.
|
||||
assert.equal(dashboard.summary.current.newWords, null);
|
||||
assert.equal(dashboard.learning, null);
|
||||
assert.deepEqual(dashboard.progress.newWords, []);
|
||||
});
|
||||
});
|
||||
|
||||
test('running totals follow the activity charts when empty days are hidden', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY - 3, videoId, { minutes: 30 });
|
||||
addRollup(db, TODAY, videoId, { minutes: 10 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day', false);
|
||||
|
||||
assert.deepEqual(
|
||||
dashboard.progress.watchTime.map((point) => point.value),
|
||||
[30, 40],
|
||||
);
|
||||
assert.deepEqual(
|
||||
dashboard.progress.watchTime.map((point) => point.label),
|
||||
dashboard.activity.watchTime.map((point) => point.label),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('the watch time running total rounds its sum, so it ends on the summary total', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY - 2, videoId, { minutes: 10.4 });
|
||||
addRollup(db, TODAY - 1, videoId, { minutes: 10.4 });
|
||||
addRollup(db, TODAY, videoId, { minutes: 10.4 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day', false);
|
||||
|
||||
assert.deepEqual(
|
||||
dashboard.activity.watchTime.map((point) => point.value),
|
||||
[10, 10, 10],
|
||||
);
|
||||
assert.deepEqual(
|
||||
dashboard.progress.watchTime.map((point) => point.value),
|
||||
[10, 21, 31],
|
||||
);
|
||||
assert.equal(dashboard.summary.current.watchTimeMin, 31);
|
||||
});
|
||||
});
|
||||
|
||||
test('the summary adds watch time in date order, so a total near .5 rounds like the chart', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
// Newest first these add up to 4.499999999999999, and oldest first to 4.5.
|
||||
addRollup(db, TODAY - 2, videoId, { minutes: 0.1 });
|
||||
addRollup(db, TODAY - 1, videoId, { minutes: 0.1 });
|
||||
addRollup(db, TODAY, videoId, { minutes: 4.3 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day');
|
||||
|
||||
assert.equal(
|
||||
dashboard.summary.current.watchTimeMin,
|
||||
dashboard.progress.watchTime.at(-1)?.value,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('new words count a dictionary word once, on its first form, without names', () => {
|
||||
withDashboardDb((db) => {
|
||||
addWord(db, { headword: '言う', word: '言う', day: TODAY - 2, rank: 50 });
|
||||
addWord(db, { headword: '言う', word: '言った', day: TODAY, rank: 50 });
|
||||
addWord(db, { headword: '東京', day: TODAY - 1, pos2: '固有名詞', rank: 800 });
|
||||
|
||||
const dashboard = getTrendsDashboard(db, '7d', 'day');
|
||||
|
||||
assert.equal(dashboard.summary.current.newWords, 1);
|
||||
assert.equal(dashboard.progress.newWords.at(-1)?.value, 1);
|
||||
assert.equal(dashboard.progress.newWords.at(-3)?.value, 1);
|
||||
assert.equal(dashboard.progress.newWords.at(-4)?.value, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('new words per thousand and frequency bands read dictionary words', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addRollup(db, TODAY, videoId, { words: 2_000 });
|
||||
addWord(db, { headword: '一', day: TODAY, rank: 500 });
|
||||
addWord(db, { headword: '二', day: TODAY, rank: 3_000 });
|
||||
addWord(db, { headword: '三', day: TODAY, rank: 30_000 });
|
||||
addWord(db, { headword: '四', day: TODAY, rank: null });
|
||||
|
||||
const { learning } = getTrendsDashboard(db, '7d', 'day');
|
||||
|
||||
assert.equal(learning?.newWordsPerThousand.at(-1)?.value, 2);
|
||||
assert.deepEqual(learning?.frequencyBands.at(-1)?.counts, {
|
||||
top1k: 1,
|
||||
top5k: 1,
|
||||
top10k: 0,
|
||||
top20k: 0,
|
||||
beyond20k: 1,
|
||||
unranked: 1,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
test('vocabulary charts and summary count dictionary words, not surface forms', () => {
|
||||
withDashboardDb((db) => {
|
||||
markLexicalDailyRollupsReady(db);
|
||||
addWord(db, { headword: '言う', word: '言う', day: TODAY - 20 });
|
||||
addWord(db, { headword: '言う', word: '言った', day: TODAY });
|
||||
addWord(db, { headword: '東京', day: TODAY, pos2: '固有名詞' });
|
||||
|
||||
const charts = getVocabularyChartData(db);
|
||||
const summary = getVocabularySummary(db, null, Number(NOW_MS));
|
||||
|
||||
assert.deepEqual(charts.newWordsTimelineWithoutNames, [{ epochDay: TODAY - 20, wordCount: 1 }]);
|
||||
assert.deepEqual(charts.newWordsTimeline, [
|
||||
{ epochDay: TODAY - 20, wordCount: 1 },
|
||||
{ epochDay: TODAY, wordCount: 1 },
|
||||
]);
|
||||
assert.equal(summary.uniqueWords, 2);
|
||||
assert.equal(summary.uniqueWordsWithoutNames, 1);
|
||||
// 言う was first seen long ago, so its newer conjugation is not a new word.
|
||||
assert.equal(summary.newThisWeek, 1);
|
||||
assert.equal(summary.newThisWeekWithoutNames, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('names tagged with merged parts of speech count as names', () => {
|
||||
withDashboardDb((db) => {
|
||||
markLexicalDailyRollupsReady(db);
|
||||
addWord(db, { headword: '美琴', day: TODAY, pos2: '固有名詞|一般' });
|
||||
addWord(db, { headword: '谷君', day: TODAY, pos2: '固有名詞|接尾' });
|
||||
addWord(db, { headword: '海', day: TODAY, pos2: '一般' });
|
||||
|
||||
assert.equal(getTrendsDashboard(db, '7d', 'day').summary.current.newWords, 1);
|
||||
const summary = getVocabularySummary(db, null, Number(NOW_MS));
|
||||
assert.equal(summary.uniqueWords, 3);
|
||||
assert.equal(summary.uniqueWordsWithoutNames, 1);
|
||||
assert.deepEqual(getVocabularyChartData(db).newWordsTimelineWithoutNames, [
|
||||
{ epochDay: TODAY, wordCount: 1 },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
test('new word counts pick up writes from this and other connections', () => {
|
||||
withDashboardDb((db) => {
|
||||
addWord(db, { headword: '海', day: TODAY });
|
||||
assert.equal(getTrendsDashboard(db, '7d', 'day').summary.current.newWords, 1);
|
||||
|
||||
addWord(db, { headword: '山', day: TODAY });
|
||||
assert.equal(getTrendsDashboard(db, '7d', 'day').summary.current.newWords, 2);
|
||||
|
||||
const dbPath = (db.prepare('PRAGMA database_list').get() as { file: string }).file;
|
||||
const other = new Database(dbPath);
|
||||
try {
|
||||
addWord(other, { headword: '川', day: TODAY });
|
||||
} finally {
|
||||
other.close();
|
||||
}
|
||||
assert.equal(getTrendsDashboard(db, '7d', 'day').summary.current.newWords, 3);
|
||||
});
|
||||
});
|
||||
|
||||
test('getSessionSummaries can narrow to a span of local days', () => {
|
||||
withDashboardDb((db) => {
|
||||
const videoId = addVideo(db, 'Show', 1);
|
||||
addSession(db, TODAY - 5, videoId);
|
||||
addSession(db, TODAY - 4, videoId);
|
||||
addSession(db, TODAY, videoId);
|
||||
|
||||
assert.equal(getSessionSummaries(db, 50).length, 3);
|
||||
assert.equal(getSessionSummaries(db, 50, { startDay: TODAY - 5, endDay: TODAY - 4 }).length, 2);
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,562 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import type { StatsDictionaryWordQuery } from '../../../types/stats-http-contract';
|
||||
import {
|
||||
getSimilarWords,
|
||||
getWordAnimeAppearances,
|
||||
getVocabularySummary,
|
||||
getWordDetail,
|
||||
getWordOccurrences,
|
||||
replaceStatsExcludedWords,
|
||||
searchSubtitleSentences,
|
||||
} from './query-lexical';
|
||||
import { markLexicalDailyRollupsReady } from './lexical-rollups';
|
||||
import { getDictionaryWordPage, getVocabularyCoverage } from './query-vocabulary-words';
|
||||
import { startSessionRecord } from './session';
|
||||
import { Database, type DatabaseSync } from './sqlite';
|
||||
import { ensureSchema, getOrCreateAnimeRecord, getOrCreateVideoRecord } from './storage';
|
||||
import { SOURCE_TYPE_LOCAL, SOURCE_TYPE_REMOTE } from './types';
|
||||
|
||||
function withDb(run: (db: DatabaseSync) => void): void {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'subminer-vocabulary-'));
|
||||
const db = new Database(path.join(dir, 'immersion.sqlite'));
|
||||
try {
|
||||
ensureSchema(db);
|
||||
run(db);
|
||||
} finally {
|
||||
db.close();
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
interface WordFixture {
|
||||
headword: string;
|
||||
word?: string;
|
||||
reading?: string;
|
||||
seen?: number;
|
||||
rank?: number | null;
|
||||
pos2?: string;
|
||||
partOfSpeech?: string;
|
||||
firstSeen?: number;
|
||||
}
|
||||
|
||||
function addWord(db: DatabaseSync, word: WordFixture): number {
|
||||
const firstSeen = word.firstSeen ?? 1_700_000_000;
|
||||
const result = db
|
||||
.prepare(
|
||||
`INSERT INTO imm_words (
|
||||
headword, word, reading, part_of_speech, pos1, pos2, pos3, first_seen, last_seen,
|
||||
frequency, frequency_rank
|
||||
) VALUES (?, ?, ?, ?, '', ?, '', ?, ?, ?, ?)`,
|
||||
)
|
||||
.run(
|
||||
word.headword,
|
||||
word.word ?? word.headword,
|
||||
word.reading ?? '',
|
||||
word.partOfSpeech ?? 'noun',
|
||||
word.pos2 ?? '',
|
||||
firstSeen,
|
||||
firstSeen,
|
||||
word.seen ?? 1,
|
||||
word.rank ?? null,
|
||||
);
|
||||
return Number(result.lastInsertRowid);
|
||||
}
|
||||
|
||||
function getIdOf(db: DatabaseSync, headword: string): number {
|
||||
const row = db.prepare('SELECT id FROM imm_words WHERE headword = ?').get(headword) as {
|
||||
id: number;
|
||||
};
|
||||
return row.id;
|
||||
}
|
||||
|
||||
function addOccurrence(
|
||||
db: DatabaseSync,
|
||||
wordId: number,
|
||||
animeTitle: string,
|
||||
count = 1,
|
||||
source: 'file' | 'stream' = 'file',
|
||||
timing: { startMs: number | null; endMs: number | null } = { startMs: 1_000, endMs: 2_000 },
|
||||
): void {
|
||||
const animeId = getOrCreateAnimeRecord(db, {
|
||||
parsedTitle: animeTitle,
|
||||
canonicalTitle: animeTitle,
|
||||
anilistId: null,
|
||||
titleRomaji: null,
|
||||
titleEnglish: null,
|
||||
titleNative: null,
|
||||
metadataJson: null,
|
||||
});
|
||||
const videoId = getOrCreateVideoRecord(
|
||||
db,
|
||||
source === 'file' ? `local:/tmp/${animeTitle}.mkv` : `remote:jellyfin://${animeTitle}`,
|
||||
{
|
||||
canonicalTitle: animeTitle,
|
||||
sourcePath: source === 'file' ? `/tmp/${animeTitle}.mkv` : null,
|
||||
sourceUrl: source === 'file' ? null : `jellyfin://${animeTitle}`,
|
||||
sourceType: source === 'file' ? SOURCE_TYPE_LOCAL : SOURCE_TYPE_REMOTE,
|
||||
},
|
||||
);
|
||||
const { sessionId } = startSessionRecord(db, videoId, 1_700_000_000_000);
|
||||
const lineId = Number(
|
||||
db
|
||||
.prepare(
|
||||
`INSERT INTO imm_subtitle_lines (
|
||||
session_id, video_id, anime_id, line_index, segment_start_ms, segment_end_ms, text
|
||||
) VALUES (?, ?, ?, 0, ?, ?, ?)`,
|
||||
)
|
||||
.run(sessionId, videoId, animeId, timing.startMs, timing.endMs, `line ${animeTitle}`)
|
||||
.lastInsertRowid,
|
||||
);
|
||||
db.prepare(
|
||||
`INSERT INTO imm_word_line_occurrences (line_id, word_id, occurrence_count) VALUES (?, ?, ?)`,
|
||||
).run(lineId, wordId, count);
|
||||
}
|
||||
|
||||
function query(overrides: Partial<StatsDictionaryWordQuery> = {}): StatsDictionaryWordQuery {
|
||||
return {
|
||||
sort: 'seen',
|
||||
descending: true,
|
||||
hideKnown: false,
|
||||
hideNames: false,
|
||||
hideKana: false,
|
||||
limit: 25,
|
||||
offset: 0,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
const headwords = (db: DatabaseSync, overrides: Partial<StatsDictionaryWordQuery>) =>
|
||||
getDictionaryWordPage(db, null, query(overrides)).rows.map((row) => row.headword);
|
||||
|
||||
test('conjugated forms fold into one word with its total and dictionary reading', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 263, rank: 36 });
|
||||
const dictionaryId = addWord(db, {
|
||||
headword: '見る',
|
||||
word: '見る',
|
||||
reading: 'みる',
|
||||
seen: 172,
|
||||
rank: 36,
|
||||
});
|
||||
addWord(db, { headword: '見る', word: '見てる', reading: 'みてる', seen: 78, rank: 36 });
|
||||
|
||||
const { rows, total } = getDictionaryWordPage(db, null, query());
|
||||
|
||||
assert.equal(total, 1);
|
||||
assert.equal(rows[0]?.seen, 513);
|
||||
assert.equal(rows[0]?.reading, 'みる');
|
||||
assert.equal(rows[0]?.wordId, dictionaryId);
|
||||
assert.equal(rows[0]?.formCount, 3);
|
||||
assert.deepEqual(
|
||||
rows[0]?.forms.map((form) => form.word),
|
||||
['見て', '見る', '見てる'],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('a form stored under two readings counts as one form', () => {
|
||||
withDb((db) => {
|
||||
const first = addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 263 });
|
||||
addWord(db, { headword: '見る', word: '見て', reading: 'み', seen: 122 });
|
||||
addWord(db, { headword: '見る', word: '見た', reading: 'みた', seen: 200 });
|
||||
|
||||
const row = getDictionaryWordPage(db, null, query()).rows[0];
|
||||
assert.equal(row?.formCount, 2);
|
||||
assert.deepEqual(
|
||||
row?.forms.map((form) => [form.word, form.seen]),
|
||||
[
|
||||
['見て', 385],
|
||||
['見た', 200],
|
||||
],
|
||||
);
|
||||
assert.equal(getWordDetail(db, first)?.formCount, 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('a word seen only conjugated has no reading rather than a conjugated one', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '待つ', word: '待って', reading: 'まって', seen: 5 });
|
||||
assert.equal(getDictionaryWordPage(db, null, query()).rows[0]?.reading, '');
|
||||
});
|
||||
});
|
||||
|
||||
test('the list reaches rarely seen words, not just the most seen ones', () => {
|
||||
withDb((db) => {
|
||||
for (let index = 0; index < 600; index += 1) {
|
||||
addWord(db, { headword: `語${index}`, seen: 1_000 - index });
|
||||
}
|
||||
const { rows, total } = getDictionaryWordPage(db, null, query({ limit: 10, offset: 590 }));
|
||||
assert.equal(total, 600);
|
||||
assert.equal(rows[9]?.headword, '語599');
|
||||
});
|
||||
});
|
||||
|
||||
test('search matches any form or reading, treating katakana and hiragana alike', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '猫', word: '猫', reading: 'ねこ' });
|
||||
addWord(db, { headword: '行く', word: '行った', reading: 'いった' });
|
||||
addWord(db, { headword: '犬', word: '犬', reading: 'いぬ' });
|
||||
|
||||
assert.deepEqual(headwords(db, { search: 'ネコ' }), ['猫']);
|
||||
assert.deepEqual(headwords(db, { search: '行った' }), ['行く']);
|
||||
assert.deepEqual(headwords(db, { search: 'いった' }), ['行く']);
|
||||
});
|
||||
});
|
||||
|
||||
for (const c of [
|
||||
{ name: 'known words', filter: { hideKnown: true }, expected: ['海', '田中', 'なる'] },
|
||||
{ name: 'names', filter: { hideNames: true }, expected: ['山', '海', 'なる'] },
|
||||
{ name: 'kana', filter: { hideKana: true }, expected: ['山', '海', '田中'] },
|
||||
{ name: 'a rank limit', filter: { maxRank: 500 }, expected: ['山', 'なる'] },
|
||||
{ name: 'a part of speech', filter: { partOfSpeech: 'verb' as const }, expected: ['なる'] },
|
||||
]) {
|
||||
test(`the list filters by ${c.name}`, () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '山', seen: 40, rank: 100 });
|
||||
addWord(db, { headword: '海', seen: 30, rank: 9_000 });
|
||||
addWord(db, { headword: '田中', seen: 20, rank: 800, pos2: '固有名詞|一般' });
|
||||
addWord(db, { headword: 'なる', seen: 10, rank: 16, partOfSpeech: 'verb' });
|
||||
|
||||
const rows = getDictionaryWordPage(db, new Set(['山']), query(c.filter)).rows;
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.headword),
|
||||
c.expected,
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
test('a word with a non-name form is not a name and ranks as that form', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '白', word: 'しろ', rank: 49_000, pos2: '固有名詞' });
|
||||
addWord(db, { headword: '白', word: '白', rank: 1_500, pos2: '一般' });
|
||||
|
||||
const row = getDictionaryWordPage(db, null, query({ hideNames: true })).rows[0];
|
||||
assert.equal(row?.isName, false);
|
||||
assert.equal(row?.rank, 1_500);
|
||||
});
|
||||
});
|
||||
|
||||
test('excluded words stay out of the list', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '音', seen: 50 });
|
||||
addWord(db, { headword: '海', seen: 10 });
|
||||
assert.deepEqual(headwords(db, {}), ['音', '海']);
|
||||
|
||||
replaceStatsExcludedWords(db, [{ headword: '音', word: '音', reading: '' }]);
|
||||
assert.deepEqual(headwords(db, {}), ['海']);
|
||||
});
|
||||
});
|
||||
|
||||
test('unranked words sort last in both rank directions', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '無', rank: null });
|
||||
addWord(db, { headword: '高', rank: 10 });
|
||||
addWord(db, { headword: '低', rank: 900 });
|
||||
|
||||
assert.deepEqual(headwords(db, { sort: 'rank', descending: false }), ['高', '低', '無']);
|
||||
assert.deepEqual(headwords(db, { sort: 'rank', descending: true }), ['低', '高', '無']);
|
||||
});
|
||||
});
|
||||
|
||||
test('titles count distinct titles across every form and can filter and sort', () => {
|
||||
withDb((db) => {
|
||||
const seen = addWord(db, { headword: '見る', word: '見て', seen: 5 });
|
||||
const seenAgain = addWord(db, { headword: '見る', word: '見た', seen: 5 });
|
||||
const single = addWord(db, { headword: '海', seen: 50 });
|
||||
addOccurrence(db, seen, 'A');
|
||||
addOccurrence(db, seenAgain, 'B');
|
||||
addOccurrence(db, single, 'C');
|
||||
|
||||
const page = getDictionaryWordPage(db, null, query());
|
||||
assert.deepEqual(
|
||||
page.rows.map((row) => [row.headword, row.titles]),
|
||||
[
|
||||
['海', 1],
|
||||
['見る', 2],
|
||||
],
|
||||
);
|
||||
const across = getDictionaryWordPage(db, null, query({ sort: 'titles', minTitles: 2 }));
|
||||
assert.deepEqual(
|
||||
across.rows.map((row) => [row.headword, row.titles]),
|
||||
[['見る', 2]],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('a word is stream only when every line, across its forms, came from a stream', () => {
|
||||
withDb((db) => {
|
||||
const streamed = addWord(db, { headword: '流', word: '流れ', seen: 30 });
|
||||
addOccurrence(db, streamed, 'Stream A', 1, 'stream');
|
||||
const mixed = addWord(db, { headword: '混', word: '混ぜ', seen: 20 });
|
||||
const mixedFile = addWord(db, { headword: '混', word: '混ぜた', seen: 10 });
|
||||
addOccurrence(db, mixed, 'Stream B', 1, 'stream');
|
||||
addOccurrence(db, mixedFile, 'File A', 1, 'file');
|
||||
addWord(db, { headword: '無', seen: 1 });
|
||||
|
||||
const streamOnly = Object.fromEntries(
|
||||
getDictionaryWordPage(db, null, query()).rows.map((row) => [row.headword, row.streamOnly]),
|
||||
);
|
||||
// A word with no stored lines isn't blocked by streams.
|
||||
assert.deepEqual(streamOnly, { 流: true, 混: false, 無: false });
|
||||
});
|
||||
});
|
||||
|
||||
test('word detail counts lines from streams across every form', () => {
|
||||
withDb((db) => {
|
||||
const streamed = addWord(db, { headword: '混', word: '混ぜ' });
|
||||
const file = addWord(db, { headword: '混', word: '混ぜた' });
|
||||
addOccurrence(db, streamed, 'Stream A', 3, 'stream');
|
||||
addOccurrence(db, streamed, 'Stream B', 1, 'stream');
|
||||
addOccurrence(db, file, 'File A', 2, 'file');
|
||||
|
||||
const detail = getWordDetail(db, file);
|
||||
assert.equal(detail?.lineCount, 3);
|
||||
assert.equal(detail?.streamLineCount, 2);
|
||||
});
|
||||
});
|
||||
|
||||
test('lines that can be mined come first in examples and search, newest first within each', () => {
|
||||
withDb((db) => {
|
||||
const wordId = addWord(db, { headword: '言う', word: '言った', reading: 'いった' });
|
||||
addOccurrence(db, wordId, 'File old', 1, 'file');
|
||||
addOccurrence(db, wordId, 'File new', 1, 'file');
|
||||
addOccurrence(db, wordId, 'File untimed', 1, 'file', { startMs: null, endMs: null });
|
||||
addOccurrence(db, wordId, 'Stream newest', 1, 'stream');
|
||||
|
||||
const expected = ['File new', 'File old', 'Stream newest', 'File untimed'];
|
||||
assert.deepEqual(
|
||||
getWordOccurrences(db, '言う', '言った', 'いった', 10, 0, true).map((row) => row.videoTitle),
|
||||
expected,
|
||||
);
|
||||
assert.deepEqual(
|
||||
searchSubtitleSentences(db, 'line').map((row) => row.videoTitle),
|
||||
expected,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('legacy rows with NULL text columns list, search, and open without failing', () => {
|
||||
withDb((db) => {
|
||||
const insert = db.prepare(
|
||||
`INSERT INTO imm_words (headword, word, reading, first_seen, last_seen, frequency)
|
||||
VALUES (?, ?, ?, 1700000000, 1700000000, 3)`,
|
||||
);
|
||||
const noHeadword = Number(insert.run(null, '猫', null).lastInsertRowid);
|
||||
const noForm = Number(insert.run('犬', null, null).lastInsertRowid);
|
||||
|
||||
const { rows } = getDictionaryWordPage(db, null, query());
|
||||
assert.deepEqual(rows.map((row) => [row.headword, row.forms[0]?.word, row.reading]).sort(), [
|
||||
['犬', '犬', ''],
|
||||
['猫', '猫', ''],
|
||||
]);
|
||||
// NULL must not become searchable text.
|
||||
assert.deepEqual(headwords(db, { search: 'null' }), []);
|
||||
|
||||
for (const id of [noHeadword, noForm]) {
|
||||
assert.equal(getWordDetail(db, id)?.frequency, 3);
|
||||
assert.deepEqual(getWordAnimeAppearances(db, id), []);
|
||||
assert.deepEqual(getSimilarWords(db, id), []);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test('maxSeen and total describe every page, for scaling bars and paging', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '一', seen: 100 });
|
||||
addWord(db, { headword: '二', seen: 5 });
|
||||
|
||||
const page = getDictionaryWordPage(db, null, query({ limit: 1, offset: 1 }));
|
||||
assert.equal(page.total, 2);
|
||||
assert.equal(page.maxSeen, 100);
|
||||
assert.deepEqual(
|
||||
page.rows.map((row) => row.headword),
|
||||
['二'],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('the summary counts a word as known when any of its forms is known', () => {
|
||||
withDb((db) => {
|
||||
markLexicalDailyRollupsReady(db);
|
||||
addWord(db, { headword: '見る', word: '見て', reading: 'みて' });
|
||||
addWord(db, { headword: '海' });
|
||||
|
||||
assert.equal(getVocabularySummary(db, new Set(['見て'])).knownWordCount, 1);
|
||||
assert.equal(
|
||||
getDictionaryWordPage(db, new Set(['見て']), query()).rows.filter((row) => row.known).length,
|
||||
1,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('coverage counts seen and known words per frequency band, without names', () => {
|
||||
withDb((db) => {
|
||||
addWord(db, { headword: '一', rank: 500 });
|
||||
addWord(db, { headword: '二', rank: 900 });
|
||||
addWord(db, { headword: '三', rank: 4_000 });
|
||||
addWord(db, { headword: '四', rank: 30_000 });
|
||||
addWord(db, { headword: '田中', rank: 700, pos2: '固有名詞' });
|
||||
addWord(db, { headword: '無', rank: null });
|
||||
|
||||
assert.deepEqual(getVocabularyCoverage(db, new Set(['一', '三', '田中'])), [
|
||||
{ band: 'top1k', seen: 2, known: 1 },
|
||||
{ band: 'top5k', seen: 1, known: 1 },
|
||||
{ band: 'top10k', seen: 0, known: 0 },
|
||||
{ band: 'top20k', seen: 0, known: 0 },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
test('word detail reports totals and a reading for the dictionary word, not one form', () => {
|
||||
withDb((db) => {
|
||||
const form = addWord(db, {
|
||||
headword: '見る',
|
||||
word: '見て',
|
||||
reading: 'みて',
|
||||
seen: 263,
|
||||
rank: 36,
|
||||
firstSeen: 1_700_000_500,
|
||||
});
|
||||
addWord(db, {
|
||||
headword: '見る',
|
||||
word: '見る',
|
||||
reading: 'みる',
|
||||
seen: 172,
|
||||
rank: 36,
|
||||
firstSeen: 1_700_000_100,
|
||||
});
|
||||
|
||||
const detail = getWordDetail(db, form);
|
||||
|
||||
assert.equal(detail?.word, '見て');
|
||||
assert.equal(detail?.frequency, 435);
|
||||
assert.equal(detail?.dictionaryReading, 'みる');
|
||||
assert.equal(detail?.frequencyRank, 36);
|
||||
assert.equal(detail?.formCount, 2);
|
||||
assert.equal(detail?.firstSeen, 1_700_000_100);
|
||||
assert.equal(detail?.lastSeen, 1_700_000_500);
|
||||
});
|
||||
});
|
||||
|
||||
test('word appearances and examples can cover every form', () => {
|
||||
withDb((db) => {
|
||||
const first = addWord(db, { headword: '見る', word: '見て', reading: 'みて' });
|
||||
const second = addWord(db, { headword: '見る', word: '見た', reading: 'みた' });
|
||||
addOccurrence(db, first, 'A', 3);
|
||||
addOccurrence(db, second, 'B', 2);
|
||||
|
||||
assert.deepEqual(
|
||||
getWordAnimeAppearances(db, first).map((row) => [row.animeTitle, row.occurrenceCount]),
|
||||
[
|
||||
['A', 3],
|
||||
['B', 2],
|
||||
],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test("related words leave out a word's own forms and show one entry per word", () => {
|
||||
withDb((db) => {
|
||||
const form = addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 10 });
|
||||
addWord(db, { headword: '見る', word: '見た', reading: 'みた', seen: 9 });
|
||||
addWord(db, { headword: '見える', word: '見えた', reading: 'みえた', seen: 4 });
|
||||
addWord(db, { headword: '見える', word: '見える', reading: 'みえる', seen: 3 });
|
||||
|
||||
const similar = getSimilarWords(db, form);
|
||||
|
||||
assert.deepEqual(
|
||||
similar.map((row) => [row.headword, row.frequency]),
|
||||
[['見える', 7]],
|
||||
);
|
||||
assert.equal(similar[0]?.word, '見えた');
|
||||
});
|
||||
});
|
||||
|
||||
test('a reading match ranks a related word without choosing the form it shows', () => {
|
||||
withDb((db) => {
|
||||
const form = addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 10 });
|
||||
// Matches on the kanji with every form, and on the reading with its minor form.
|
||||
const kanjiAndReading = addWord(db, {
|
||||
headword: '見える',
|
||||
word: '見えた',
|
||||
reading: 'みえた',
|
||||
seen: 9,
|
||||
});
|
||||
addWord(db, { headword: '見える', word: '見えて', reading: 'みて', seen: 2 });
|
||||
// Matches only on the reading of its minor form, so its other forms must still count.
|
||||
const readingOnly = addWord(db, {
|
||||
headword: '満ちる',
|
||||
word: '満ちた',
|
||||
reading: 'みちた',
|
||||
seen: 7,
|
||||
});
|
||||
addWord(db, { headword: '満ちる', word: '満ちて', reading: 'みて', seen: 1 });
|
||||
// More seen, but no reading match, so it still ranks after both.
|
||||
addWord(db, { headword: '見物', word: '見物', reading: 'けんぶつ', seen: 100 });
|
||||
// A hidden form supplies neither the shown form nor the total.
|
||||
const hidden = addWord(db, { headword: '満ちる', word: '満ち', reading: 'みち', seen: 50 });
|
||||
db.prepare('UPDATE imm_words SET vocabulary_visible = 0 WHERE id = ?').run(hidden);
|
||||
|
||||
assert.deepEqual(
|
||||
getSimilarWords(db, form).map((row) => [row.wordId, row.word, row.reading, row.frequency]),
|
||||
[
|
||||
[kanjiAndReading, '見えた', 'みえた', 11],
|
||||
[readingOnly, '満ちた', 'みちた', 8],
|
||||
[getIdOf(db, '見物'), '見物', 'けんぶつ', 100],
|
||||
],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('related words show the same form when a word has tied top forms', () => {
|
||||
withDb((db) => {
|
||||
const form = addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 10 });
|
||||
const first = addWord(db, { headword: '見える', word: '見えた', reading: 'みえた', seen: 5 });
|
||||
addWord(db, { headword: '見える', word: '見えて', reading: 'みえて', seen: 5 });
|
||||
|
||||
assert.equal(getSimilarWords(db, form)[0]?.wordId, first);
|
||||
});
|
||||
});
|
||||
|
||||
test('related words leave out rows with a blank headword', () => {
|
||||
withDb((db) => {
|
||||
const form = addWord(db, { headword: '見る', word: '見て', reading: 'みて', seen: 10 });
|
||||
for (const [index, headword] of ['', ' ', '\t', '\v', '\f', ' '].entries()) {
|
||||
addWord(db, { headword, word: `空${index}`, reading: 'みて', seen: 50 });
|
||||
}
|
||||
addWord(db, { headword: '満ちる', word: '満ちて', reading: 'みて', seen: 1 });
|
||||
|
||||
assert.deepEqual(
|
||||
getSimilarWords(db, form).map((row) => row.headword),
|
||||
['満ちる'],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('all-forms examples include legacy rows stored without a headword', () => {
|
||||
withDb((db) => {
|
||||
const insert = db.prepare(
|
||||
`INSERT INTO imm_words (headword, word, reading, first_seen, last_seen, frequency)
|
||||
VALUES (?, ?, ?, 1700000000, 1700000000, 1)`,
|
||||
);
|
||||
const titlesFor = (headword: string): string[] =>
|
||||
getWordOccurrences(db, headword, '', '', 10, 0, true)
|
||||
.map((row) => row.videoTitle)
|
||||
.sort();
|
||||
|
||||
addOccurrence(db, Number(insert.run('猫', '猫', 'ねこ').lastInsertRowid), 'Headword');
|
||||
addOccurrence(db, Number(insert.run(null, '猫', 'ねこ').lastInsertRowid), 'Null headword');
|
||||
addOccurrence(db, Number(insert.run(' ', '猫', 'ねこ').lastInsertRowid), 'Blank headword');
|
||||
addOccurrence(db, Number(insert.run(null, null, 'いぬ').lastInsertRowid), 'Reading only');
|
||||
addOccurrence(db, Number(insert.run('犬', '猫', 'いぬ').lastInsertRowid), 'Other headword');
|
||||
|
||||
assert.deepEqual(titlesFor('猫'), ['Blank headword', 'Headword', 'Null headword']);
|
||||
assert.deepEqual(titlesFor('いぬ'), ['Reading only']);
|
||||
assert.deepEqual(titlesFor('犬'), ['Other headword']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,375 @@
|
||||
import type {
|
||||
StatsDictionaryWordPos,
|
||||
StatsDictionaryWordQuery,
|
||||
} from '../../../types/stats-http-contract';
|
||||
import type {
|
||||
DictionaryWordForm,
|
||||
DictionaryWordPage,
|
||||
DictionaryWordRow,
|
||||
VocabularyCoverageBand,
|
||||
VocabularyCoverageBandId,
|
||||
} from '../../../types/stats-wire';
|
||||
import type { DatabaseSync } from './sqlite';
|
||||
import {
|
||||
databaseWriteVersion,
|
||||
DICTIONARY_WORD_KEY_SQL,
|
||||
getExcludedWordIds,
|
||||
isProperNounPos2,
|
||||
mergeFormsByWord,
|
||||
NO_LOCAL_FILE_SQL,
|
||||
} from './query-lexical';
|
||||
import { forEachIdChunk, makePlaceholders } from './query-shared';
|
||||
import { nowMs } from './time';
|
||||
|
||||
const MAX_FORMS_PER_ROW = 8;
|
||||
// Counting titles for every word walks every occurrence (about 900ms on 500k rows),
|
||||
// and playback writes constantly, so that count tolerates being this stale.
|
||||
const ALL_TITLE_COUNTS_MAX_AGE_MS = 5 * 60_000;
|
||||
|
||||
const COVERAGE_BAND_LIMITS: ReadonlyArray<{ band: VocabularyCoverageBandId; maxRank: number }> = [
|
||||
{ band: 'top1k', maxRank: 1_000 },
|
||||
{ band: 'top5k', maxRank: 5_000 },
|
||||
{ band: 'top10k', maxRank: 10_000 },
|
||||
{ band: 'top20k', maxRank: 20_000 },
|
||||
];
|
||||
|
||||
const PART_OF_SPEECH_GROUPS: Record<StatsDictionaryWordPos, ReadonlySet<string>> = {
|
||||
noun: new Set(['noun']),
|
||||
verb: new Set(['verb']),
|
||||
adjective: new Set(['i_adjective', 'na_adjective']),
|
||||
};
|
||||
|
||||
const KANA_ONLY_TEXT = /^[\p{Script=Hiragana}\p{Script=Katakana}ーゝゞヽヾ]+$/u;
|
||||
|
||||
interface DictionaryWordFormRow {
|
||||
id: number;
|
||||
key: string;
|
||||
word: string;
|
||||
reading: string;
|
||||
seen: number | null;
|
||||
rank: number | null;
|
||||
pos2: string | null;
|
||||
partOfSpeech: string | null;
|
||||
firstSeen: number | null;
|
||||
lastSeen: number | null;
|
||||
}
|
||||
|
||||
interface DictionaryWord {
|
||||
key: string;
|
||||
seen: number;
|
||||
rank: number | null;
|
||||
isName: boolean;
|
||||
isKana: boolean;
|
||||
firstSeen: number;
|
||||
lastSeen: number;
|
||||
searchText: string;
|
||||
/** Most seen first. */
|
||||
forms: DictionaryWordFormRow[];
|
||||
}
|
||||
|
||||
const wordCache = new WeakMap<DatabaseSync, { version: string; words: DictionaryWord[] }>();
|
||||
const allTitleCountsCache = new WeakMap<
|
||||
DatabaseSync,
|
||||
{ version: string; computedAtMs: number; counts: Map<string, number> }
|
||||
>();
|
||||
|
||||
function toSeconds(value: number | null): number {
|
||||
const numeric = Number(value ?? 0);
|
||||
return Math.abs(numeric) >= 10_000_000_000 ? numeric / 1000 : numeric;
|
||||
}
|
||||
|
||||
/** Lowercase hiragana, so a search matches regardless of kana type or case. */
|
||||
function normalizeSearchText(text: string): string {
|
||||
return text
|
||||
.toLowerCase()
|
||||
.replace(/[ァ-ヶ]/g, (char) => String.fromCharCode(char.charCodeAt(0) - 0x60));
|
||||
}
|
||||
|
||||
function loadDictionaryWords(db: DatabaseSync): DictionaryWord[] {
|
||||
const version = databaseWriteVersion(db);
|
||||
const cached = wordCache.get(db);
|
||||
if (cached?.version === version) return cached.words;
|
||||
|
||||
const excludedIds = getExcludedWordIds(db);
|
||||
const exclusion =
|
||||
excludedIds.length > 0 ? `AND id NOT IN (${makePlaceholders(excludedIds)})` : '';
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT id, ${DICTIONARY_WORD_KEY_SQL} AS key,
|
||||
COALESCE(NULLIF(TRIM(word), ''), ${DICTIONARY_WORD_KEY_SQL}) AS word,
|
||||
COALESCE(reading, '') AS reading, frequency AS seen,
|
||||
frequency_rank AS rank, pos2, part_of_speech AS partOfSpeech,
|
||||
first_seen AS firstSeen, last_seen AS lastSeen
|
||||
FROM imm_words
|
||||
WHERE vocabulary_visible = 1 ${exclusion}
|
||||
ORDER BY frequency DESC, id
|
||||
`,
|
||||
)
|
||||
.all(...excludedIds) as unknown as DictionaryWordFormRow[];
|
||||
|
||||
const byKey = new Map<string, DictionaryWord>();
|
||||
for (const row of rows) {
|
||||
if (!row.key) continue;
|
||||
const firstSeen = toSeconds(row.firstSeen);
|
||||
const lastSeen = toSeconds(row.lastSeen);
|
||||
const isName = isProperNounPos2(row.pos2);
|
||||
const entry = byKey.get(row.key);
|
||||
if (entry) {
|
||||
entry.seen += row.seen ?? 0;
|
||||
entry.forms.push(row);
|
||||
entry.firstSeen = Math.min(entry.firstSeen, firstSeen);
|
||||
entry.lastSeen = Math.max(entry.lastSeen, lastSeen);
|
||||
entry.isName &&= isName;
|
||||
entry.searchText += `\n${row.word}\n${row.reading}`;
|
||||
} else {
|
||||
byKey.set(row.key, {
|
||||
key: row.key,
|
||||
seen: row.seen ?? 0,
|
||||
rank: null,
|
||||
isName,
|
||||
isKana: KANA_ONLY_TEXT.test(row.key),
|
||||
firstSeen,
|
||||
lastSeen,
|
||||
searchText: `${row.key}\n${row.word}\n${row.reading}`,
|
||||
forms: [row],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const words = [...byKey.values()];
|
||||
for (const word of words) {
|
||||
word.searchText = normalizeSearchText(word.searchText);
|
||||
word.rank = bestRank(word.forms);
|
||||
}
|
||||
wordCache.set(db, { version, words });
|
||||
return words;
|
||||
}
|
||||
|
||||
/** Rank from the word's non-name forms when it has any, so 白 ranks as the color. */
|
||||
function bestRank(forms: DictionaryWordFormRow[]): number | null {
|
||||
const nonNames = forms.filter((form) => !isProperNounPos2(form.pos2));
|
||||
let best: number | null = null;
|
||||
for (const form of nonNames.length > 0 ? nonNames : forms) {
|
||||
if (form.rank !== null && form.rank > 0 && (best === null || form.rank < best)) {
|
||||
best = form.rank;
|
||||
}
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
function isKnown(word: DictionaryWord, knownWords: ReadonlySet<string> | null): boolean {
|
||||
if (!knownWords) return false;
|
||||
return knownWords.has(word.key) || word.forms.some((form) => knownWords.has(form.word));
|
||||
}
|
||||
|
||||
function dictionaryForm(word: DictionaryWord): DictionaryWordFormRow | undefined {
|
||||
return word.forms.find((form) => form.word === word.key);
|
||||
}
|
||||
|
||||
function getAllTitleCounts(db: DatabaseSync): Map<string, number> {
|
||||
const version = databaseWriteVersion(db);
|
||||
const cached = allTitleCountsCache.get(db);
|
||||
if (
|
||||
cached &&
|
||||
(cached.version === version || nowMs() - cached.computedAtMs < ALL_TITLE_COUNTS_MAX_AGE_MS)
|
||||
) {
|
||||
return cached.counts;
|
||||
}
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT ${DICTIONARY_WORD_KEY_SQL} AS key, COUNT(DISTINCT sl.anime_id) AS titles
|
||||
FROM imm_words
|
||||
JOIN imm_word_line_occurrences o ON o.word_id = imm_words.id
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE sl.anime_id IS NOT NULL AND vocabulary_visible = 1
|
||||
GROUP BY key
|
||||
`,
|
||||
)
|
||||
.all() as Array<{ key: string; titles: number }>;
|
||||
const counts = new Map(rows.map((row) => [row.key, row.titles]));
|
||||
allTitleCountsCache.set(db, { version, computedAtMs: nowMs(), counts });
|
||||
return counts;
|
||||
}
|
||||
|
||||
function getTitleCountsForWords(db: DatabaseSync, words: DictionaryWord[]): Map<string, number> {
|
||||
const keyByWordId = new Map<number, string>();
|
||||
for (const word of words) {
|
||||
for (const form of word.forms) keyByWordId.set(form.id, word.key);
|
||||
}
|
||||
const animeByKey = new Map<string, Set<number>>();
|
||||
forEachIdChunk([...keyByWordId.keys()], (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT DISTINCT o.word_id AS wordId, sl.anime_id AS animeId
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines sl ON sl.line_id = o.line_id
|
||||
WHERE o.word_id IN (${makePlaceholders(chunk)}) AND sl.anime_id IS NOT NULL
|
||||
`,
|
||||
)
|
||||
.all(...chunk) as Array<{ wordId: number; animeId: number }>;
|
||||
for (const row of rows) {
|
||||
const key = keyByWordId.get(row.wordId);
|
||||
if (key === undefined) continue;
|
||||
const anime = animeByKey.get(key) ?? new Set<number>();
|
||||
anime.add(row.animeId);
|
||||
animeByKey.set(key, anime);
|
||||
}
|
||||
});
|
||||
return new Map([...animeByKey].map(([key, anime]) => [key, anime.size]));
|
||||
}
|
||||
|
||||
/** Words whose every stored line came from a stream. Words with no lines don't count. */
|
||||
function getStreamOnlyWords(db: DatabaseSync, words: DictionaryWord[]): Set<string> {
|
||||
const keyByWordId = new Map<number, string>();
|
||||
for (const word of words) {
|
||||
for (const form of word.forms) keyByWordId.set(form.id, word.key);
|
||||
}
|
||||
const withLines = new Set<string>();
|
||||
const withFile = new Set<string>();
|
||||
forEachIdChunk([...keyByWordId.keys()], (chunk) => {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT o.word_id AS wordId, MIN(${NO_LOCAL_FILE_SQL}) AS allStreamed
|
||||
FROM imm_word_line_occurrences o
|
||||
JOIN imm_subtitle_lines l ON l.line_id = o.line_id
|
||||
JOIN imm_videos v ON v.video_id = l.video_id
|
||||
WHERE o.word_id IN (${makePlaceholders(chunk)})
|
||||
GROUP BY o.word_id
|
||||
`,
|
||||
)
|
||||
.all(...chunk) as Array<{ wordId: number; allStreamed: number }>;
|
||||
for (const row of rows) {
|
||||
const key = keyByWordId.get(row.wordId);
|
||||
if (key === undefined) continue;
|
||||
withLines.add(key);
|
||||
if (row.allStreamed === 0) withFile.add(key);
|
||||
}
|
||||
});
|
||||
return new Set([...withLines].filter((key) => !withFile.has(key)));
|
||||
}
|
||||
|
||||
function compareWords(
|
||||
left: DictionaryWord,
|
||||
right: DictionaryWord,
|
||||
query: StatsDictionaryWordQuery,
|
||||
titleCounts: Map<string, number> | null,
|
||||
): number {
|
||||
const sign = query.descending ? -1 : 1;
|
||||
const byText = left.key < right.key ? -1 : left.key > right.key ? 1 : 0;
|
||||
switch (query.sort) {
|
||||
case 'headword':
|
||||
return byText * sign;
|
||||
case 'rank':
|
||||
// Unranked words go last in either direction.
|
||||
if (left.rank === null || right.rank === null) {
|
||||
return (left.rank === null ? 1 : 0) - (right.rank === null ? 1 : 0) || byText;
|
||||
}
|
||||
return (left.rank - right.rank) * sign || byText;
|
||||
case 'titles':
|
||||
return (
|
||||
((titleCounts?.get(left.key) ?? 0) - (titleCounts?.get(right.key) ?? 0)) * sign ||
|
||||
right.seen - left.seen ||
|
||||
byText
|
||||
);
|
||||
case 'firstSeen':
|
||||
return (left.firstSeen - right.firstSeen) * sign || byText;
|
||||
case 'lastSeen':
|
||||
return (left.lastSeen - right.lastSeen) * sign || byText;
|
||||
case 'seen':
|
||||
return (left.seen - right.seen) * sign || byText;
|
||||
}
|
||||
}
|
||||
|
||||
function toRow(
|
||||
word: DictionaryWord,
|
||||
knownWords: ReadonlySet<string> | null,
|
||||
titleCounts: Map<string, number>,
|
||||
streamOnlyWords: ReadonlySet<string>,
|
||||
): DictionaryWordRow {
|
||||
const dictionary = dictionaryForm(word);
|
||||
const topForm = word.forms[0]!;
|
||||
const reading = word.forms.find(
|
||||
(form) => form.word === word.key && form.reading.trim() !== '',
|
||||
)?.reading;
|
||||
const forms = mergeFormsByWord<DictionaryWordForm>(
|
||||
word.forms.map((form) => ({
|
||||
wordId: form.id,
|
||||
word: form.word,
|
||||
reading: form.reading,
|
||||
seen: form.seen ?? 0,
|
||||
})),
|
||||
);
|
||||
return {
|
||||
wordId: (dictionary ?? topForm).id,
|
||||
headword: word.key,
|
||||
reading: reading ?? '',
|
||||
partOfSpeech: topForm.partOfSpeech,
|
||||
rank: word.rank,
|
||||
seen: word.seen,
|
||||
titles: titleCounts.get(word.key) ?? 0,
|
||||
firstSeen: word.firstSeen,
|
||||
lastSeen: word.lastSeen,
|
||||
known: isKnown(word, knownWords),
|
||||
isName: word.isName,
|
||||
streamOnly: streamOnlyWords.has(word.key),
|
||||
formCount: forms.length,
|
||||
forms: forms.slice(0, MAX_FORMS_PER_ROW),
|
||||
};
|
||||
}
|
||||
|
||||
/** One page of dictionary words, folded across their forms and filtered on the server. */
|
||||
export function getDictionaryWordPage(
|
||||
db: DatabaseSync,
|
||||
knownWords: ReadonlySet<string> | null,
|
||||
query: StatsDictionaryWordQuery,
|
||||
): DictionaryWordPage {
|
||||
const needle = query.search ? normalizeSearchText(query.search) : '';
|
||||
const posGroup = query.partOfSpeech ? PART_OF_SPEECH_GROUPS[query.partOfSpeech] : null;
|
||||
const needsAllTitleCounts = query.sort === 'titles' || query.minTitles !== undefined;
|
||||
const allTitleCounts = needsAllTitleCounts ? getAllTitleCounts(db) : null;
|
||||
|
||||
const matches = loadDictionaryWords(db).filter((word) => {
|
||||
if (query.hideNames && word.isName) return false;
|
||||
if (query.hideKana && word.isKana) return false;
|
||||
if (query.maxRank !== undefined && (word.rank === null || word.rank > query.maxRank)) {
|
||||
return false;
|
||||
}
|
||||
if (posGroup && !posGroup.has(word.forms[0]?.partOfSpeech ?? '')) return false;
|
||||
if (needle && !word.searchText.includes(needle)) return false;
|
||||
if (query.minTitles !== undefined && (allTitleCounts?.get(word.key) ?? 0) < query.minTitles) {
|
||||
return false;
|
||||
}
|
||||
return !(query.hideKnown && isKnown(word, knownWords));
|
||||
});
|
||||
|
||||
matches.sort((left, right) => compareWords(left, right, query, allTitleCounts));
|
||||
const pageWords = matches.slice(query.offset, query.offset + query.limit);
|
||||
const titleCounts = allTitleCounts ?? getTitleCountsForWords(db, pageWords);
|
||||
const streamOnlyWords = getStreamOnlyWords(db, pageWords);
|
||||
return {
|
||||
rows: pageWords.map((word) => toRow(word, knownWords, titleCounts, streamOnlyWords)),
|
||||
total: matches.length,
|
||||
maxSeen: matches.reduce((max, word) => Math.max(max, word.seen), 0),
|
||||
};
|
||||
}
|
||||
|
||||
/** How many seen words in each frequency band are known. Names don't count. */
|
||||
export function getVocabularyCoverage(
|
||||
db: DatabaseSync,
|
||||
knownWords: ReadonlySet<string>,
|
||||
): VocabularyCoverageBand[] {
|
||||
const bands = COVERAGE_BAND_LIMITS.map(({ band }) => ({ band, seen: 0, known: 0 }));
|
||||
for (const word of loadDictionaryWords(db)) {
|
||||
if (word.isName || word.rank === null) continue;
|
||||
const index = COVERAGE_BAND_LIMITS.findIndex(({ maxRank }) => word.rank! <= maxRank);
|
||||
const band = bands[index];
|
||||
if (!band) continue;
|
||||
band.seen += 1;
|
||||
if (isKnown(word, knownWords)) band.known += 1;
|
||||
}
|
||||
return bands;
|
||||
}
|
||||
@@ -624,7 +624,14 @@ export interface WordDetailRow {
|
||||
pos1: string | null;
|
||||
pos2: string | null;
|
||||
pos3: string | null;
|
||||
/** Times seen across every form of this dictionary word. */
|
||||
frequency: number;
|
||||
frequencyRank: number | null;
|
||||
dictionaryReading: string;
|
||||
formCount: number;
|
||||
forms: Array<{ wordId: number; word: string; reading: string; seen: number }>;
|
||||
lineCount: number;
|
||||
streamLineCount: number;
|
||||
firstSeen: number;
|
||||
lastSeen: number;
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
enrichSessionsWithKnownWordMetrics,
|
||||
loadKnownWordsSet,
|
||||
parseEventTypesQuery,
|
||||
parseDayRange,
|
||||
parseIntQuery,
|
||||
parsePositiveId,
|
||||
parseTrendFillEmpty,
|
||||
@@ -64,17 +65,26 @@ export function registerStatsAnalyticsRoutes(
|
||||
});
|
||||
|
||||
app.get('/api/stats/trends/dashboard', async (c) => {
|
||||
const range = parseTrendRange(c.req.query('range'));
|
||||
const range = parseTrendRange(
|
||||
c.req.query('range'),
|
||||
c.req.query('fromDay'),
|
||||
c.req.query('toDay'),
|
||||
);
|
||||
const groupBy = parseTrendGroupBy(c.req.query('groupBy'));
|
||||
const fillEmpty = parseTrendFillEmpty(c.req.query('fillEmpty'));
|
||||
const title = c.req.query('title')?.trim() || null;
|
||||
return c.json(
|
||||
statsJson('trendsDashboard', await tracker.getTrendsDashboard(range, groupBy, fillEmpty)),
|
||||
statsJson(
|
||||
'trendsDashboard',
|
||||
await tracker.getTrendsDashboard(range, groupBy, fillEmpty, title),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
app.get('/api/stats/sessions', async (c) => {
|
||||
const limit = parseIntQuery(c.req.query('limit'), 50, 500);
|
||||
const rawSessions = await tracker.getSessionSummaries(limit);
|
||||
const dayRange = parseDayRange(c.req.query('fromDay'), c.req.query('toDay')) ?? undefined;
|
||||
const rawSessions = await tracker.getSessionSummaries(limit, dayRange);
|
||||
const sessions = await enrichSessionsWithKnownWordMetrics(
|
||||
tracker,
|
||||
rawSessions,
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
parseBooleanQuery,
|
||||
parseDuplicateLineCleanupBody,
|
||||
parseExcludedWordsBody,
|
||||
parseDictionaryWordQuery,
|
||||
parseIntQuery,
|
||||
parsePositiveId,
|
||||
parsePositiveIdList,
|
||||
@@ -49,6 +50,20 @@ export function registerStatsLibraryRoutes(
|
||||
return c.json(statsJson('vocabularyCharts', await tracker.getVocabularyChartData()));
|
||||
});
|
||||
|
||||
app.get('/api/stats/vocabulary/words', async (c) => {
|
||||
const page = await tracker.getDictionaryWords(
|
||||
loadKnownWordsSet(options?.knownWordCachePath),
|
||||
parseDictionaryWordQuery((name) => c.req.query(name)),
|
||||
);
|
||||
return c.json(statsJson('dictionaryWords', page));
|
||||
});
|
||||
|
||||
app.get('/api/stats/vocabulary/coverage', async (c) => {
|
||||
const knownWords = loadKnownWordsSet(options?.knownWordCachePath);
|
||||
const coverage = knownWords ? await tracker.getVocabularyCoverage(knownWords) : [];
|
||||
return c.json(statsJson('vocabularyCoverage', coverage));
|
||||
});
|
||||
|
||||
app.get('/api/stats/excluded-words', async (c) => {
|
||||
return c.json(statsJson('excludedWords', await tracker.getStatsExcludedWords()));
|
||||
});
|
||||
@@ -78,12 +93,23 @@ export function registerStatsLibraryRoutes(
|
||||
const headword = (c.req.query('headword') ?? '').trim();
|
||||
const word = (c.req.query('word') ?? '').trim();
|
||||
const reading = (c.req.query('reading') ?? '').trim();
|
||||
if (!headword || !word) {
|
||||
const allForms = c.req.query('allForms') === 'true';
|
||||
// Legacy rows can lack a headword or form, so an all-forms lookup falls back the same
|
||||
// way the word list does. An exact lookup can only match a row that has both.
|
||||
const dictionaryWord = headword || word || reading;
|
||||
if (allForms ? !dictionaryWord : !headword || !word) {
|
||||
return c.json(statsJson('wordOccurrences', []), 400);
|
||||
}
|
||||
const limit = parseIntQuery(c.req.query('limit'), 50, 500);
|
||||
const offset = parseIntQuery(c.req.query('offset'), 0, 10_000);
|
||||
const occurrences = await tracker.getWordOccurrences(headword, word, reading, limit, offset);
|
||||
const occurrences = await tracker.getWordOccurrences(
|
||||
dictionaryWord,
|
||||
word,
|
||||
reading,
|
||||
limit,
|
||||
offset,
|
||||
allForms,
|
||||
);
|
||||
return c.json(statsJson('wordOccurrences', occurrences));
|
||||
});
|
||||
|
||||
@@ -102,7 +128,7 @@ export function registerStatsLibraryRoutes(
|
||||
});
|
||||
|
||||
app.get('/api/stats/kanji', async (c) => {
|
||||
const limit = parseIntQuery(c.req.query('limit'), 100, 500);
|
||||
const limit = parseIntQuery(c.req.query('limit'), 100, 10_000);
|
||||
const kanji = await tracker.getKanjiStats(limit);
|
||||
return c.json(statsJson('kanji', kanji));
|
||||
});
|
||||
|
||||
@@ -12,6 +12,13 @@ import {
|
||||
} from '../../../anki-integration/known-word-cache-format.js';
|
||||
import { createLogger } from '../../../logger.js';
|
||||
import type { AnkiConnectConfig } from '../../../types.js';
|
||||
import type {
|
||||
StatsDayRange,
|
||||
StatsDictionaryWordQuery,
|
||||
StatsDictionaryWordSort,
|
||||
StatsTrendGroupBy,
|
||||
StatsTrendRange,
|
||||
} from '../../../types/stats-http-contract.js';
|
||||
import type { StatsExcludedWord } from '../../../types/stats-wire.js';
|
||||
import type { ImmersionTrackerService } from '../immersion-tracker-service.js';
|
||||
import { splitSentenceSearchTerms } from '../immersion-tracker/query-lexical.js';
|
||||
@@ -58,14 +65,61 @@ export function parsePositiveId(raw: string | undefined): number | null {
|
||||
return isPositiveSafeInteger(value) && String(value) === raw ? value : null;
|
||||
}
|
||||
|
||||
export function parseTrendRange(raw: string | undefined): '7d' | '30d' | '90d' | '365d' | 'all' {
|
||||
/** Parses an inclusive `fromDay`/`toDay` pair; anything malformed or reversed means no range. */
|
||||
export function parseDayRange(
|
||||
rawFrom: string | undefined,
|
||||
rawTo: string | undefined,
|
||||
): StatsDayRange | null {
|
||||
const startDay = parsePositiveId(rawFrom);
|
||||
const endDay = parsePositiveId(rawTo);
|
||||
return startDay !== null && endDay !== null && startDay <= endDay ? { startDay, endDay } : null;
|
||||
}
|
||||
|
||||
export function parseTrendRange(
|
||||
raw: string | undefined,
|
||||
rawFrom?: string,
|
||||
rawTo?: string,
|
||||
): StatsTrendRange {
|
||||
if (raw === 'custom') return parseDayRange(rawFrom, rawTo) ?? '30d';
|
||||
return raw === '7d' || raw === '30d' || raw === '90d' || raw === '365d' || raw === 'all'
|
||||
? raw
|
||||
: '30d';
|
||||
}
|
||||
|
||||
export function parseTrendGroupBy(raw: string | undefined): 'day' | 'month' {
|
||||
return raw === 'month' ? 'month' : 'day';
|
||||
export function parseTrendGroupBy(raw: string | undefined): StatsTrendGroupBy {
|
||||
return raw === 'month' || raw === 'week' ? raw : 'day';
|
||||
}
|
||||
|
||||
const DICTIONARY_WORD_SORTS: readonly StatsDictionaryWordSort[] = [
|
||||
'seen',
|
||||
'rank',
|
||||
'titles',
|
||||
'firstSeen',
|
||||
'lastSeen',
|
||||
'headword',
|
||||
];
|
||||
const DICTIONARY_WORD_PAGE_LIMIT = 200;
|
||||
|
||||
/** Reads the word list query string. Anything unrecognized falls back to the default. */
|
||||
export function parseDictionaryWordQuery(
|
||||
read: (name: string) => string | undefined,
|
||||
): StatsDictionaryWordQuery {
|
||||
const sort = DICTIONARY_WORD_SORTS.find((value) => value === read('sort')) ?? 'seen';
|
||||
const pos = read('pos');
|
||||
const flag = (name: string): boolean => read(name) === 'true';
|
||||
return {
|
||||
search: read('search')?.trim() || undefined,
|
||||
sort,
|
||||
descending: read('order') !== 'asc',
|
||||
hideKnown: flag('hideKnown'),
|
||||
hideNames: flag('hideNames'),
|
||||
hideKana: flag('hideKana'),
|
||||
partOfSpeech: pos === 'noun' || pos === 'verb' || pos === 'adjective' ? pos : undefined,
|
||||
maxRank: parsePositiveId(read('maxRank')) ?? undefined,
|
||||
minTitles: parsePositiveId(read('minTitles')) ?? undefined,
|
||||
limit: parseIntQuery(read('limit'), 25, DICTIONARY_WORD_PAGE_LIMIT),
|
||||
offset: parseIntQuery(read('offset'), 0),
|
||||
};
|
||||
}
|
||||
|
||||
export function parseTrendFillEmpty(raw: string | undefined): boolean {
|
||||
|
||||
@@ -3,6 +3,7 @@ import type {
|
||||
AnimeLibraryItem,
|
||||
AnimeWord,
|
||||
DailyRollup,
|
||||
DictionaryWordPage,
|
||||
EpisodeDetailData,
|
||||
EpisodesPerDay,
|
||||
KanjiDetailData,
|
||||
@@ -22,6 +23,7 @@ import type {
|
||||
StatsExcludedWord,
|
||||
StreakCalendarDay,
|
||||
TrendsDashboardData,
|
||||
VocabularyCoverageBand,
|
||||
VocabularyEntry,
|
||||
VocabularyOccurrenceEntry,
|
||||
WatchTimePerAnime,
|
||||
@@ -29,8 +31,14 @@ import type {
|
||||
} from './stats-wire';
|
||||
import type { TmdbMediaType } from '../shared/media-kind';
|
||||
|
||||
export type StatsTrendRange = '7d' | '30d' | '90d' | '365d' | 'all';
|
||||
export type StatsTrendGroupBy = 'day' | 'month';
|
||||
export type StatsTrendPresetRange = '7d' | '30d' | '90d' | '365d' | 'all';
|
||||
/** Inclusive local calendar days since the Unix epoch. */
|
||||
export interface StatsDayRange {
|
||||
startDay: number;
|
||||
endDay: number;
|
||||
}
|
||||
export type StatsTrendRange = StatsTrendPresetRange | StatsDayRange;
|
||||
export type StatsTrendGroupBy = 'day' | 'week' | 'month';
|
||||
export type StatsMineMode = 'word' | 'sentence' | 'audio';
|
||||
|
||||
/** Body of `POST /api/stats/maintenance/duplicate-lines`. */
|
||||
@@ -63,12 +71,37 @@ export interface StatsVocabularySummary {
|
||||
|
||||
export interface StatsVocabularyCharts {
|
||||
ready: boolean;
|
||||
topWords: Array<{ wordId: number; headword: string; frequency: number }>;
|
||||
topWordsWithoutNames: Array<{ wordId: number; headword: string; frequency: number }>;
|
||||
newWordsTimeline: Array<{ epochDay: number; wordCount: number }>;
|
||||
newWordsTimelineWithoutNames: Array<{ epochDay: number; wordCount: number }>;
|
||||
}
|
||||
|
||||
export type StatsDictionaryWordSort =
|
||||
| 'seen'
|
||||
| 'rank'
|
||||
| 'titles'
|
||||
| 'firstSeen'
|
||||
| 'lastSeen'
|
||||
| 'headword';
|
||||
export type StatsDictionaryWordPos = 'noun' | 'verb' | 'adjective';
|
||||
|
||||
/** Query for `GET /api/stats/vocabulary/words`. */
|
||||
export interface StatsDictionaryWordQuery {
|
||||
/** Matches any form or reading, treating katakana and hiragana alike. */
|
||||
search?: string;
|
||||
sort: StatsDictionaryWordSort;
|
||||
descending: boolean;
|
||||
hideKnown: boolean;
|
||||
hideNames: boolean;
|
||||
hideKana: boolean;
|
||||
partOfSpeech?: StatsDictionaryWordPos;
|
||||
/** Keep only ranked words at or better than this frequency rank. */
|
||||
maxRank?: number;
|
||||
/** Keep only words seen in at least this many titles. */
|
||||
minTitles?: number;
|
||||
limit: number;
|
||||
offset: number;
|
||||
}
|
||||
|
||||
export interface StatsAnilistSearchResult {
|
||||
id: number;
|
||||
episodes: number | null;
|
||||
@@ -204,6 +237,8 @@ export interface StatsJsonResponseMap {
|
||||
vocabulary: VocabularyEntry[];
|
||||
vocabularySummary: StatsVocabularySummary;
|
||||
vocabularyCharts: StatsVocabularyCharts;
|
||||
dictionaryWords: DictionaryWordPage;
|
||||
vocabularyCoverage: VocabularyCoverageBand[];
|
||||
excludedWords: StatsExcludedWord[];
|
||||
setExcludedWords: StatsOkResponse;
|
||||
duplicateLineCleanup: StatsDuplicateLineCleanupResult;
|
||||
@@ -259,13 +294,15 @@ export interface StatsHttpClient {
|
||||
getOverview: () => Promise<OverviewData>;
|
||||
getDailyRollups: (limit?: number) => Promise<DailyRollup[]>;
|
||||
getMonthlyRollups: (limit?: number) => Promise<MonthlyRollup[]>;
|
||||
getSessions: (limit?: number) => Promise<SessionSummary[]>;
|
||||
getSessions: (limit?: number, dayRange?: StatsDayRange) => Promise<SessionSummary[]>;
|
||||
getSessionTimeline: (id: number, limit?: number) => Promise<SessionTimelinePoint[]>;
|
||||
getSessionEvents: (id: number, limit?: number, eventTypes?: number[]) => Promise<SessionEvent[]>;
|
||||
getSessionKnownWordsTimeline: (id: number) => Promise<StatsSessionKnownWordsTimelinePoint[]>;
|
||||
getVocabulary: (limit?: number) => Promise<VocabularyEntry[]>;
|
||||
getVocabularySummary: () => Promise<StatsVocabularySummary>;
|
||||
getVocabularyCharts: () => Promise<StatsVocabularyCharts>;
|
||||
getDictionaryWords: (query: StatsDictionaryWordQuery) => Promise<DictionaryWordPage>;
|
||||
getVocabularyCoverage: () => Promise<VocabularyCoverageBand[]>;
|
||||
getExcludedWords: () => Promise<StatsExcludedWord[]>;
|
||||
setExcludedWords: (words: StatsExcludedWord[]) => Promise<void>;
|
||||
cleanupDuplicateLines: (
|
||||
@@ -277,6 +314,7 @@ export interface StatsHttpClient {
|
||||
reading: string,
|
||||
limit?: number,
|
||||
offset?: number,
|
||||
allForms?: boolean,
|
||||
) => Promise<VocabularyOccurrenceEntry[]>;
|
||||
searchSentences: (
|
||||
query: string,
|
||||
@@ -306,6 +344,7 @@ export interface StatsHttpClient {
|
||||
range: StatsTrendRange,
|
||||
groupBy: StatsTrendGroupBy,
|
||||
fillEmpty?: boolean,
|
||||
title?: string | null,
|
||||
) => Promise<TrendsDashboardData>;
|
||||
getWordDetail: (wordId: number) => Promise<WordDetailData>;
|
||||
getKanjiDetail: (kanjiId: number) => Promise<KanjiDetailData>;
|
||||
|
||||
+126
-16
@@ -78,6 +78,55 @@ export interface VocabularyEntry {
|
||||
lastSeen: number;
|
||||
}
|
||||
|
||||
/** One written form of a dictionary word, such as 見て for 見る. */
|
||||
export interface DictionaryWordForm {
|
||||
wordId: number;
|
||||
word: string;
|
||||
reading: string;
|
||||
seen: number;
|
||||
}
|
||||
|
||||
/** A dictionary word with every form it was seen in folded together. */
|
||||
export interface DictionaryWordRow {
|
||||
/** The form to open: the dictionary form when it was seen, otherwise the most seen one. */
|
||||
wordId: number;
|
||||
headword: string;
|
||||
/** Reading of the dictionary form, or empty when only conjugated forms were seen. */
|
||||
reading: string;
|
||||
partOfSpeech: string | null;
|
||||
rank: number | null;
|
||||
seen: number;
|
||||
/** Titles it appeared in. Computed for the returned rows. */
|
||||
titles: number;
|
||||
firstSeen: number;
|
||||
lastSeen: number;
|
||||
known: boolean;
|
||||
isName: boolean;
|
||||
/** Every line with this word came from a stream, so it can't be mined. */
|
||||
streamOnly: boolean;
|
||||
formCount: number;
|
||||
/** The most seen forms, capped so a word with hundreds of forms stays small. */
|
||||
forms: DictionaryWordForm[];
|
||||
}
|
||||
|
||||
export interface DictionaryWordPage {
|
||||
rows: DictionaryWordRow[];
|
||||
/** Words matching the query across all pages. */
|
||||
total: number;
|
||||
/** Highest `seen` across all pages, for scaling inline bars. */
|
||||
maxSeen: number;
|
||||
}
|
||||
|
||||
export const VOCABULARY_COVERAGE_BANDS = ['top1k', 'top5k', 'top10k', 'top20k'] as const;
|
||||
export type VocabularyCoverageBandId = (typeof VOCABULARY_COVERAGE_BANDS)[number];
|
||||
|
||||
/** Words seen in a frequency band, and how many of those are known. Names are left out. */
|
||||
export interface VocabularyCoverageBand {
|
||||
band: VocabularyCoverageBandId;
|
||||
seen: number;
|
||||
known: number;
|
||||
}
|
||||
|
||||
export interface StatsExcludedWord {
|
||||
headword: string;
|
||||
word: string;
|
||||
@@ -343,16 +392,59 @@ export interface TrendChartPoint {
|
||||
value: number;
|
||||
}
|
||||
|
||||
/** A rate over a bucket. `null` means too little data to rate, which charts draw as a gap. */
|
||||
export interface TrendRatePoint {
|
||||
label: string;
|
||||
value: number | null;
|
||||
}
|
||||
|
||||
/** One x-axis bucket, clipped to the requested date range. Epoch days are local calendar days. */
|
||||
export interface TrendBucket {
|
||||
label: string;
|
||||
startDay: number;
|
||||
endDay: number;
|
||||
}
|
||||
|
||||
export interface TrendPerAnimePoint {
|
||||
epochDay: number;
|
||||
animeTitle: string;
|
||||
value: number;
|
||||
}
|
||||
|
||||
export const TREND_FREQUENCY_BANDS = [
|
||||
'top1k',
|
||||
'top5k',
|
||||
'top10k',
|
||||
'top20k',
|
||||
'beyond20k',
|
||||
'unranked',
|
||||
] as const;
|
||||
export type TrendFrequencyBand = (typeof TREND_FREQUENCY_BANDS)[number];
|
||||
|
||||
export interface TrendFrequencyBandPoint {
|
||||
label: string;
|
||||
counts: Record<TrendFrequencyBand, number>;
|
||||
}
|
||||
|
||||
export interface TrendSummaryTotals {
|
||||
watchTimeMin: number;
|
||||
words: number;
|
||||
/** Null when the dashboard is filtered to one title: new words are not tracked per title. */
|
||||
newWords: number | null;
|
||||
cards: number;
|
||||
episodes: number;
|
||||
activeDays: number;
|
||||
}
|
||||
|
||||
export interface LibrarySummaryRow {
|
||||
title: string;
|
||||
/** Null for videos that are not linked to a library title. */
|
||||
animeId: number | null;
|
||||
/** Most recently watched video, the click-through target when `animeId` is null. */
|
||||
videoId: number | null;
|
||||
watchTimeMin: number;
|
||||
videos: number;
|
||||
episodesFinished: number;
|
||||
sessions: number;
|
||||
cards: number;
|
||||
words: number;
|
||||
@@ -363,15 +455,24 @@ export interface LibrarySummaryRow {
|
||||
}
|
||||
|
||||
export interface TrendsDashboardData {
|
||||
/** The days the dashboard covers. For `all` this is the span of recorded data. */
|
||||
window: { startDay: number; endDay: number };
|
||||
/** Axis shared by the `activity`, `progress`, and `ratios` series, index for index. */
|
||||
buckets: TrendBucket[];
|
||||
summary: {
|
||||
current: TrendSummaryTotals;
|
||||
/** The equally long period just before `window`; null for `all`. */
|
||||
previous: TrendSummaryTotals | null;
|
||||
};
|
||||
activity: {
|
||||
watchTime: TrendChartPoint[];
|
||||
cards: TrendChartPoint[];
|
||||
words: TrendChartPoint[];
|
||||
sessions: TrendChartPoint[];
|
||||
episodes: TrendChartPoint[];
|
||||
};
|
||||
/** Running totals counted from the start of `window`, each point the total at its period's end. */
|
||||
progress: {
|
||||
watchTime: TrendChartPoint[];
|
||||
sessions: TrendChartPoint[];
|
||||
words: TrendChartPoint[];
|
||||
newWords: TrendChartPoint[];
|
||||
cards: TrendChartPoint[];
|
||||
@@ -379,22 +480,21 @@ export interface TrendsDashboardData {
|
||||
lookups: TrendChartPoint[];
|
||||
};
|
||||
ratios: {
|
||||
lookupsPerHundred: TrendChartPoint[];
|
||||
cardsPerHour: TrendChartPoint[];
|
||||
readingSpeed: TrendChartPoint[];
|
||||
lookupsPerHundred: TrendRatePoint[];
|
||||
cardsPerHour: TrendRatePoint[];
|
||||
readingSpeed: TrendRatePoint[];
|
||||
};
|
||||
/** Vocabulary growth signals. Null when filtered to one title. */
|
||||
learning: {
|
||||
newWordsPerThousand: TrendRatePoint[];
|
||||
frequencyBands: TrendFrequencyBandPoint[];
|
||||
} | null;
|
||||
/** Active minutes as `[dayOfWeek][hour]`, with 0 = Sunday and local-time hours. */
|
||||
watchTimeByDayHour: number[][];
|
||||
/** Every title in `window`, unaffected by the title filter. */
|
||||
librarySummary: LibrarySummaryRow[];
|
||||
/** Sparse per-title values for each active day; the client accumulates them for charts. */
|
||||
animeDaily: {
|
||||
watchTime: TrendPerAnimePoint[];
|
||||
episodes: TrendPerAnimePoint[];
|
||||
cards: TrendPerAnimePoint[];
|
||||
words: TrendPerAnimePoint[];
|
||||
};
|
||||
patterns: {
|
||||
watchTimeByDayOfWeek: TrendChartPoint[];
|
||||
watchTimeByHour: TrendChartPoint[];
|
||||
};
|
||||
/** Sparse active minutes per title per day, unaffected by the title filter. */
|
||||
titleTimeline: TrendPerAnimePoint[];
|
||||
}
|
||||
|
||||
export interface WordDetailData {
|
||||
@@ -407,7 +507,17 @@ export interface WordDetailData {
|
||||
pos1: string | null;
|
||||
pos2: string | null;
|
||||
pos3: string | null;
|
||||
/** Times seen across every form of this dictionary word. */
|
||||
frequency: number;
|
||||
frequencyRank: number | null;
|
||||
/** Reading of the dictionary form, or empty when only conjugated forms were seen. */
|
||||
dictionaryReading: string;
|
||||
formCount: number;
|
||||
forms: DictionaryWordForm[];
|
||||
/** Stored subtitle lines with this word, across every form. */
|
||||
lineCount: number;
|
||||
/** Those lines that came from a stream, which can't be mined. */
|
||||
streamLineCount: number;
|
||||
firstSeen: number;
|
||||
lastSeen: number;
|
||||
};
|
||||
|
||||
+32
-4
@@ -10,10 +10,13 @@ import {
|
||||
createInitialStatsView,
|
||||
navigateToAnime as navigateToAnimeState,
|
||||
navigateToSession as navigateToSessionState,
|
||||
navigateToSessionsDays,
|
||||
openAnimeEpisodeDetail,
|
||||
openOverviewMediaDetail,
|
||||
openSessionsMediaDetail,
|
||||
openTrendsMediaDetail,
|
||||
switchTab,
|
||||
type SessionsDayFilter,
|
||||
} from './lib/stats-navigation';
|
||||
|
||||
const AnimeTab = lazy(() =>
|
||||
@@ -72,7 +75,8 @@ export function App() {
|
||||
const [mountedTabs, setMountedTabs] = useState<Set<TabId>>(() => new Set(['overview']));
|
||||
const [globalWordId, setGlobalWordId] = useState<number | null>(null);
|
||||
const { excluded, isExcluded, toggleExclusion, removeExclusion, clearAll } = useExcludedWords();
|
||||
const { activeTab, selectedAnimeId, focusedSessionId, mediaDetail } = viewState;
|
||||
const { activeTab, selectedAnimeId, focusedSessionId, sessionsDayFilter, mediaDetail } =
|
||||
viewState;
|
||||
|
||||
const activateTab = useCallback((tabId: TabId) => {
|
||||
setViewState((prev) => switchTab(prev, tabId));
|
||||
@@ -104,6 +108,16 @@ export function App() {
|
||||
});
|
||||
}, []);
|
||||
|
||||
const openSessionsForDays = useCallback((filter: SessionsDayFilter) => {
|
||||
setViewState((prev) => navigateToSessionsDays(prev, filter));
|
||||
setMountedTabs((prev) => {
|
||||
if (prev.has('sessions')) return prev;
|
||||
const next = new Set(prev);
|
||||
next.add('sessions');
|
||||
return next;
|
||||
});
|
||||
}, []);
|
||||
|
||||
const navigateToEpisodeDetail = useCallback(
|
||||
(animeId: number, videoId: number, sessionId: number | null = null) => {
|
||||
setViewState((prev) => openAnimeEpisodeDetail(prev, animeId, videoId, sessionId));
|
||||
@@ -122,6 +136,10 @@ export function App() {
|
||||
setViewState((prev) => openSessionsMediaDetail(prev, videoId));
|
||||
}, []);
|
||||
|
||||
const navigateToTrendsMediaDetail = useCallback((videoId: number) => {
|
||||
setViewState((prev) => openTrendsMediaDetail(prev, videoId));
|
||||
}, []);
|
||||
|
||||
const openWordDetail = useCallback((wordId: number) => {
|
||||
setGlobalWordId(wordId);
|
||||
}, []);
|
||||
@@ -171,7 +189,9 @@ export function App() {
|
||||
? 'Back to Overview'
|
||||
: mediaDetail.origin.type === 'sessions'
|
||||
? 'Back to Sessions'
|
||||
: 'Back to Library'
|
||||
: mediaDetail.origin.type === 'trends'
|
||||
? 'Back to Trends'
|
||||
: 'Back to Library'
|
||||
}
|
||||
onNavigateToAnime={navigateToAnime}
|
||||
/>
|
||||
@@ -222,7 +242,11 @@ export function App() {
|
||||
className="animate-fade-in"
|
||||
>
|
||||
<Suspense fallback={<LoadingSurface label="Loading trends..." />}>
|
||||
<TrendsTab />
|
||||
<TrendsTab
|
||||
onNavigateToAnime={navigateToAnime}
|
||||
onNavigateToMediaDetail={navigateToTrendsMediaDetail}
|
||||
onOpenSessionsForDays={openSessionsForDays}
|
||||
/>
|
||||
</Suspense>
|
||||
</section>
|
||||
) : null}
|
||||
@@ -239,7 +263,7 @@ export function App() {
|
||||
onNavigateToAnime={navigateToAnime}
|
||||
onOpenWordDetail={openWordDetail}
|
||||
excluded={excluded}
|
||||
isExcluded={isExcluded}
|
||||
onToggleExclusion={toggleExclusion}
|
||||
onRemoveExclusion={removeExclusion}
|
||||
onClearExclusions={clearAll}
|
||||
/>
|
||||
@@ -274,6 +298,10 @@ export function App() {
|
||||
setViewState((prev) => ({ ...prev, focusedSessionId: null }))
|
||||
}
|
||||
onNavigateToMediaDetail={navigateToSessionsMediaDetail}
|
||||
dayFilter={sessionsDayFilter}
|
||||
onClearDayFilter={() =>
|
||||
setViewState((prev) => ({ ...prev, sessionsDayFilter: null }))
|
||||
}
|
||||
/>
|
||||
</Suspense>
|
||||
</section>
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
/** Stands in for the mine buttons when a line can't be mined, and says why. */
|
||||
export function MineUnavailableNote({ reason }: { reason: string }) {
|
||||
return <span className="text-xs text-ctp-peach">{reason}</span>;
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
interface SummaryTileProps {
|
||||
label: string;
|
||||
value: string;
|
||||
detail: string | null;
|
||||
/** Smaller value for text such as dates. */
|
||||
compact?: boolean;
|
||||
}
|
||||
|
||||
/** One headline number. Group several in a grid for a summary strip. */
|
||||
export function SummaryTile({ label, value, detail, compact = false }: SummaryTileProps) {
|
||||
return (
|
||||
<div className="min-w-0">
|
||||
<div className="text-[11px] text-ctp-subtext0">{label}</div>
|
||||
<div
|
||||
className={`font-semibold leading-tight text-ctp-text ${compact ? 'mt-1 text-sm' : 'text-2xl'}`}
|
||||
>
|
||||
{value}
|
||||
</div>
|
||||
{detail ? <div className="text-[11px] text-ctp-overlay2">{detail}</div> : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
renderSentenceWithMatches,
|
||||
} from '../../lib/sentence-search';
|
||||
import { buildStatsMineCardParams, getStatsMineCardError } from '../../lib/mining';
|
||||
import { MineUnavailableNote } from '../common/MineUnavailableNote';
|
||||
import type { SentenceSearchResult } from '../../types/stats';
|
||||
|
||||
const SEARCH_LIMIT = 50;
|
||||
@@ -215,51 +216,53 @@ export function SearchTab() {
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{availability.exactMatch && (
|
||||
<button
|
||||
type="button"
|
||||
title={wordAudioDisabledReason || 'Create a word card from this sentence'}
|
||||
className="rounded-md border border-ctp-mauve/50 px-3 py-1.5 text-xs font-medium text-ctp-mauve transition hover:bg-ctp-mauve/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={wordStatus?.loading || !availability.canMineWordAudio}
|
||||
onClick={() => void handleMine(result, index, 'word')}
|
||||
>
|
||||
{buttonLabel(
|
||||
'word',
|
||||
wordStatus,
|
||||
availability.canMineWordAudio ? '' : 'Unavailable',
|
||||
)}
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
title={sentenceDisabledReason || 'Create a sentence card from this line'}
|
||||
className="rounded-md border border-ctp-green/50 px-3 py-1.5 text-xs font-medium text-ctp-green transition hover:bg-ctp-green/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={sentenceStatus?.loading || !availability.canMineSentence}
|
||||
onClick={() => void handleMine(result, index, 'sentence')}
|
||||
>
|
||||
{buttonLabel(
|
||||
'sentence',
|
||||
sentenceStatus,
|
||||
availability.canMineSentence ? '' : 'Unavailable',
|
||||
{availability.unavailableReason ? (
|
||||
<MineUnavailableNote reason={availability.unavailableReason} />
|
||||
) : (
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{availability.exactMatch && (
|
||||
<button
|
||||
type="button"
|
||||
title={wordAudioDisabledReason || 'Create a word card from this sentence'}
|
||||
className="rounded-md border border-ctp-mauve/50 px-3 py-1.5 text-xs font-medium text-ctp-mauve transition hover:bg-ctp-mauve/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={wordStatus?.loading || !availability.canMineWordAudio}
|
||||
onClick={() => void handleMine(result, index, 'word')}
|
||||
>
|
||||
{buttonLabel(
|
||||
'word',
|
||||
wordStatus,
|
||||
availability.canMineWordAudio ? '' : 'No exact match',
|
||||
)}
|
||||
</button>
|
||||
)}
|
||||
</button>
|
||||
{availability.exactMatch && (
|
||||
<button
|
||||
type="button"
|
||||
title={wordAudioDisabledReason || 'Create an audio card from this sentence'}
|
||||
className="rounded-md border border-ctp-blue/50 px-3 py-1.5 text-xs font-medium text-ctp-blue transition hover:bg-ctp-blue/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={audioStatus?.loading || !availability.canMineWordAudio}
|
||||
onClick={() => void handleMine(result, index, 'audio')}
|
||||
title={sentenceDisabledReason || 'Create a sentence card from this line'}
|
||||
className="rounded-md border border-ctp-green/50 px-3 py-1.5 text-xs font-medium text-ctp-green transition hover:bg-ctp-green/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={sentenceStatus?.loading || !availability.canMineSentence}
|
||||
onClick={() => void handleMine(result, index, 'sentence')}
|
||||
>
|
||||
{buttonLabel(
|
||||
'audio',
|
||||
audioStatus,
|
||||
availability.canMineWordAudio ? '' : 'Unavailable',
|
||||
)}
|
||||
{buttonLabel('sentence', sentenceStatus, '')}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
{availability.exactMatch && (
|
||||
<button
|
||||
type="button"
|
||||
title={
|
||||
wordAudioDisabledReason || 'Create an audio card from this sentence'
|
||||
}
|
||||
className="rounded-md border border-ctp-blue/50 px-3 py-1.5 text-xs font-medium text-ctp-blue transition hover:bg-ctp-blue/10 disabled:cursor-not-allowed disabled:border-ctp-surface2 disabled:text-ctp-overlay1 disabled:opacity-60"
|
||||
disabled={audioStatus?.loading || !availability.canMineWordAudio}
|
||||
onClick={() => void handleMine(result, index, 'audio')}
|
||||
>
|
||||
{buttonLabel(
|
||||
'audio',
|
||||
audioStatus,
|
||||
availability.canMineWordAudio ? '' : 'No exact match',
|
||||
)}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<p className="mt-4 rounded-lg bg-ctp-base/70 px-4 py-3 text-base leading-7 text-ctp-text">
|
||||
|
||||
@@ -6,6 +6,7 @@ import { apiClient } from '../../lib/api-client';
|
||||
import { confirmBucketDelete, confirmSessionDelete } from '../../lib/delete-confirm';
|
||||
import { formatDuration, formatNumber, formatSessionDayLabel } from '../../lib/formatters';
|
||||
import { groupSessionsByVideo, type SessionBucket } from '../../lib/session-grouping';
|
||||
import type { SessionsDayFilter } from '../../lib/stats-navigation';
|
||||
import type { SessionSummary } from '../../types/stats';
|
||||
|
||||
function groupSessionsByDay(sessions: SessionSummary[]): Map<string, SessionSummary[]> {
|
||||
@@ -79,18 +80,28 @@ export function decrementDeletingSessionCounts(
|
||||
return next;
|
||||
}
|
||||
|
||||
// Enough to show every session of any span a chart bar can open; the API caps limit at 500.
|
||||
const DAY_FILTER_SESSION_LIMIT = 500;
|
||||
|
||||
interface SessionsTabProps {
|
||||
initialSessionId?: number | null;
|
||||
onClearInitialSession?: () => void;
|
||||
onNavigateToMediaDetail?: (videoId: number) => void;
|
||||
dayFilter?: SessionsDayFilter | null;
|
||||
onClearDayFilter?: () => void;
|
||||
}
|
||||
|
||||
export function SessionsTab({
|
||||
initialSessionId,
|
||||
onClearInitialSession,
|
||||
onNavigateToMediaDetail,
|
||||
dayFilter,
|
||||
onClearDayFilter,
|
||||
}: SessionsTabProps = {}) {
|
||||
const { sessions, loading, error } = useSessions();
|
||||
const { sessions, loading, error } = useSessions(
|
||||
dayFilter ? DAY_FILTER_SESSION_LIMIT : undefined,
|
||||
dayFilter,
|
||||
);
|
||||
const [expandedId, setExpandedId] = useState<number | null>(null);
|
||||
const [expandedBuckets, setExpandedBuckets] = useState<Set<string>>(() => new Set());
|
||||
const [search, setSearch] = useState('');
|
||||
@@ -212,6 +223,20 @@ export function SessionsTab({
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{dayFilter ? (
|
||||
<div className="flex items-center gap-3 rounded-lg border border-ctp-surface1 bg-ctp-surface0/70 px-3 py-2 text-sm">
|
||||
<span className="text-ctp-subtext0">
|
||||
Sessions from <span className="text-ctp-text">{dayFilter.label}</span>
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClearDayFilter}
|
||||
className="ml-auto rounded-md border border-ctp-surface2 px-2 py-1 text-xs text-ctp-text transition hover:border-ctp-blue hover:text-ctp-blue"
|
||||
>
|
||||
Show all sessions
|
||||
</button>
|
||||
</div>
|
||||
) : null}
|
||||
<input
|
||||
type="search"
|
||||
aria-label="Search sessions by title"
|
||||
|
||||
@@ -1,14 +1,32 @@
|
||||
import type { TimeRange, GroupBy } from '../../hooks/useTrends';
|
||||
import { useEffect, useState } from 'react';
|
||||
import type { GroupBy, PresetTimeRange, TimeRange } from '../../hooks/useTrends';
|
||||
import { epochDayToIsoDate, isoDateToEpochDay, todayLocalDay } from '../../lib/formatters';
|
||||
import type { StatsDayRange } from '../../types/stats';
|
||||
|
||||
interface DateRangeSelectorProps {
|
||||
range: TimeRange;
|
||||
groupBy: GroupBy;
|
||||
fillEmpty: boolean;
|
||||
/** The days currently shown, used to seed the date inputs when switching to Custom. */
|
||||
shownDays: { startDay: number; endDay: number } | null;
|
||||
onRangeChange: (r: TimeRange) => void;
|
||||
onGroupByChange: (g: GroupBy) => void;
|
||||
onFillEmptyChange: (fillEmpty: boolean) => void;
|
||||
}
|
||||
|
||||
const CUSTOM_RANGE = 'custom';
|
||||
type RangeOption = PresetTimeRange | typeof CUSTOM_RANGE;
|
||||
|
||||
const RANGE_OPTIONS: RangeOption[] = ['7d', '30d', '90d', '365d', 'all', CUSTOM_RANGE];
|
||||
const DEFAULT_CUSTOM_DAYS = 30;
|
||||
const DATE_INPUT_CLASS =
|
||||
'rounded-md border border-ctp-surface2 bg-ctp-surface0 px-1.5 py-1 text-xs text-ctp-text transition hover:border-ctp-blue';
|
||||
|
||||
function formatRangeOption(option: RangeOption): string {
|
||||
if (option === 'all') return 'All';
|
||||
return option === CUSTOM_RANGE ? 'Custom' : option;
|
||||
}
|
||||
|
||||
function SegmentedControl<T extends string>({
|
||||
label,
|
||||
options,
|
||||
@@ -29,6 +47,7 @@ function SegmentedControl<T extends string>({
|
||||
{options.map((opt) => (
|
||||
<button
|
||||
key={opt}
|
||||
type="button"
|
||||
onClick={() => onChange(opt)}
|
||||
aria-pressed={value === opt}
|
||||
className={`px-2.5 py-1 rounded-md text-xs transition-colors ${
|
||||
@@ -45,32 +64,109 @@ function SegmentedControl<T extends string>({
|
||||
);
|
||||
}
|
||||
|
||||
interface DateDrafts {
|
||||
start: string;
|
||||
end: string;
|
||||
}
|
||||
|
||||
/** The range two date inputs describe, or null while either is incomplete or they are reversed. */
|
||||
export function parseDateDrafts(drafts: DateDrafts): StatsDayRange | null {
|
||||
const startDay = isoDateToEpochDay(drafts.start);
|
||||
const endDay = isoDateToEpochDay(drafts.end);
|
||||
return startDay !== null && endDay !== null && startDay <= endDay ? { startDay, endDay } : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Date inputs report every keystroke, so typing 2025 passes through years 0002 and
|
||||
* 0020. Each input shows its own draft, or it would snap back and lose the digits.
|
||||
* Both drafts are checked together on each edit, so a range can move forward by
|
||||
* editing either end first. They reset whenever the range changes.
|
||||
*/
|
||||
function CustomRangeInputs({
|
||||
range,
|
||||
onRangeChange,
|
||||
}: {
|
||||
range: StatsDayRange;
|
||||
onRangeChange: (range: StatsDayRange) => void;
|
||||
}) {
|
||||
const start = epochDayToIsoDate(range.startDay);
|
||||
const end = epochDayToIsoDate(range.endDay);
|
||||
const [drafts, setDrafts] = useState<DateDrafts>({ start, end });
|
||||
|
||||
useEffect(() => {
|
||||
setDrafts({ start, end });
|
||||
}, [start, end]);
|
||||
|
||||
const updateDrafts = (next: DateDrafts) => {
|
||||
setDrafts(next);
|
||||
const parsed = parseDateDrafts(next);
|
||||
if (parsed) onRangeChange(parsed);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex items-center gap-1.5 text-xs text-ctp-overlay2">
|
||||
<input
|
||||
type="date"
|
||||
aria-label="Range start"
|
||||
className={DATE_INPUT_CLASS}
|
||||
value={drafts.start}
|
||||
max={drafts.end}
|
||||
onChange={(event) => updateDrafts({ ...drafts, start: event.target.value })}
|
||||
/>
|
||||
to
|
||||
<input
|
||||
type="date"
|
||||
aria-label="Range end"
|
||||
className={DATE_INPUT_CLASS}
|
||||
value={drafts.end}
|
||||
min={drafts.start}
|
||||
max={epochDayToIsoDate(todayLocalDay())}
|
||||
onChange={(event) => updateDrafts({ ...drafts, end: event.target.value })}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function DateRangeSelector({
|
||||
range,
|
||||
groupBy,
|
||||
fillEmpty,
|
||||
shownDays,
|
||||
onRangeChange,
|
||||
onGroupByChange,
|
||||
onFillEmptyChange,
|
||||
}: DateRangeSelectorProps) {
|
||||
const selectRange = (option: RangeOption) => {
|
||||
if (option !== CUSTOM_RANGE) {
|
||||
onRangeChange(option);
|
||||
return;
|
||||
}
|
||||
if (typeof range !== 'string') return;
|
||||
const today = todayLocalDay();
|
||||
onRangeChange(shownDays ?? { startDay: today - (DEFAULT_CUSTOM_DAYS - 1), endDay: today });
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex flex-wrap items-center gap-4 text-sm">
|
||||
<SegmentedControl
|
||||
label="Range"
|
||||
options={['7d', '30d', '90d', '365d', 'all'] as TimeRange[]}
|
||||
value={range}
|
||||
onChange={onRangeChange}
|
||||
formatLabel={(r) => (r === 'all' ? 'All' : r)}
|
||||
options={RANGE_OPTIONS}
|
||||
value={typeof range === 'string' ? range : CUSTOM_RANGE}
|
||||
onChange={selectRange}
|
||||
formatLabel={formatRangeOption}
|
||||
/>
|
||||
{typeof range === 'string' ? null : (
|
||||
<CustomRangeInputs range={range} onRangeChange={onRangeChange} />
|
||||
)}
|
||||
<SegmentedControl
|
||||
label="Group by"
|
||||
options={['day', 'month'] as GroupBy[]}
|
||||
options={['day', 'week', 'month'] as GroupBy[]}
|
||||
value={groupBy}
|
||||
onChange={onGroupByChange}
|
||||
formatLabel={(g) => g.charAt(0).toUpperCase() + g.slice(1)}
|
||||
/>
|
||||
<SegmentedControl
|
||||
label="Empty days"
|
||||
label={`Empty ${groupBy}s`}
|
||||
options={['show', 'hide']}
|
||||
value={fillEmpty ? 'show' : 'hide'}
|
||||
onChange={(v) => onFillEmptyChange(v === 'show')}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import { formatHours } from '../../lib/formatters';
|
||||
|
||||
interface DayHourHeatmapProps {
|
||||
/** Active minutes as `[dayOfWeek][hour]`, with 0 = Sunday. */
|
||||
grid: number[][];
|
||||
}
|
||||
|
||||
const DAY_NAMES = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'];
|
||||
const DAY_ORDER = [1, 2, 3, 4, 5, 6, 0];
|
||||
// The day starts at 06:00 so an evening that runs past midnight stays one block
|
||||
// instead of being split across both ends of the chart.
|
||||
const FIRST_HOUR = 6;
|
||||
const HOUR_LABEL_EVERY = 3;
|
||||
const MIN_CELL_OPACITY = 0.18;
|
||||
|
||||
export const HEATMAP_HOURS = Array.from({ length: 24 }, (_, index) => (index + FIRST_HOUR) % 24);
|
||||
|
||||
function formatHourLabel(hour: number): string {
|
||||
return `${String(hour).padStart(2, '0')}:00`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Square-root scale, so a few very long sessions don't wash out the rest of the
|
||||
* grid. Returns 0 for an empty cell, which is drawn as background rather than color.
|
||||
*/
|
||||
export function heatmapCellOpacity(minutes: number, maxMinutes: number): number {
|
||||
if (minutes <= 0 || maxMinutes <= 0) return 0;
|
||||
return MIN_CELL_OPACITY + (1 - MIN_CELL_OPACITY) * Math.sqrt(minutes / maxMinutes);
|
||||
}
|
||||
|
||||
export function DayHourHeatmap({ grid }: DayHourHeatmapProps) {
|
||||
const maxMinutes = Math.max(0, ...grid.flat());
|
||||
const columns = `2.25rem repeat(24, minmax(0, 1fr)) 3rem`;
|
||||
|
||||
return (
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="mb-2 text-xs font-semibold text-ctp-text">Watch Time by Day and Hour</h3>
|
||||
{maxMinutes === 0 ? (
|
||||
<div className="text-xs text-ctp-overlay2">No data</div>
|
||||
) : (
|
||||
<div className="grid items-center gap-0.5" style={{ gridTemplateColumns: columns }}>
|
||||
{DAY_ORDER.map((day) => {
|
||||
const hours = grid[day] ?? [];
|
||||
const total = hours.reduce((sum, minutes) => sum + minutes, 0);
|
||||
return (
|
||||
<div key={day} className="contents">
|
||||
<div className="text-[10px] text-ctp-subtext0">{DAY_NAMES[day]}</div>
|
||||
{HEATMAP_HOURS.map((hour) => {
|
||||
const minutes = hours[hour] ?? 0;
|
||||
const opacity = heatmapCellOpacity(minutes, maxMinutes);
|
||||
return (
|
||||
<div
|
||||
key={hour}
|
||||
title={`${DAY_NAMES[day]} ${formatHourLabel(hour)}: ${formatHours(minutes)}`}
|
||||
className={`h-6 rounded-sm ${opacity === 0 ? 'bg-ctp-surface1/30' : 'bg-ctp-blue'}`}
|
||||
style={opacity === 0 ? undefined : { opacity }}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
<div className="text-right text-[10px] text-ctp-subtext0 tabular-nums">
|
||||
{formatHours(total)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
<div />
|
||||
{HEATMAP_HOURS.map((hour, index) => (
|
||||
<div key={hour} className="text-[10px] whitespace-nowrap text-ctp-overlay2">
|
||||
{index % HOUR_LABEL_EVERY === 0 ? formatHourLabel(hour) : ''}
|
||||
</div>
|
||||
))}
|
||||
<div />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts';
|
||||
import { CHART_DEFAULTS, CHART_THEME, TOOLTIP_CONTENT_STYLE } from '../../lib/chart-theme';
|
||||
import { formatNumber } from '../../lib/formatters';
|
||||
import {
|
||||
TREND_FREQUENCY_BANDS,
|
||||
type TrendFrequencyBand,
|
||||
type TrendFrequencyBandPoint,
|
||||
} from '../../types/stats';
|
||||
|
||||
interface FrequencyBandChartProps {
|
||||
data: TrendFrequencyBandPoint[];
|
||||
}
|
||||
|
||||
const BAND_LABELS: Record<TrendFrequencyBand, string> = {
|
||||
top1k: 'Top 1k',
|
||||
top5k: '1k to 5k',
|
||||
top10k: '5k to 10k',
|
||||
top20k: '10k to 20k',
|
||||
beyond20k: 'Beyond 20k',
|
||||
unranked: 'Unranked',
|
||||
};
|
||||
|
||||
// Bands are ordered, so one hue runs from faint (common) to strong (rare): the bar
|
||||
// brightens as new vocabulary moves further into rare words. Unranked words have
|
||||
// no place in that order and get a neutral color.
|
||||
const BAND_FILLS: Record<TrendFrequencyBand, { fill: string; fillOpacity: number }> = {
|
||||
top1k: { fill: '#8aadf4', fillOpacity: 0.2 },
|
||||
top5k: { fill: '#8aadf4', fillOpacity: 0.36 },
|
||||
top10k: { fill: '#8aadf4', fillOpacity: 0.54 },
|
||||
top20k: { fill: '#8aadf4', fillOpacity: 0.74 },
|
||||
beyond20k: { fill: '#8aadf4', fillOpacity: 1 },
|
||||
unranked: { fill: '#6e738d', fillOpacity: 0.6 },
|
||||
};
|
||||
|
||||
interface BandTooltipEntry {
|
||||
dataKey?: unknown;
|
||||
value?: unknown;
|
||||
}
|
||||
|
||||
function BandTooltip({
|
||||
active,
|
||||
label,
|
||||
payload,
|
||||
}: {
|
||||
active?: boolean;
|
||||
label?: string | number;
|
||||
payload?: BandTooltipEntry[];
|
||||
}) {
|
||||
if (!active || !payload?.length) return null;
|
||||
const rows = payload.map((entry) => ({
|
||||
band: TREND_FREQUENCY_BANDS.find((band) => band === entry.dataKey),
|
||||
count: typeof entry.value === 'number' ? entry.value : 0,
|
||||
}));
|
||||
const total = rows.reduce((sum, row) => sum + row.count, 0);
|
||||
return (
|
||||
<div style={{ ...TOOLTIP_CONTENT_STYLE, padding: '6px 10px' }}>
|
||||
<div style={{ color: CHART_THEME.tooltipLabel, marginBottom: 4, fontWeight: 600 }}>
|
||||
{label}: {formatNumber(total)} new words
|
||||
</div>
|
||||
{rows.map(({ band, count }) =>
|
||||
band ? (
|
||||
<div key={band} className="flex justify-between gap-4">
|
||||
<span>{BAND_LABELS[band]}</span>
|
||||
<span className="tabular-nums">
|
||||
{formatNumber(count)} ({total > 0 ? Math.round((count / total) * 100) : 0}%)
|
||||
</span>
|
||||
</div>
|
||||
) : null,
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function FrequencyBandChart({ data }: FrequencyBandChartProps) {
|
||||
const rows = data.map((point) => ({ label: point.label, ...point.counts }));
|
||||
return (
|
||||
<div className="h-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="mb-2 text-xs font-semibold text-ctp-text">New Words by Frequency Rank</h3>
|
||||
{rows.length === 0 ? (
|
||||
<div className="text-xs text-ctp-overlay2">No data</div>
|
||||
) : (
|
||||
<>
|
||||
<ResponsiveContainer width="100%" height={CHART_DEFAULTS.height}>
|
||||
<BarChart data={rows} margin={CHART_DEFAULTS.margin} stackOffset="expand">
|
||||
<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
width={40}
|
||||
tickFormatter={(value: number) => `${Math.round(value * 100)}%`}
|
||||
/>
|
||||
<Tooltip content={<BandTooltip />} wrapperStyle={{ zIndex: 50 }} />
|
||||
{TREND_FREQUENCY_BANDS.map((band) => (
|
||||
<Bar key={band} dataKey={band} stackId="bands" {...BAND_FILLS[band]} />
|
||||
))}
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
<div className="mt-2 flex flex-wrap gap-x-3 gap-y-1">
|
||||
{TREND_FREQUENCY_BANDS.map((band) => (
|
||||
<span key={band} className="flex items-center gap-1 text-[10px] text-ctp-subtext0">
|
||||
<span
|
||||
className="inline-block h-2 w-2 shrink-0 rounded-sm"
|
||||
style={{
|
||||
backgroundColor: BAND_FILLS[band].fill,
|
||||
opacity: BAND_FILLS[band].fillOpacity,
|
||||
}}
|
||||
/>
|
||||
{BAND_LABELS[band]}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,23 +1,21 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts';
|
||||
import type { LibrarySummaryRow } from '../../types/stats';
|
||||
import { CHART_DEFAULTS, CHART_THEME, TOOLTIP_CONTENT_STYLE } from '../../lib/chart-theme';
|
||||
import { epochDayToDate, formatDuration, formatNumber } from '../../lib/formatters';
|
||||
|
||||
interface LibrarySummarySectionProps {
|
||||
rows: LibrarySummaryRow[];
|
||||
hiddenTitles: ReadonlySet<string>;
|
||||
selectedTitle: string | null;
|
||||
onSelectTitle: (title: string) => void;
|
||||
onOpenTitle: (row: LibrarySummaryRow) => void;
|
||||
}
|
||||
|
||||
const LEADERBOARD_LIMIT = 10;
|
||||
const LEADERBOARD_HEIGHT = 260;
|
||||
const LEADERBOARD_BAR_COLOR = '#8aadf4';
|
||||
const TABLE_MAX_HEIGHT = 480;
|
||||
|
||||
type SortColumn =
|
||||
| 'title'
|
||||
| 'watchTimeMin'
|
||||
| 'videos'
|
||||
| 'episodesFinished'
|
||||
| 'sessions'
|
||||
| 'cards'
|
||||
| 'words'
|
||||
@@ -37,6 +35,7 @@ const COLUMNS: ColumnDef[] = [
|
||||
{ id: 'title', label: 'Title', align: 'left' },
|
||||
{ id: 'watchTimeMin', label: 'Watch Time', align: 'right' },
|
||||
{ id: 'videos', label: 'Videos', align: 'right' },
|
||||
{ id: 'episodesFinished', label: 'Finished', align: 'right' },
|
||||
{ id: 'sessions', label: 'Sessions', align: 'right' },
|
||||
{ id: 'cards', label: 'Cards', align: 'right' },
|
||||
{ id: 'words', label: 'Words', align: 'right' },
|
||||
@@ -45,11 +44,6 @@ const COLUMNS: ColumnDef[] = [
|
||||
{ id: 'firstWatched', label: 'Date Range', align: 'right' },
|
||||
];
|
||||
|
||||
function truncateTitle(title: string, maxChars: number): string {
|
||||
if (title.length <= maxChars) return title;
|
||||
return `${title.slice(0, maxChars - 1)}…`;
|
||||
}
|
||||
|
||||
function formatDateRange(firstEpochDay: number, lastEpochDay: number): string {
|
||||
const fmt = (epochDay: number) =>
|
||||
epochDayToDate(epochDay).toLocaleDateString(undefined, {
|
||||
@@ -76,10 +70,6 @@ function compareRows(
|
||||
return a.title.localeCompare(b.title) * sign;
|
||||
}
|
||||
|
||||
if (column === 'firstWatched') {
|
||||
return (a.firstWatched - b.firstWatched) * sign;
|
||||
}
|
||||
|
||||
if (column === 'lookupsPerHundred') {
|
||||
const aVal = a.lookupsPerHundred;
|
||||
const bVal = b.lookupsPerHundred;
|
||||
@@ -89,39 +79,28 @@ function compareRows(
|
||||
return (aVal - bVal) * sign;
|
||||
}
|
||||
|
||||
const aVal = a[column] as number;
|
||||
const bVal = b[column] as number;
|
||||
return (aVal - bVal) * sign;
|
||||
return (a[column] - b[column]) * sign;
|
||||
}
|
||||
|
||||
export function LibrarySummarySection({ rows, hiddenTitles }: LibrarySummarySectionProps) {
|
||||
export function LibrarySummarySection({
|
||||
rows,
|
||||
selectedTitle,
|
||||
onSelectTitle,
|
||||
onOpenTitle,
|
||||
}: LibrarySummarySectionProps) {
|
||||
const [sortColumn, setSortColumn] = useState<SortColumn>('watchTimeMin');
|
||||
const [sortDirection, setSortDirection] = useState<SortDirection>('desc');
|
||||
|
||||
const visibleRows = useMemo(
|
||||
() => rows.filter((row) => !hiddenTitles.has(row.title)),
|
||||
[rows, hiddenTitles],
|
||||
);
|
||||
|
||||
const sortedRows = useMemo(
|
||||
() => [...visibleRows].sort((a, b) => compareRows(a, b, sortColumn, sortDirection)),
|
||||
[visibleRows, sortColumn, sortDirection],
|
||||
() => [...rows].sort((a, b) => compareRows(a, b, sortColumn, sortDirection)),
|
||||
[rows, sortColumn, sortDirection],
|
||||
);
|
||||
const maxWatchTime = useMemo(
|
||||
() => rows.reduce((max, row) => Math.max(max, row.watchTimeMin), 0),
|
||||
[rows],
|
||||
);
|
||||
|
||||
const leaderboard = useMemo(
|
||||
() =>
|
||||
[...visibleRows]
|
||||
.sort((a, b) => b.watchTimeMin - a.watchTimeMin)
|
||||
.slice(0, LEADERBOARD_LIMIT)
|
||||
.map((row) => ({
|
||||
title: row.title,
|
||||
displayTitle: truncateTitle(row.title, 24),
|
||||
watchTimeMin: row.watchTimeMin,
|
||||
})),
|
||||
[visibleRows],
|
||||
);
|
||||
|
||||
if (visibleRows.length === 0) {
|
||||
if (rows.length === 0) {
|
||||
return (
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<div className="text-xs text-ctp-overlay2">No library activity in the selected window.</div>
|
||||
@@ -139,86 +118,85 @@ export function LibrarySummarySection({ rows, hiddenTitles }: LibrarySummarySect
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">Top Titles by Watch Time (min)</h3>
|
||||
<ResponsiveContainer width="100%" height={LEADERBOARD_HEIGHT}>
|
||||
<BarChart
|
||||
data={leaderboard}
|
||||
layout="vertical"
|
||||
margin={{ top: 8, right: 16, bottom: 8, left: 8 }}
|
||||
>
|
||||
<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />
|
||||
<XAxis
|
||||
type="number"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
<YAxis
|
||||
type="category"
|
||||
dataKey="displayTitle"
|
||||
width={160}
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
interval={0}
|
||||
/>
|
||||
<Tooltip
|
||||
contentStyle={TOOLTIP_CONTENT_STYLE}
|
||||
formatter={(value: number) => [`${value} min`, 'Watch Time']}
|
||||
labelFormatter={(_label, payload) => {
|
||||
const datum = payload?.[0]?.payload as { title?: string } | undefined;
|
||||
return datum?.title ?? '';
|
||||
}}
|
||||
/>
|
||||
<Bar dataKey="watchTimeMin" fill={LEADERBOARD_BAR_COLOR} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</div>
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">Per-Title Summary</h3>
|
||||
<div className="overflow-auto" style={{ maxHeight: TABLE_MAX_HEIGHT }}>
|
||||
<table className="w-full text-xs">
|
||||
<thead className="sticky top-0 bg-ctp-surface0">
|
||||
<tr className="border-b border-ctp-surface1 text-ctp-subtext0">
|
||||
{COLUMNS.map((column) => {
|
||||
const isActive = column.id === sortColumn;
|
||||
const indicator = isActive ? (sortDirection === 'asc' ? ' ▲' : ' ▼') : '';
|
||||
return (
|
||||
<th
|
||||
key={column.id}
|
||||
scope="col"
|
||||
className={`px-2 py-2 font-medium select-none cursor-pointer hover:text-ctp-text ${
|
||||
column.align === 'right' ? 'text-right' : 'text-left'
|
||||
} ${isActive ? 'text-ctp-text' : ''}`}
|
||||
onClick={() => handleHeaderClick(column.id)}
|
||||
>
|
||||
{column.label}
|
||||
{indicator}
|
||||
</th>
|
||||
);
|
||||
})}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{sortedRows.map((row) => (
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">Per-Title Summary</h3>
|
||||
<div className="overflow-auto" style={{ maxHeight: TABLE_MAX_HEIGHT }}>
|
||||
<table className="w-full text-xs">
|
||||
<thead className="sticky top-0 bg-ctp-surface0">
|
||||
<tr className="border-b border-ctp-surface1 text-ctp-subtext0">
|
||||
{COLUMNS.map((column) => {
|
||||
const isActive = column.id === sortColumn;
|
||||
const indicator = isActive ? (sortDirection === 'asc' ? ' ▲' : ' ▼') : '';
|
||||
return (
|
||||
<th
|
||||
key={column.id}
|
||||
scope="col"
|
||||
aria-sort={
|
||||
isActive ? (sortDirection === 'asc' ? 'ascending' : 'descending') : undefined
|
||||
}
|
||||
className={`px-2 py-2 font-medium select-none cursor-pointer hover:text-ctp-text ${
|
||||
column.align === 'right' ? 'text-right' : 'text-left'
|
||||
} ${isActive ? 'text-ctp-text' : ''}`}
|
||||
onClick={() => handleHeaderClick(column.id)}
|
||||
>
|
||||
{column.label}
|
||||
{indicator}
|
||||
</th>
|
||||
);
|
||||
})}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{sortedRows.map((row) => {
|
||||
const isSelected = row.title === selectedTitle;
|
||||
return (
|
||||
<tr
|
||||
key={row.title}
|
||||
className="border-b border-ctp-surface1 last:border-b-0 hover:bg-ctp-surface1/40"
|
||||
className={`border-b border-ctp-surface1 last:border-b-0 hover:bg-ctp-surface1/40 ${
|
||||
isSelected ? 'bg-ctp-surface1/60' : ''
|
||||
}`}
|
||||
>
|
||||
<td
|
||||
className="px-2 py-2 text-left text-ctp-text max-w-[240px] truncate"
|
||||
title={row.title}
|
||||
>
|
||||
{row.title}
|
||||
<td className="px-2 py-2 text-left max-w-[240px]">
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={isSelected}
|
||||
title={`Filter the page to ${row.title}`}
|
||||
className={`min-w-0 truncate text-left transition hover:text-ctp-blue ${
|
||||
isSelected ? 'font-semibold text-ctp-blue' : 'text-ctp-text'
|
||||
}`}
|
||||
onClick={() => onSelectTitle(row.title)}
|
||||
>
|
||||
{row.title}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Open ${row.title}`}
|
||||
title="Open this title"
|
||||
className="shrink-0 text-ctp-overlay2 transition hover:text-ctp-blue"
|
||||
onClick={() => onOpenTitle(row)}
|
||||
>
|
||||
↗
|
||||
</button>
|
||||
</div>
|
||||
</td>
|
||||
<td className="px-2 py-2 text-right text-ctp-text tabular-nums">
|
||||
{formatWatchTime(row.watchTimeMin)}
|
||||
<div>{formatWatchTime(row.watchTimeMin)}</div>
|
||||
<div className="ml-auto mt-1 h-1 w-20 overflow-hidden rounded-full bg-ctp-surface1/60">
|
||||
<div
|
||||
className="ml-auto h-full rounded-full bg-ctp-blue"
|
||||
style={{
|
||||
width: `${maxWatchTime > 0 ? (row.watchTimeMin / maxWatchTime) * 100 : 0}%`,
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
</td>
|
||||
<td className="px-2 py-2 text-right text-ctp-text tabular-nums">
|
||||
{formatNumber(row.videos)}
|
||||
</td>
|
||||
<td className="px-2 py-2 text-right text-ctp-text tabular-nums">
|
||||
{formatNumber(row.episodesFinished)}
|
||||
</td>
|
||||
<td className="px-2 py-2 text-right text-ctp-text tabular-nums">
|
||||
{formatNumber(row.sessions)}
|
||||
</td>
|
||||
@@ -238,11 +216,11 @@ export function LibrarySummarySection({ rows, hiddenTitles }: LibrarySummarySect
|
||||
{formatDateRange(row.firstWatched, row.lastWatched)}
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,122 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {
|
||||
buildLineData,
|
||||
sortTooltipEntries,
|
||||
tooltipColumnCount,
|
||||
type PerAnimeDataPoint,
|
||||
} from './StackedTrendChart';
|
||||
|
||||
function makePoints(titleCount: number): PerAnimeDataPoint[] {
|
||||
const points: PerAnimeDataPoint[] = [];
|
||||
for (let i = 0; i < titleCount; i += 1) {
|
||||
points.push({ epochDay: 20_000 + i, animeTitle: `Title ${i}`, value: titleCount - i });
|
||||
}
|
||||
return points;
|
||||
}
|
||||
|
||||
test('sortTooltipEntries orders rows by value descending, tolerating string values', () => {
|
||||
const sorted = sortTooltipEntries([
|
||||
{ name: 'A', value: 5 },
|
||||
{ name: 'B', value: 20 },
|
||||
{ name: 'C', value: '12.5' },
|
||||
{ name: 'D', value: undefined },
|
||||
]);
|
||||
|
||||
assert.deepEqual(
|
||||
sorted.map((entry) => entry.name),
|
||||
['B', 'C', 'A', 'D'],
|
||||
);
|
||||
});
|
||||
|
||||
test('tooltipColumnCount wraps into extra columns as items grow, capped at 3', () => {
|
||||
assert.equal(tooltipColumnCount(1), 1);
|
||||
assert.equal(tooltipColumnCount(8), 1);
|
||||
assert.equal(tooltipColumnCount(9), 2);
|
||||
assert.equal(tooltipColumnCount(16), 2);
|
||||
assert.equal(tooltipColumnCount(17), 3);
|
||||
assert.equal(tooltipColumnCount(40), 3);
|
||||
});
|
||||
|
||||
test('buildLineData keeps every title as a series instead of capping at the top 7', () => {
|
||||
const { points, seriesKeys } = buildLineData(makePoints(17));
|
||||
|
||||
assert.equal(seriesKeys.length, 17);
|
||||
assert.ok(seriesKeys.includes('Title 16'));
|
||||
for (const row of points) {
|
||||
for (const key of seriesKeys) {
|
||||
assert.ok(key in row);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('buildLineData caps series at maxSeries when set, keeping the top titles', () => {
|
||||
const { points, seriesKeys } = buildLineData(makePoints(17), 7);
|
||||
|
||||
assert.equal(seriesKeys.length, 7);
|
||||
assert.deepEqual(
|
||||
seriesKeys,
|
||||
Array.from({ length: 7 }, (_, i) => `Title ${i}`),
|
||||
);
|
||||
for (const row of points) {
|
||||
assert.ok(!('Title 16' in row));
|
||||
}
|
||||
});
|
||||
|
||||
test('buildLineData orders series by total value descending', () => {
|
||||
const { seriesKeys } = buildLineData([
|
||||
{ epochDay: 20_000, animeTitle: 'Small', value: 1 },
|
||||
{ epochDay: 20_000, animeTitle: 'Big', value: 10 },
|
||||
{ epochDay: 20_001, animeTitle: 'Medium', value: 5 },
|
||||
]);
|
||||
|
||||
assert.deepEqual(seriesKeys, ['Big', 'Medium', 'Small']);
|
||||
});
|
||||
|
||||
test('buildLineData total mode ranks by final cumulative value, not the sum of snapshots', () => {
|
||||
// "Early" plateaus at 10 across four days (snapshot sum 40); "Late" reaches a
|
||||
// larger final value of 12 in a single day. Ranking by final value keeps Late
|
||||
// first; the old snapshot-sum ranking would have put Early first.
|
||||
const raw: PerAnimeDataPoint[] = [
|
||||
{ epochDay: 20_000, animeTitle: 'Early', value: 10 },
|
||||
{ epochDay: 20_001, animeTitle: 'Early', value: 10 },
|
||||
{ epochDay: 20_002, animeTitle: 'Early', value: 10 },
|
||||
{ epochDay: 20_003, animeTitle: 'Early', value: 10 },
|
||||
{ epochDay: 20_003, animeTitle: 'Late', value: 12 },
|
||||
];
|
||||
|
||||
const { seriesKeys } = buildLineData(raw, undefined, 'total');
|
||||
assert.deepEqual(seriesKeys, ['Late', 'Early']);
|
||||
});
|
||||
|
||||
test('buildLineData recent mode keeps the most recently active titles over the largest', () => {
|
||||
// "Old Giant" has a huge cumulative total but stopped growing early; "Fresh"
|
||||
// is small but its cumulative value increased on the latest day.
|
||||
const raw: PerAnimeDataPoint[] = [
|
||||
{ epochDay: 20_000, animeTitle: 'Old Giant', value: 500 },
|
||||
{ epochDay: 20_001, animeTitle: 'Old Giant', value: 500 },
|
||||
{ epochDay: 20_002, animeTitle: 'Old Giant', value: 500 },
|
||||
{ epochDay: 20_000, animeTitle: 'Fresh', value: 1 },
|
||||
{ epochDay: 20_001, animeTitle: 'Fresh', value: 2 },
|
||||
{ epochDay: 20_002, animeTitle: 'Fresh', value: 5 },
|
||||
];
|
||||
|
||||
const total = buildLineData(raw, 1, 'total');
|
||||
assert.deepEqual(total.seriesKeys, ['Old Giant']);
|
||||
|
||||
const recent = buildLineData(raw, 1, 'recent');
|
||||
assert.deepEqual(recent.seriesKeys, ['Fresh']);
|
||||
});
|
||||
|
||||
test('buildLineData recent mode breaks ties by total then name', () => {
|
||||
// Both titles last increased on day 20_002; the larger total wins the tie.
|
||||
const raw: PerAnimeDataPoint[] = [
|
||||
{ epochDay: 20_001, animeTitle: 'A', value: 2 },
|
||||
{ epochDay: 20_002, animeTitle: 'A', value: 4 },
|
||||
{ epochDay: 20_001, animeTitle: 'B', value: 3 },
|
||||
{ epochDay: 20_002, animeTitle: 'B', value: 9 },
|
||||
];
|
||||
|
||||
const { seriesKeys } = buildLineData(raw, 1, 'recent');
|
||||
assert.deepEqual(seriesKeys, ['B']);
|
||||
});
|
||||
@@ -1,275 +0,0 @@
|
||||
import {
|
||||
AreaChart,
|
||||
Area,
|
||||
CartesianGrid,
|
||||
XAxis,
|
||||
YAxis,
|
||||
Tooltip,
|
||||
ResponsiveContainer,
|
||||
} from 'recharts';
|
||||
import { CHART_DEFAULTS, CHART_THEME, TOOLTIP_CONTENT_STYLE } from '../../lib/chart-theme';
|
||||
import { epochDayToDate } from '../../lib/formatters';
|
||||
|
||||
export interface PerAnimeDataPoint {
|
||||
epochDay: number;
|
||||
animeTitle: string;
|
||||
value: number;
|
||||
}
|
||||
|
||||
interface StackedTrendChartProps {
|
||||
title: string;
|
||||
data: PerAnimeDataPoint[];
|
||||
colorPalette?: string[];
|
||||
maxSeries?: number | null;
|
||||
maxSeriesMode?: SeriesRankMode;
|
||||
}
|
||||
|
||||
const DEFAULT_LINE_COLORS = [
|
||||
'#8aadf4',
|
||||
'#c6a0f6',
|
||||
'#a6da95',
|
||||
'#f5a97f',
|
||||
'#f5bde6',
|
||||
'#91d7e3',
|
||||
'#ee99a0',
|
||||
'#f4dbd6',
|
||||
];
|
||||
|
||||
// Wrap the tooltip into extra columns once a single column would get too tall,
|
||||
// keeping it compact instead of overflowing behind the charts below it.
|
||||
const TOOLTIP_ROWS_PER_COLUMN = 8;
|
||||
const TOOLTIP_MAX_COLUMNS = 3;
|
||||
|
||||
export function tooltipColumnCount(itemCount: number): number {
|
||||
const columns = Math.ceil(itemCount / TOOLTIP_ROWS_PER_COLUMN);
|
||||
return Math.min(TOOLTIP_MAX_COLUMNS, Math.max(1, columns));
|
||||
}
|
||||
|
||||
interface TooltipEntry {
|
||||
name?: string | number;
|
||||
value?: string | number;
|
||||
color?: string;
|
||||
}
|
||||
|
||||
interface StackedTooltipProps {
|
||||
active?: boolean;
|
||||
label?: string | number;
|
||||
payload?: TooltipEntry[];
|
||||
}
|
||||
|
||||
function tooltipEntryValue(entry: TooltipEntry): number {
|
||||
const numeric = typeof entry.value === 'number' ? entry.value : Number(entry.value);
|
||||
return Number.isFinite(numeric) ? numeric : 0;
|
||||
}
|
||||
|
||||
// Tooltip rows are sorted by value (largest first) so the biggest contributors
|
||||
// read top-down, independent of the series/stacking order.
|
||||
export function sortTooltipEntries<T extends TooltipEntry>(entries: readonly T[]): T[] {
|
||||
return [...entries].sort((a, b) => tooltipEntryValue(b) - tooltipEntryValue(a));
|
||||
}
|
||||
|
||||
function StackedTooltip({ active, label, payload }: StackedTooltipProps) {
|
||||
if (!active || !payload || payload.length === 0) {
|
||||
return null;
|
||||
}
|
||||
const sorted = sortTooltipEntries(payload);
|
||||
const columns = tooltipColumnCount(sorted.length);
|
||||
return (
|
||||
<div
|
||||
style={{
|
||||
...TOOLTIP_CONTENT_STYLE,
|
||||
padding: '6px 10px',
|
||||
maxWidth: '90vw',
|
||||
}}
|
||||
>
|
||||
{label !== undefined && (
|
||||
<div style={{ color: CHART_THEME.tooltipLabel, marginBottom: 4, fontWeight: 600 }}>
|
||||
{label}
|
||||
</div>
|
||||
)}
|
||||
<div
|
||||
style={{
|
||||
display: 'grid',
|
||||
gridTemplateColumns: `repeat(${columns}, max-content)`,
|
||||
columnGap: 16,
|
||||
rowGap: 2,
|
||||
}}
|
||||
>
|
||||
{sorted.map((entry, index) => (
|
||||
<div
|
||||
key={`${entry.name ?? index}`}
|
||||
style={{ display: 'flex', alignItems: 'center', gap: 6, whiteSpace: 'nowrap' }}
|
||||
>
|
||||
<span
|
||||
style={{
|
||||
display: 'inline-block',
|
||||
width: 8,
|
||||
height: 8,
|
||||
borderRadius: 9999,
|
||||
background: entry.color,
|
||||
flexShrink: 0,
|
||||
}}
|
||||
/>
|
||||
<span>{entry.name}</span>
|
||||
<span style={{ marginLeft: 16, color: CHART_THEME.tooltipLabel }}>{entry.value}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export type SeriesRankMode = 'total' | 'recent';
|
||||
|
||||
// Per-title ranking key. The data is cumulative per title, so `total` is the
|
||||
// final (latest) cumulative value — not the sum of daily snapshots, which would
|
||||
// over-weight titles that plateaued high early. `recentDay` is the latest epoch
|
||||
// day the value increased, i.e. its last day of real activity — used to keep
|
||||
// the most recently watched titles.
|
||||
function rankTitles(raw: PerAnimeDataPoint[]): Map<string, { total: number; recentDay: number }> {
|
||||
const pointsByTitle = new Map<string, PerAnimeDataPoint[]>();
|
||||
for (const entry of raw) {
|
||||
const list = pointsByTitle.get(entry.animeTitle) ?? [];
|
||||
list.push(entry);
|
||||
pointsByTitle.set(entry.animeTitle, list);
|
||||
}
|
||||
|
||||
const stats = new Map<string, { total: number; recentDay: number }>();
|
||||
for (const [title, points] of pointsByTitle) {
|
||||
const sorted = [...points].sort((a, b) => a.epochDay - b.epochDay);
|
||||
let previous = 0;
|
||||
let recentDay = Number.NEGATIVE_INFINITY;
|
||||
for (const point of sorted) {
|
||||
if (point.value > previous) {
|
||||
recentDay = point.epochDay;
|
||||
}
|
||||
previous = point.value;
|
||||
}
|
||||
const total = sorted.length > 0 ? sorted[sorted.length - 1]!.value : 0;
|
||||
stats.set(title, { total, recentDay });
|
||||
}
|
||||
return stats;
|
||||
}
|
||||
|
||||
export function buildLineData(
|
||||
raw: PerAnimeDataPoint[],
|
||||
maxSeries?: number | null,
|
||||
mode: SeriesRankMode = 'total',
|
||||
) {
|
||||
const stats = rankTitles(raw);
|
||||
|
||||
let seriesKeys = [...stats.entries()]
|
||||
.sort((a, b) => {
|
||||
if (mode === 'recent') {
|
||||
return (
|
||||
b[1].recentDay - a[1].recentDay || b[1].total - a[1].total || a[0].localeCompare(b[0])
|
||||
);
|
||||
}
|
||||
return b[1].total - a[1].total || a[0].localeCompare(b[0]);
|
||||
})
|
||||
.map(([title]) => title);
|
||||
if (typeof maxSeries === 'number' && maxSeries > 0) {
|
||||
seriesKeys = seriesKeys.slice(0, maxSeries);
|
||||
}
|
||||
const seriesSet = new Set(seriesKeys);
|
||||
|
||||
const byDay = new Map<number, Record<string, number>>();
|
||||
for (const entry of raw) {
|
||||
if (!seriesSet.has(entry.animeTitle)) continue;
|
||||
const row = byDay.get(entry.epochDay) ?? {};
|
||||
row[entry.animeTitle] = (row[entry.animeTitle] ?? 0) + Math.round(entry.value * 10) / 10;
|
||||
byDay.set(entry.epochDay, row);
|
||||
}
|
||||
|
||||
const points = [...byDay.entries()]
|
||||
.sort(([a], [b]) => a - b)
|
||||
.map(([epochDay, values]) => {
|
||||
const row: Record<string, string | number> = {
|
||||
label: epochDayToDate(epochDay).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
}),
|
||||
};
|
||||
for (const title of seriesKeys) {
|
||||
row[title] = values[title] ?? 0;
|
||||
}
|
||||
return row;
|
||||
});
|
||||
|
||||
return { points, seriesKeys };
|
||||
}
|
||||
|
||||
export function StackedTrendChart({
|
||||
title,
|
||||
data,
|
||||
colorPalette,
|
||||
maxSeries,
|
||||
maxSeriesMode,
|
||||
}: StackedTrendChartProps) {
|
||||
const { points, seriesKeys } = buildLineData(data, maxSeries, maxSeriesMode);
|
||||
const colors = colorPalette ?? DEFAULT_LINE_COLORS;
|
||||
|
||||
if (points.length === 0) {
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">{title}</h3>
|
||||
<div className="text-xs text-ctp-overlay2">No data</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">{title}</h3>
|
||||
<ResponsiveContainer width="100%" height={CHART_DEFAULTS.height}>
|
||||
<AreaChart data={points} margin={CHART_DEFAULTS.margin}>
|
||||
<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
width={32}
|
||||
/>
|
||||
<Tooltip
|
||||
content={<StackedTooltip />}
|
||||
wrapperStyle={{ zIndex: 50 }}
|
||||
allowEscapeViewBox={{ x: false, y: true }}
|
||||
/>
|
||||
{seriesKeys.map((key, i) => (
|
||||
<Area
|
||||
key={key}
|
||||
type="monotone"
|
||||
dataKey={key}
|
||||
stroke={colors[i % colors.length]}
|
||||
fill={colors[i % colors.length]}
|
||||
fillOpacity={0.15}
|
||||
strokeWidth={1.5}
|
||||
connectNulls
|
||||
/>
|
||||
))}
|
||||
</AreaChart>
|
||||
</ResponsiveContainer>
|
||||
<div className="flex flex-wrap gap-x-3 gap-y-1 mt-2">
|
||||
{seriesKeys.map((key, i) => (
|
||||
<span
|
||||
key={key}
|
||||
className="flex items-center gap-1 text-[10px] text-ctp-subtext0 whitespace-nowrap"
|
||||
title={key}
|
||||
>
|
||||
<span
|
||||
className="inline-block w-2 h-2 rounded-full shrink-0"
|
||||
style={{ backgroundColor: colors[i % colors.length] }}
|
||||
/>
|
||||
<span>{key}</span>
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
import type { TrendSummaryTotals, TrendsDashboardData } from '../../types/stats';
|
||||
import { formatCompactNumber, formatHours, formatNumber } from '../../lib/formatters';
|
||||
import { SummaryTile } from '../layout/SummaryTile';
|
||||
|
||||
interface SummaryStripProps {
|
||||
summary: TrendsDashboardData['summary'];
|
||||
days: number;
|
||||
}
|
||||
|
||||
type NumericKey = Exclude<keyof TrendSummaryTotals, 'activeDays'>;
|
||||
|
||||
const TILES: Array<{ key: NumericKey; label: string; format: (value: number) => string }> = [
|
||||
{ key: 'watchTimeMin', label: 'Watch time', format: formatHours },
|
||||
{ key: 'words', label: 'Words seen', format: formatCompactNumber },
|
||||
{ key: 'newWords', label: 'New words', format: formatNumber },
|
||||
{ key: 'cards', label: 'Cards mined', format: formatNumber },
|
||||
{ key: 'episodes', label: 'Episodes finished', format: formatNumber },
|
||||
];
|
||||
|
||||
/** Compares a total with the same-length period before it; null when both are zero. */
|
||||
export function describePeriodChange(
|
||||
current: number,
|
||||
previous: number,
|
||||
days: number,
|
||||
): string | null {
|
||||
const period = `previous ${days} day${days === 1 ? '' : 's'}`;
|
||||
if (previous === 0) return current === 0 ? null : `None in ${period}`;
|
||||
if (current === previous) return `Same as ${period}`;
|
||||
const percent = Math.round(((current - previous) / previous) * 100);
|
||||
if (percent === 0) return `Under 1% from ${period}`;
|
||||
return `${percent > 0 ? '+' : '\u2212'}${Math.abs(percent)}% vs ${period}`;
|
||||
}
|
||||
|
||||
function isEmptyPeriod(totals: TrendSummaryTotals): boolean {
|
||||
return totals.activeDays === 0 && TILES.every(({ key }) => !totals[key]);
|
||||
}
|
||||
|
||||
export function SummaryStrip({ summary, days }: SummaryStripProps) {
|
||||
const { current } = summary;
|
||||
// An empty period gets one note, because the same sentence on every tile reads as missing data.
|
||||
const previousIsEmpty = summary.previous !== null && isEmptyPeriod(summary.previous);
|
||||
const previous = previousIsEmpty ? null : summary.previous;
|
||||
return (
|
||||
<section
|
||||
aria-label="Period summary"
|
||||
className="grid grid-cols-2 gap-4 rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4 sm:grid-cols-3 lg:grid-cols-6"
|
||||
>
|
||||
{TILES.map(({ key, label, format }) => {
|
||||
const value = current[key];
|
||||
if (value === null) return null;
|
||||
const previousValue = previous?.[key] ?? null;
|
||||
return (
|
||||
<SummaryTile
|
||||
key={key}
|
||||
label={label}
|
||||
value={format(value)}
|
||||
detail={
|
||||
previousValue === null ? null : describePeriodChange(value, previousValue, days)
|
||||
}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
<SummaryTile
|
||||
label="Active days"
|
||||
value={formatNumber(current.activeDays)}
|
||||
detail={previous ? `${formatNumber(previous.activeDays)} in previous ${days}` : null}
|
||||
/>
|
||||
{previousIsEmpty ? (
|
||||
<p className="col-span-full text-[11px] text-ctp-overlay2">
|
||||
Not watched in the previous {days} day{days === 1 ? '' : 's'}
|
||||
</p>
|
||||
) : null}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
interface TitleFilterProps {
|
||||
titles: string[];
|
||||
query: string;
|
||||
selectedTitle: string | null;
|
||||
onQueryChange: (query: string) => void;
|
||||
onSelectTitle: (title: string | null) => void;
|
||||
}
|
||||
|
||||
const TITLE_OPTIONS_ID = 'trends-title-options';
|
||||
|
||||
/**
|
||||
* Whether an input event came from picking a datalist option rather than typing.
|
||||
* Firefox reports insertReplacementText. Chromium, which the stats window runs on,
|
||||
* sends a plain Event with no inputType, while typing always reports one.
|
||||
*/
|
||||
export function isDatalistSelection(event: Event): boolean {
|
||||
const inputType = 'inputType' in event ? event.inputType : undefined;
|
||||
return inputType === undefined || inputType === 'insertReplacementText';
|
||||
}
|
||||
|
||||
/**
|
||||
* Typing narrows the title lists below. Picking a suggestion, or pressing Enter on
|
||||
* an exact title, filters the whole page to it, as choosing it from a list row does.
|
||||
* Typing alone never selects, because one title can be the start of another.
|
||||
*/
|
||||
export function TitleFilter({
|
||||
titles,
|
||||
query,
|
||||
selectedTitle,
|
||||
onQueryChange,
|
||||
onSelectTitle,
|
||||
}: TitleFilterProps) {
|
||||
const selectIfTitle = (value: string): boolean => {
|
||||
if (!titles.includes(value)) return false;
|
||||
onSelectTitle(value);
|
||||
onQueryChange('');
|
||||
return true;
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex min-w-0 flex-wrap items-center gap-2 text-xs">
|
||||
<input
|
||||
type="search"
|
||||
aria-label="Filter by title"
|
||||
placeholder="Filter by title..."
|
||||
list={TITLE_OPTIONS_ID}
|
||||
value={query}
|
||||
onChange={(event) => {
|
||||
const value = event.target.value;
|
||||
if (!isDatalistSelection(event.nativeEvent) || !selectIfTitle(value)) {
|
||||
onQueryChange(value);
|
||||
}
|
||||
}}
|
||||
onKeyDown={(event) => {
|
||||
// Enter during IME composition confirms the conversion, not the filter.
|
||||
if (event.key === 'Enter' && !event.nativeEvent.isComposing) {
|
||||
selectIfTitle(event.currentTarget.value);
|
||||
}
|
||||
}}
|
||||
className="w-56 max-w-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-2.5 py-1 text-xs text-ctp-text placeholder:text-ctp-overlay2 focus:border-ctp-blue focus:outline-none"
|
||||
/>
|
||||
<datalist id={TITLE_OPTIONS_ID}>
|
||||
{titles.map((title) => (
|
||||
<option key={title} value={title} />
|
||||
))}
|
||||
</datalist>
|
||||
{selectedTitle ? (
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Clear title filter: ${selectedTitle}`}
|
||||
title="Clear title filter"
|
||||
className="flex max-w-xs items-center gap-1.5 rounded-full border border-ctp-blue/60 bg-ctp-blue/12 px-2.5 py-1 text-ctp-blue transition hover:border-ctp-blue"
|
||||
onClick={() => onSelectTitle(null)}
|
||||
>
|
||||
<span className="truncate">{selectedTitle}</span>
|
||||
<span aria-hidden="true">×</span>
|
||||
</button>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import type { LibrarySummaryRow, TrendPerAnimePoint } from '../../types/stats';
|
||||
import { epochDayToDate, formatHours, formatNumber } from '../../lib/formatters';
|
||||
import { heatmapCellOpacity } from './DayHourHeatmap';
|
||||
import { buildTitleTimeline } from './title-timeline';
|
||||
|
||||
interface TitleTimelineProps {
|
||||
/** Titles to show, most watched first. */
|
||||
rows: LibrarySummaryRow[];
|
||||
points: TrendPerAnimePoint[];
|
||||
window: { startDay: number; endDay: number };
|
||||
selectedTitle: string | null;
|
||||
onSelectTitle: (title: string) => void;
|
||||
onOpenTitle: (row: LibrarySummaryRow) => void;
|
||||
}
|
||||
|
||||
const DEFAULT_VISIBLE_ROWS = 12;
|
||||
|
||||
function formatCellDate(epochDay: number): string {
|
||||
return epochDayToDate(epochDay).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
year: 'numeric',
|
||||
});
|
||||
}
|
||||
|
||||
/** One row per title, shaded on the days it was watched, so viewing bursts are visible at a glance. */
|
||||
export function TitleTimeline({
|
||||
rows,
|
||||
points,
|
||||
window,
|
||||
selectedTitle,
|
||||
onSelectTitle,
|
||||
onOpenTitle,
|
||||
}: TitleTimelineProps) {
|
||||
const [showAll, setShowAll] = useState(false);
|
||||
const model = useMemo(() => buildTitleTimeline(points, window), [points, window]);
|
||||
const visibleRows = showAll ? rows : rows.slice(0, DEFAULT_VISIBLE_ROWS);
|
||||
|
||||
return (
|
||||
<div className="col-span-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="mb-2 text-xs font-semibold text-ctp-text">
|
||||
What You Watched and When
|
||||
{model.columnDays > 1 ? (
|
||||
<span className="ml-2 font-normal text-ctp-overlay2">One column per week</span>
|
||||
) : null}
|
||||
</h3>
|
||||
{rows.length === 0 ? (
|
||||
<div className="text-xs text-ctp-overlay2">No titles in this range.</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="grid grid-cols-[minmax(7rem,14rem)_minmax(0,1fr)_2.5rem_2.5rem] items-center gap-x-2 gap-y-px">
|
||||
<div />
|
||||
<div className="relative h-4">
|
||||
{model.ticks.map((tick) => (
|
||||
<span
|
||||
key={tick.leftPercent}
|
||||
className="absolute top-0 text-[10px] whitespace-nowrap text-ctp-overlay2"
|
||||
style={{ left: `${tick.leftPercent}%` }}
|
||||
>
|
||||
{tick.label}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
<div className="text-right text-[10px] text-ctp-overlay2">Time</div>
|
||||
<div className="text-right text-[10px] text-ctp-overlay2">Cards</div>
|
||||
{visibleRows.map((row) => {
|
||||
const isSelected = row.title === selectedTitle;
|
||||
return (
|
||||
<div key={row.title} className="contents">
|
||||
<div className="flex min-w-0 items-center gap-1">
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={isSelected}
|
||||
title={`Filter the page to ${row.title}`}
|
||||
className={`min-w-0 truncate text-left text-[11px] transition hover:text-ctp-blue ${
|
||||
isSelected ? 'font-semibold text-ctp-blue' : 'text-ctp-text'
|
||||
}`}
|
||||
onClick={() => onSelectTitle(row.title)}
|
||||
>
|
||||
{row.title}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Open ${row.title}`}
|
||||
title="Open this title"
|
||||
className="shrink-0 text-[11px] text-ctp-overlay2 transition hover:text-ctp-blue"
|
||||
onClick={() => onOpenTitle(row)}
|
||||
>
|
||||
↗
|
||||
</button>
|
||||
</div>
|
||||
<div
|
||||
className={`relative h-4 rounded-sm ${isSelected ? 'bg-ctp-surface2/60' : 'bg-ctp-surface1/30'}`}
|
||||
>
|
||||
{model.ticks.map((tick) => (
|
||||
<span
|
||||
key={tick.leftPercent}
|
||||
className="absolute inset-y-0 w-px bg-ctp-surface2/60"
|
||||
style={{ left: `${tick.leftPercent}%` }}
|
||||
/>
|
||||
))}
|
||||
{(model.cellsByTitle.get(row.title) ?? []).map((cell) => (
|
||||
<span
|
||||
key={cell.startDay}
|
||||
title={`${row.title}\n${model.columnDays > 1 ? 'Week of ' : ''}${formatCellDate(cell.startDay)}: ${formatHours(cell.minutes)}`}
|
||||
className="absolute inset-y-px min-w-[2px] rounded-[1px] bg-ctp-blue"
|
||||
style={{
|
||||
left: `${cell.leftPercent}%`,
|
||||
width: `${cell.widthPercent}%`,
|
||||
opacity: heatmapCellOpacity(cell.minutes, model.maxCellMinutes),
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
<div className="text-right text-[10px] text-ctp-subtext0 tabular-nums">
|
||||
{formatHours(row.watchTimeMin)}
|
||||
</div>
|
||||
<div className="text-right text-[10px] text-ctp-subtext0 tabular-nums">
|
||||
{formatNumber(row.cards)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
{rows.length > DEFAULT_VISIBLE_ROWS ? (
|
||||
<button
|
||||
type="button"
|
||||
className="mt-2 text-[11px] text-ctp-overlay2 transition hover:text-ctp-blue"
|
||||
onClick={() => setShowAll((value) => !value)}
|
||||
>
|
||||
{showAll ? 'Show fewer' : `Show all ${formatNumber(rows.length)} titles`}
|
||||
</button>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -10,39 +10,85 @@ import {
|
||||
ResponsiveContainer,
|
||||
} from 'recharts';
|
||||
import { CHART_DEFAULTS, CHART_THEME, TOOLTIP_CONTENT_STYLE } from '../../lib/chart-theme';
|
||||
import { formatCompactNumber, formatNumber } from '../../lib/formatters';
|
||||
|
||||
interface TrendChartProps {
|
||||
title: string;
|
||||
data: Array<{ label: string; value: number }>;
|
||||
/** A null value is a gap: no data to plot, as opposed to a real zero. */
|
||||
data: Array<{ label: string; value: number | null }>;
|
||||
color: string;
|
||||
type: 'bar' | 'line';
|
||||
formatter?: (value: number) => string;
|
||||
onBarClick?: (label: string) => void;
|
||||
/** Formats axis ticks. Defaults to compact numbers such as 24K. */
|
||||
tickFormatter?: (value: number) => string;
|
||||
/** Formats the tooltip value. Defaults to the full number. */
|
||||
valueFormatter?: (value: number) => string;
|
||||
onBarClick?: (label: string, index: number) => void;
|
||||
height?: number;
|
||||
}
|
||||
|
||||
export function TrendChart({ title, data, color, type, formatter, onBarClick }: TrendChartProps) {
|
||||
const formatValue = (v: number) => (formatter ? [formatter(v), title] : [String(v), title]);
|
||||
const Y_AXIS_WIDTH = 40;
|
||||
|
||||
/** A point with gaps on both sides draws no line segment, so it gets a dot instead. */
|
||||
export function isIsolatedPoint(values: ReadonlyArray<number | null>, index: number): boolean {
|
||||
return (
|
||||
values[index] !== null &&
|
||||
values[index] !== undefined &&
|
||||
(values[index - 1] ?? null) === null &&
|
||||
(values[index + 1] ?? null) === null
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Leads a running total with zero. Each point is the total at the end of its period, so
|
||||
* without this the line would begin at the first day's whole total.
|
||||
*/
|
||||
export function startFromZero<T extends { label: string; value: number }>(
|
||||
points: T[],
|
||||
): Array<T | { label: string; value: number }> {
|
||||
return points.length > 0 ? [{ label: 'Start', value: 0 }, ...points] : points;
|
||||
}
|
||||
|
||||
export function TrendChart({
|
||||
title,
|
||||
data,
|
||||
color,
|
||||
type,
|
||||
tickFormatter = formatCompactNumber,
|
||||
valueFormatter = formatNumber,
|
||||
onBarClick,
|
||||
height = CHART_DEFAULTS.height,
|
||||
}: TrendChartProps) {
|
||||
const formatValue = (value: unknown) =>
|
||||
typeof value === 'number' ? [valueFormatter(value), title] : ['Not enough data', title];
|
||||
const values = data.map((point) => point.value);
|
||||
|
||||
const xAxis = (
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
);
|
||||
const yAxis = (
|
||||
<YAxis
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
width={Y_AXIS_WIDTH}
|
||||
tickFormatter={tickFormatter}
|
||||
/>
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<div className="h-full bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text mb-2">{title}</h3>
|
||||
<ResponsiveContainer width="100%" height={CHART_DEFAULTS.height}>
|
||||
<ResponsiveContainer width="100%" height={height}>
|
||||
{type === 'bar' ? (
|
||||
<BarChart data={data} margin={CHART_DEFAULTS.margin}>
|
||||
<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
width={32}
|
||||
tickFormatter={formatter}
|
||||
/>
|
||||
{xAxis}
|
||||
{yAxis}
|
||||
<Tooltip contentStyle={TOOLTIP_CONTENT_STYLE} formatter={formatValue} />
|
||||
<Bar
|
||||
dataKey="value"
|
||||
@@ -50,28 +96,32 @@ export function TrendChart({ title, data, color, type, formatter, onBarClick }:
|
||||
radius={[2, 2, 0, 0]}
|
||||
cursor={onBarClick ? 'pointer' : undefined}
|
||||
onClick={
|
||||
onBarClick ? (entry: { label: string }) => onBarClick(entry.label) : undefined
|
||||
onBarClick
|
||||
? (entry: { label: string }, index: number) => onBarClick(entry.label, index)
|
||||
: undefined
|
||||
}
|
||||
/>
|
||||
</BarChart>
|
||||
) : (
|
||||
<LineChart data={data} margin={CHART_DEFAULTS.margin}>
|
||||
<CartesianGrid stroke={CHART_THEME.grid} {...CHART_DEFAULTS.grid} />
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: CHART_DEFAULTS.tickFontSize, fill: CHART_THEME.tick }}
|
||||
axisLine={{ stroke: CHART_THEME.axisLine }}
|
||||
tickLine={false}
|
||||
width={32}
|
||||
tickFormatter={formatter}
|
||||
/>
|
||||
{xAxis}
|
||||
{yAxis}
|
||||
<Tooltip contentStyle={TOOLTIP_CONTENT_STYLE} formatter={formatValue} />
|
||||
<Line dataKey="value" stroke={color} strokeWidth={2} dot={false} />
|
||||
<Line
|
||||
dataKey="value"
|
||||
stroke={color}
|
||||
strokeWidth={2}
|
||||
dot={(props: { cx?: number; cy?: number; index?: number }) => (
|
||||
<circle
|
||||
key={props.index}
|
||||
cx={props.cx}
|
||||
cy={props.cy}
|
||||
r={props.index !== undefined && isIsolatedPoint(values, props.index) ? 2.5 : 0}
|
||||
fill={color}
|
||||
/>
|
||||
)}
|
||||
/>
|
||||
</LineChart>
|
||||
)}
|
||||
</ResponsiveContainer>
|
||||
|
||||
@@ -1,43 +1,138 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { AnimeVisibilityFilter } from './TrendsTab';
|
||||
import { parseDateDrafts } from './DateRangeSelector';
|
||||
import { HEATMAP_HOURS, heatmapCellOpacity } from './DayHourHeatmap';
|
||||
import { describePeriodChange, SummaryStrip } from './SummaryStrip';
|
||||
import { isDatalistSelection } from './TitleFilter';
|
||||
import { isIsolatedPoint, startFromZero } from './TrendChart';
|
||||
|
||||
test('AnimeVisibilityFilter offers a per-chart title limit selector', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<AnimeVisibilityFilter
|
||||
animeTitles={['KonoSuba']}
|
||||
hiddenAnime={new Set()}
|
||||
maxTitles={7}
|
||||
maxTitlesMode="total"
|
||||
onShowAll={() => {}}
|
||||
onHideAll={() => {}}
|
||||
onToggleAnime={() => {}}
|
||||
onMaxTitlesChange={() => {}}
|
||||
onMaxTitlesModeChange={() => {}}
|
||||
/>,
|
||||
);
|
||||
|
||||
assert.match(markup, /per chart/);
|
||||
assert.match(markup, /<option value="all">All<\/option>/);
|
||||
assert.match(markup, /<option value="7" selected="">/);
|
||||
test('heatmap hours start at 06:00 so a late evening is not split across the axis', () => {
|
||||
assert.equal(HEATMAP_HOURS[0], 6);
|
||||
assert.equal(HEATMAP_HOURS.indexOf(23) + 1, HEATMAP_HOURS.indexOf(0));
|
||||
assert.equal(HEATMAP_HOURS.length, 24);
|
||||
});
|
||||
|
||||
test('AnimeVisibilityFilter keeps the ranking mode selectable even when showing all titles', () => {
|
||||
test('heatmap opacity leaves empty cells uncolored and keeps small values visible', () => {
|
||||
assert.equal(heatmapCellOpacity(0, 100), 0);
|
||||
assert.ok(heatmapCellOpacity(1, 100) > 0.2);
|
||||
assert.equal(heatmapCellOpacity(100, 100), 1);
|
||||
});
|
||||
|
||||
for (const c of [
|
||||
{ name: 'a decrease', current: 88, previous: 100, expected: '\u221212% vs previous 30 days' },
|
||||
{ name: 'an increase', current: 150, previous: 100, expected: '+50% vs previous 30 days' },
|
||||
{ name: 'no change', current: 100, previous: 100, expected: 'Same as previous 30 days' },
|
||||
{
|
||||
name: 'a tiny change',
|
||||
current: 1001,
|
||||
previous: 1000,
|
||||
expected: 'Under 1% from previous 30 days',
|
||||
},
|
||||
{ name: 'nothing before', current: 5, previous: 0, expected: 'None in previous 30 days' },
|
||||
{ name: 'nothing at all', current: 0, previous: 0, expected: null },
|
||||
]) {
|
||||
test(`describePeriodChange reports ${c.name}`, () => {
|
||||
assert.equal(describePeriodChange(c.current, c.previous, 30), c.expected);
|
||||
});
|
||||
}
|
||||
|
||||
test('SummaryStrip hides new words while filtered to a title and comparison on all-time', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<AnimeVisibilityFilter
|
||||
animeTitles={['KonoSuba']}
|
||||
hiddenAnime={new Set()}
|
||||
maxTitles={null}
|
||||
maxTitlesMode="total"
|
||||
onShowAll={() => {}}
|
||||
onHideAll={() => {}}
|
||||
onToggleAnime={() => {}}
|
||||
onMaxTitlesChange={() => {}}
|
||||
onMaxTitlesModeChange={() => {}}
|
||||
<SummaryStrip
|
||||
days={231}
|
||||
summary={{
|
||||
current: {
|
||||
watchTimeMin: 90,
|
||||
words: 24_000,
|
||||
newWords: null,
|
||||
cards: 3,
|
||||
episodes: 1,
|
||||
activeDays: 2,
|
||||
},
|
||||
previous: null,
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
assert.match(markup, /1\.5h/);
|
||||
assert.doesNotMatch(markup, /New words/);
|
||||
assert.doesNotMatch(markup, /previous/);
|
||||
});
|
||||
|
||||
assert.match(markup, /aria-label="Title ranking mode"/);
|
||||
assert.doesNotMatch(markup, /aria-label="Title ranking mode"[^>]*disabled/);
|
||||
const SUMMARY_CURRENT = {
|
||||
watchTimeMin: 210,
|
||||
words: 15_900,
|
||||
newWords: null,
|
||||
cards: 35,
|
||||
episodes: 11,
|
||||
activeDays: 3,
|
||||
};
|
||||
const EMPTY_PERIOD = {
|
||||
watchTimeMin: 0,
|
||||
words: 0,
|
||||
newWords: null,
|
||||
cards: 0,
|
||||
episodes: 0,
|
||||
activeDays: 0,
|
||||
};
|
||||
|
||||
test('SummaryStrip shows one note instead of repeating an empty previous period', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<SummaryStrip days={30} summary={{ current: SUMMARY_CURRENT, previous: EMPTY_PERIOD }} />,
|
||||
);
|
||||
assert.equal(markup.match(/previous 30/g)?.length, 1);
|
||||
assert.match(markup, /Not watched in the previous 30 days/);
|
||||
assert.doesNotMatch(markup, /None in previous/);
|
||||
});
|
||||
|
||||
test('SummaryStrip still compares each tile when the previous period has any activity', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<SummaryStrip
|
||||
days={30}
|
||||
summary={{
|
||||
current: SUMMARY_CURRENT,
|
||||
previous: { ...EMPTY_PERIOD, watchTimeMin: 105, words: 7_950, activeDays: 1 },
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
assert.match(markup, /\+100% vs previous 30 days/);
|
||||
assert.match(markup, /None in previous 30 days/);
|
||||
assert.match(markup, /1 in previous 30/);
|
||||
assert.doesNotMatch(markup, /Not watched/);
|
||||
});
|
||||
|
||||
test('startFromZero leads a running total with a zero point', () => {
|
||||
const points = [
|
||||
{ label: 'Oct 24', value: 106 },
|
||||
{ label: 'Oct 25', value: 155 },
|
||||
];
|
||||
assert.deepEqual(startFromZero(points), [{ label: 'Start', value: 0 }, ...points]);
|
||||
assert.deepEqual(startFromZero([]), []);
|
||||
});
|
||||
|
||||
test('isIsolatedPoint is true only for a value with gaps on both sides', () => {
|
||||
const values = [null, 3, null, 4, 5, null];
|
||||
assert.equal(isIsolatedPoint(values, 1), true);
|
||||
assert.equal(isIsolatedPoint(values, 3), false);
|
||||
assert.equal(isIsolatedPoint(values, 2), false);
|
||||
});
|
||||
|
||||
test('isDatalistSelection separates picking a suggestion from typing', () => {
|
||||
const inputEvent = (inputType: string) => Object.assign(new Event('input'), { inputType });
|
||||
assert.equal(isDatalistSelection(new Event('input')), true);
|
||||
assert.equal(isDatalistSelection(inputEvent('insertReplacementText')), true);
|
||||
assert.equal(isDatalistSelection(inputEvent('insertText')), false);
|
||||
assert.equal(isDatalistSelection(inputEvent('deleteContentBackward')), false);
|
||||
});
|
||||
|
||||
test('parseDateDrafts waits for two complete dates in order', () => {
|
||||
const end = '2026-03-01';
|
||||
assert.deepEqual(parseDateDrafts({ start: '2026-02-01', end }), {
|
||||
startDay: 20_485,
|
||||
endDay: 20_513,
|
||||
});
|
||||
// What a date input reports partway through typing a year, or when cleared.
|
||||
assert.equal(parseDateDrafts({ start: '0002-02-01', end }), null);
|
||||
assert.equal(parseDateDrafts({ start: '', end }), null);
|
||||
assert.equal(parseDateDrafts({ start: '2026-03-02', end }), null);
|
||||
});
|
||||
@@ -1,368 +1,367 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import { useTrends, type TimeRange, type GroupBy } from '../../hooks/useTrends';
|
||||
import { useMemo, useState, type ReactNode } from 'react';
|
||||
import { useTrends, type GroupBy, type TimeRange } from '../../hooks/useTrends';
|
||||
import { formatDuration, formatNumber } from '../../lib/formatters';
|
||||
import type { SessionsDayFilter } from '../../lib/stats-navigation';
|
||||
import type { LibrarySummaryRow, TrendBucket, TrendChartPoint } from '../../types/stats';
|
||||
import { DateRangeSelector } from './DateRangeSelector';
|
||||
import { TrendChart } from './TrendChart';
|
||||
import { StackedTrendChart } from './StackedTrendChart';
|
||||
import {
|
||||
MAX_TITLES_MODES,
|
||||
MAX_TITLES_OPTIONS,
|
||||
buildAnimeVisibilityOptions,
|
||||
filterHiddenAnimeData,
|
||||
loadHiddenTitles,
|
||||
loadMaxTitles,
|
||||
loadMaxTitlesMode,
|
||||
loadShowEmptyDays,
|
||||
pruneHiddenAnime,
|
||||
saveHiddenTitles,
|
||||
saveMaxTitles,
|
||||
saveMaxTitlesMode,
|
||||
saveShowEmptyDays,
|
||||
type MaxTitlesMode,
|
||||
} from './anime-visibility';
|
||||
import { DayHourHeatmap } from './DayHourHeatmap';
|
||||
import { FrequencyBandChart } from './FrequencyBandChart';
|
||||
import { LibrarySummarySection } from './LibrarySummarySection';
|
||||
import { buildAnimeCumulative } from './anime-cumulative';
|
||||
import { SummaryStrip } from './SummaryStrip';
|
||||
import { TitleFilter } from './TitleFilter';
|
||||
import { TitleTimeline } from './TitleTimeline';
|
||||
import { startFromZero, TrendChart } from './TrendChart';
|
||||
import {
|
||||
RUNNING_TOTAL_METRICS,
|
||||
loadGroupBy,
|
||||
loadRange,
|
||||
loadRunningTotalMetric,
|
||||
loadShowEmptyDays,
|
||||
saveGroupBy,
|
||||
saveRange,
|
||||
saveRunningTotalMetric,
|
||||
saveShowEmptyDays,
|
||||
type RunningTotalMetric,
|
||||
} from './trends-preferences';
|
||||
|
||||
const MAX_TITLES_MODE_LABELS: Record<MaxTitlesMode, string> = {
|
||||
recent: 'most recent',
|
||||
total: 'top',
|
||||
interface TrendsTabProps {
|
||||
onNavigateToAnime?: (animeId: number) => void;
|
||||
onNavigateToMediaDetail?: (videoId: number) => void;
|
||||
onOpenSessionsForDays?: (filter: SessionsDayFilter) => void;
|
||||
}
|
||||
|
||||
const WATCH_TIME_COLOR = '#8aadf4';
|
||||
const WORDS_COLOR = '#8bd5ca';
|
||||
const NEW_WORDS_COLOR = '#c6a0f6';
|
||||
const EPISODES_COLOR = '#91d7e3';
|
||||
const LOOKUPS_COLOR = '#f5a97f';
|
||||
const READING_SPEED_COLOR = '#a6da95';
|
||||
const CARDS_COLOR = 'var(--color-ctp-cards-mined)';
|
||||
|
||||
const RUNNING_TOTAL_LABELS: Record<RunningTotalMetric, string> = {
|
||||
watchTime: 'Watch time',
|
||||
words: 'Words seen',
|
||||
newWords: 'New words',
|
||||
cards: 'Cards mined',
|
||||
episodes: 'Episodes finished',
|
||||
lookups: 'Lookups',
|
||||
};
|
||||
|
||||
function SectionHeader({ children }: { children: React.ReactNode }) {
|
||||
const RUNNING_TOTAL_COLORS: Record<RunningTotalMetric, string> = {
|
||||
watchTime: WATCH_TIME_COLOR,
|
||||
words: WORDS_COLOR,
|
||||
newWords: NEW_WORDS_COLOR,
|
||||
cards: CARDS_COLOR,
|
||||
episodes: EPISODES_COLOR,
|
||||
lookups: LOOKUPS_COLOR,
|
||||
};
|
||||
|
||||
function SectionHeader({ children, hint }: { children: ReactNode; hint?: string }) {
|
||||
return (
|
||||
<div className="col-span-full mt-6 mb-2 flex items-center gap-3">
|
||||
<h3 className="text-ctp-subtext0 text-xs font-semibold uppercase tracking-widest shrink-0">
|
||||
{children}
|
||||
</h3>
|
||||
<div className="flex-1 h-px bg-gradient-to-r from-ctp-surface1 to-transparent" />
|
||||
{hint ? <span className="shrink-0 text-[11px] text-ctp-overlay2">{hint}</span> : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
interface AnimeVisibilityFilterProps {
|
||||
animeTitles: string[];
|
||||
hiddenAnime: ReadonlySet<string>;
|
||||
maxTitles: number | null;
|
||||
maxTitlesMode: MaxTitlesMode;
|
||||
onShowAll: () => void;
|
||||
onHideAll: () => void;
|
||||
onToggleAnime: (title: string) => void;
|
||||
onMaxTitlesChange: (value: number | null) => void;
|
||||
onMaxTitlesModeChange: (mode: MaxTitlesMode) => void;
|
||||
/** Chart values in hours, so axis ticks land on round hours instead of odd minute counts. */
|
||||
function toHours(points: TrendChartPoint[]): TrendChartPoint[] {
|
||||
return points.map((point) => ({ ...point, value: point.value / 60 }));
|
||||
}
|
||||
|
||||
export function AnimeVisibilityFilter({
|
||||
animeTitles,
|
||||
hiddenAnime,
|
||||
maxTitles,
|
||||
maxTitlesMode,
|
||||
onShowAll,
|
||||
onHideAll,
|
||||
onToggleAnime,
|
||||
onMaxTitlesChange,
|
||||
onMaxTitlesModeChange,
|
||||
}: AnimeVisibilityFilterProps) {
|
||||
if (animeTitles.length === 0) {
|
||||
return null;
|
||||
}
|
||||
const formatHourTick = (hours: number): string => `${+hours.toFixed(2)}h`;
|
||||
const formatHourValue = (hours: number): string => formatDuration(hours * 3_600_000);
|
||||
|
||||
return (
|
||||
<div className="col-span-full -mt-1 mb-1 rounded-lg border border-ctp-surface1 bg-ctp-surface0/70 p-3">
|
||||
<div className="mb-2 flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<h4 className="text-xs font-semibold uppercase tracking-widest text-ctp-subtext0">
|
||||
Title Visibility
|
||||
</h4>
|
||||
<p className="mt-1 text-xs text-ctp-overlay1">
|
||||
Shared across all anime trend charts. Default: show everything. Selections are saved.
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<label className="flex items-center gap-1 text-[11px] text-ctp-subtext0">
|
||||
Show
|
||||
<select
|
||||
aria-label="Title ranking mode"
|
||||
className="rounded-md border border-ctp-surface2 bg-ctp-surface0 px-1.5 py-1 text-[11px] font-medium text-ctp-text transition hover:border-ctp-blue"
|
||||
value={maxTitlesMode}
|
||||
onChange={(event) => onMaxTitlesModeChange(event.target.value as MaxTitlesMode)}
|
||||
>
|
||||
{MAX_TITLES_MODES.map((mode) => (
|
||||
<option key={mode} value={mode}>
|
||||
{MAX_TITLES_MODE_LABELS[mode]}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
<select
|
||||
aria-label="Titles per chart"
|
||||
className="rounded-md border border-ctp-surface2 bg-ctp-surface0 px-1.5 py-1 text-[11px] font-medium text-ctp-text transition hover:border-ctp-blue"
|
||||
value={maxTitles === null ? 'all' : String(maxTitles)}
|
||||
onChange={(event) =>
|
||||
onMaxTitlesChange(event.target.value === 'all' ? null : Number(event.target.value))
|
||||
}
|
||||
>
|
||||
<option value="all">All</option>
|
||||
{MAX_TITLES_OPTIONS.map((count) => (
|
||||
<option key={count} value={count}>
|
||||
{count}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
per chart
|
||||
</label>
|
||||
<button
|
||||
type="button"
|
||||
className="rounded-md border border-ctp-surface2 px-2 py-1 text-[11px] font-medium text-ctp-text transition hover:border-ctp-blue hover:text-ctp-blue"
|
||||
onClick={onShowAll}
|
||||
>
|
||||
All
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="rounded-md border border-ctp-surface2 px-2 py-1 text-[11px] font-medium text-ctp-text transition hover:border-ctp-peach hover:text-ctp-peach"
|
||||
onClick={onHideAll}
|
||||
>
|
||||
None
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{animeTitles.map((title) => {
|
||||
const isVisible = !hiddenAnime.has(title);
|
||||
return (
|
||||
<button
|
||||
key={title}
|
||||
type="button"
|
||||
aria-pressed={isVisible}
|
||||
className={`max-w-full rounded-full border px-3 py-1 text-xs transition ${
|
||||
isVisible
|
||||
? 'border-ctp-blue/60 bg-ctp-blue/12 text-ctp-blue'
|
||||
: 'border-ctp-surface2 bg-transparent text-ctp-subtext0'
|
||||
}`}
|
||||
onClick={() => onToggleAnime(title)}
|
||||
title={title}
|
||||
>
|
||||
<span className="block truncate">{title}</span>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
function bucketSessionsLabel(bucket: TrendBucket, groupBy: GroupBy): string {
|
||||
return groupBy === 'week' ? `the week of ${bucket.label}` : bucket.label;
|
||||
}
|
||||
|
||||
export function TrendsTab() {
|
||||
const [range, setRange] = useState<TimeRange>('30d');
|
||||
const [groupBy, setGroupBy] = useState<GroupBy>('day');
|
||||
export function TrendsTab({
|
||||
onNavigateToAnime,
|
||||
onNavigateToMediaDetail,
|
||||
onOpenSessionsForDays,
|
||||
}: TrendsTabProps = {}) {
|
||||
const [range, setRange] = useState<TimeRange>(() => loadRange());
|
||||
const [groupBy, setGroupBy] = useState<GroupBy>(() => loadGroupBy());
|
||||
const [showEmptyDays, setShowEmptyDays] = useState(() => loadShowEmptyDays());
|
||||
const [hiddenAnime, setHiddenAnime] = useState<Set<string>>(() => loadHiddenTitles());
|
||||
const [maxTitles, setMaxTitles] = useState<number | null>(() => loadMaxTitles());
|
||||
const [maxTitlesMode, setMaxTitlesMode] = useState<MaxTitlesMode>(() => loadMaxTitlesMode());
|
||||
const { data, loading, error } = useTrends(range, groupBy, showEmptyDays);
|
||||
const animeCumulative = useMemo(
|
||||
() => (data ? buildAnimeCumulative(data.animeDaily) : null),
|
||||
[data],
|
||||
const [runningTotalMetric, setRunningTotalMetric] = useState(() => loadRunningTotalMetric());
|
||||
const [selectedTitle, setSelectedTitle] = useState<string | null>(null);
|
||||
const [titleQuery, setTitleQuery] = useState('');
|
||||
const { data, loading, refreshing, error } = useTrends(
|
||||
range,
|
||||
groupBy,
|
||||
showEmptyDays,
|
||||
selectedTitle,
|
||||
);
|
||||
|
||||
const updateHiddenAnime = (next: Set<string>) => {
|
||||
setHiddenAnime(next);
|
||||
saveHiddenTitles(next);
|
||||
const updateRange = (next: TimeRange) => {
|
||||
setRange(next);
|
||||
saveRange(next);
|
||||
};
|
||||
const updateMaxTitles = (value: number | null) => {
|
||||
setMaxTitles(value);
|
||||
saveMaxTitles(value);
|
||||
};
|
||||
const updateMaxTitlesMode = (mode: MaxTitlesMode) => {
|
||||
setMaxTitlesMode(mode);
|
||||
saveMaxTitlesMode(mode);
|
||||
const updateGroupBy = (next: GroupBy) => {
|
||||
setGroupBy(next);
|
||||
saveGroupBy(next);
|
||||
};
|
||||
const updateShowEmptyDays = (show: boolean) => {
|
||||
setShowEmptyDays(show);
|
||||
saveShowEmptyDays(show);
|
||||
};
|
||||
const cardsMinedColor = 'var(--color-ctp-cards-mined)';
|
||||
const cardsMinedStackedColors = [
|
||||
cardsMinedColor,
|
||||
'#8aadf4',
|
||||
'#c6a0f6',
|
||||
'#f5a97f',
|
||||
'#f5bde6',
|
||||
'#91d7e3',
|
||||
'#ee99a0',
|
||||
'#f4dbd6',
|
||||
];
|
||||
const updateRunningTotalMetric = (metric: RunningTotalMetric) => {
|
||||
setRunningTotalMetric(metric);
|
||||
saveRunningTotalMetric(metric);
|
||||
};
|
||||
const toggleSelectedTitle = (title: string) => {
|
||||
setSelectedTitle((current) => (current === title ? null : title));
|
||||
};
|
||||
|
||||
if (loading) return <div className="text-ctp-overlay2 p-4">Loading...</div>;
|
||||
if (error) return <div className="text-ctp-red p-4">Error: {error}</div>;
|
||||
if (!data || !animeCumulative) return null;
|
||||
const openTitle = (row: LibrarySummaryRow) => {
|
||||
if (row.animeId !== null) {
|
||||
onNavigateToAnime?.(row.animeId);
|
||||
} else if (row.videoId !== null) {
|
||||
onNavigateToMediaDetail?.(row.videoId);
|
||||
}
|
||||
};
|
||||
|
||||
const animeTitles = buildAnimeVisibilityOptions([
|
||||
animeCumulative.episodes,
|
||||
animeCumulative.cards,
|
||||
animeCumulative.words,
|
||||
animeCumulative.watchTime,
|
||||
]);
|
||||
const activeHiddenAnime = pruneHiddenAnime(hiddenAnime, animeTitles);
|
||||
const allTitles = useMemo(() => data?.librarySummary.map((row) => row.title) ?? [], [data]);
|
||||
const visibleTitleRows = useMemo(() => {
|
||||
const rows = data?.librarySummary ?? [];
|
||||
const query = titleQuery.trim().toLowerCase();
|
||||
return query ? rows.filter((row) => row.title.toLowerCase().includes(query)) : rows;
|
||||
}, [data, titleQuery]);
|
||||
|
||||
const filteredAnimeProgress = filterHiddenAnimeData(animeCumulative.episodes, activeHiddenAnime);
|
||||
const filteredCardsProgress = filterHiddenAnimeData(animeCumulative.cards, activeHiddenAnime);
|
||||
const filteredWordsProgress = filterHiddenAnimeData(animeCumulative.words, activeHiddenAnime);
|
||||
const filteredWatchTimeProgress = filterHiddenAnimeData(
|
||||
animeCumulative.watchTime,
|
||||
activeHiddenAnime,
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
const controls = (
|
||||
<div className="flex flex-wrap items-center gap-x-4 gap-y-2">
|
||||
<DateRangeSelector
|
||||
range={range}
|
||||
groupBy={groupBy}
|
||||
fillEmpty={showEmptyDays}
|
||||
onRangeChange={setRange}
|
||||
onGroupByChange={setGroupBy}
|
||||
shownDays={data?.window ?? null}
|
||||
onRangeChange={updateRange}
|
||||
onGroupByChange={updateGroupBy}
|
||||
onFillEmptyChange={updateShowEmptyDays}
|
||||
/>
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
|
||||
<SectionHeader>Activity (per {groupBy === 'month' ? 'month' : 'day'})</SectionHeader>
|
||||
<TrendChart
|
||||
title="Watch Time (min)"
|
||||
data={data.activity.watchTime}
|
||||
color="#8aadf4"
|
||||
type="bar"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Cards Mined"
|
||||
data={data.activity.cards}
|
||||
color={cardsMinedColor}
|
||||
type="bar"
|
||||
/>
|
||||
<TrendChart title="Words Seen" data={data.activity.words} color="#8bd5ca" type="bar" />
|
||||
<TrendChart title="Sessions" data={data.activity.sessions} color="#b7bdf8" type="bar" />
|
||||
<TitleFilter
|
||||
titles={allTitles}
|
||||
query={titleQuery}
|
||||
selectedTitle={selectedTitle}
|
||||
onQueryChange={setTitleQuery}
|
||||
onSelectTitle={setSelectedTitle}
|
||||
/>
|
||||
{refreshing && data ? (
|
||||
<span role="status" className="text-[11px] text-ctp-overlay2">
|
||||
Updating...
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
|
||||
<SectionHeader>Cumulative Totals</SectionHeader>
|
||||
<TrendChart
|
||||
title="Watch Time, cumulative (min)"
|
||||
data={data.progress.watchTime}
|
||||
color="#8aadf4"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Words Seen (cumulative)"
|
||||
data={data.progress.words}
|
||||
color="#8bd5ca"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="New Words Seen (cumulative)"
|
||||
data={data.progress.newWords}
|
||||
color="#c6a0f6"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Cards Mined (cumulative)"
|
||||
data={data.progress.cards}
|
||||
color={cardsMinedColor}
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Episodes Watched (cumulative)"
|
||||
data={data.progress.episodes}
|
||||
color="#91d7e3"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Sessions (cumulative)"
|
||||
data={data.progress.sessions}
|
||||
color="#b7bdf8"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Lookups (cumulative)"
|
||||
data={data.progress.lookups}
|
||||
color="#f5bde6"
|
||||
type="line"
|
||||
/>
|
||||
if (loading) {
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{controls}
|
||||
<div className="text-ctp-overlay2 p-4">Loading...</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
if (!data) {
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
{controls}
|
||||
{error ? <div className="text-ctp-red p-4">Error: {error}</div> : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
<SectionHeader>Efficiency</SectionHeader>
|
||||
<TrendChart
|
||||
title="Words / Min"
|
||||
data={data.ratios.readingSpeed}
|
||||
color="#a6da95"
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Cards / Hour"
|
||||
data={data.ratios.cardsPerHour}
|
||||
color={cardsMinedColor}
|
||||
type="line"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Lookups / 100 Words"
|
||||
data={data.ratios.lookupsPerHundred}
|
||||
color="#f5a97f"
|
||||
type="line"
|
||||
/>
|
||||
const openBucketSessions = (_label: string, index: number) => {
|
||||
const bucket = data.buckets[index];
|
||||
if (!bucket) return;
|
||||
onOpenSessionsForDays?.({
|
||||
startDay: bucket.startDay,
|
||||
endDay: bucket.endDay,
|
||||
label: bucketSessionsLabel(bucket, groupBy),
|
||||
});
|
||||
};
|
||||
const barClickProps = onOpenSessionsForDays ? { onBarClick: openBucketSessions } : {};
|
||||
|
||||
const windowDays = data.window.endDay - data.window.startDay + 1;
|
||||
const runningTotalOptions = RUNNING_TOTAL_METRICS.filter(
|
||||
(metric) => metric !== 'newWords' || data.learning !== null,
|
||||
);
|
||||
const activeRunningTotal = runningTotalOptions.includes(runningTotalMetric)
|
||||
? runningTotalMetric
|
||||
: 'watchTime';
|
||||
const isWatchTimeTotal = activeRunningTotal === 'watchTime';
|
||||
|
||||
return (
|
||||
<div className="space-y-4" aria-busy={refreshing}>
|
||||
{controls}
|
||||
{error ? <div className="text-sm text-ctp-red">Error: {error}</div> : null}
|
||||
<SummaryStrip summary={data.summary} days={windowDays} />
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-6 gap-4">
|
||||
<SectionHeader
|
||||
hint={onOpenSessionsForDays ? 'Click a bar to open its sessions' : undefined}
|
||||
>
|
||||
Activity per {groupBy}
|
||||
</SectionHeader>
|
||||
<div className="xl:col-span-3">
|
||||
<TrendChart
|
||||
title="Watch Time (hours)"
|
||||
data={toHours(data.activity.watchTime)}
|
||||
color={WATCH_TIME_COLOR}
|
||||
type="bar"
|
||||
tickFormatter={formatHourTick}
|
||||
valueFormatter={formatHourValue}
|
||||
{...barClickProps}
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-3">
|
||||
<TrendChart
|
||||
title="Cards Mined"
|
||||
data={data.activity.cards}
|
||||
color={CARDS_COLOR}
|
||||
type="bar"
|
||||
{...barClickProps}
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-3">
|
||||
<TrendChart
|
||||
title="Words Seen"
|
||||
data={data.activity.words}
|
||||
color={WORDS_COLOR}
|
||||
type="bar"
|
||||
{...barClickProps}
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-3">
|
||||
<TrendChart
|
||||
title="Episodes Finished"
|
||||
data={data.activity.episodes}
|
||||
color={EPISODES_COLOR}
|
||||
type="bar"
|
||||
{...barClickProps}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{data.learning ? (
|
||||
<>
|
||||
<SectionHeader hint="Dictionary forms, names excluded">Learning</SectionHeader>
|
||||
<div className="xl:col-span-3">
|
||||
<TrendChart
|
||||
title={
|
||||
groupBy === 'day'
|
||||
? 'New Words per 1,000 Seen (rolling 7 days)'
|
||||
: 'New Words per 1,000 Seen'
|
||||
}
|
||||
data={data.learning.newWordsPerThousand}
|
||||
color={NEW_WORDS_COLOR}
|
||||
type="line"
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-3">
|
||||
<FrequencyBandChart data={data.learning.frequencyBands} />
|
||||
</div>
|
||||
</>
|
||||
) : (
|
||||
<div className="col-span-full text-xs text-ctp-overlay2">
|
||||
New words are tracked across your whole library, so the Learning charts are hidden while
|
||||
filtering to one title.
|
||||
</div>
|
||||
)}
|
||||
|
||||
<SectionHeader
|
||||
hint={
|
||||
groupBy === 'day'
|
||||
? 'Rolling 7 days, gaps mean too little data'
|
||||
: 'Gaps mean too little data'
|
||||
}
|
||||
>
|
||||
Efficiency
|
||||
</SectionHeader>
|
||||
<div className="xl:col-span-2">
|
||||
<TrendChart
|
||||
title="Words / Min"
|
||||
data={data.ratios.readingSpeed}
|
||||
color={READING_SPEED_COLOR}
|
||||
type="line"
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-2">
|
||||
<TrendChart
|
||||
title="Cards / Hour"
|
||||
data={data.ratios.cardsPerHour}
|
||||
color={CARDS_COLOR}
|
||||
type="line"
|
||||
/>
|
||||
</div>
|
||||
<div className="xl:col-span-2">
|
||||
<TrendChart
|
||||
title="Lookups / 100 Words"
|
||||
data={data.ratios.lookupsPerHundred}
|
||||
color={LOOKUPS_COLOR}
|
||||
type="line"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<SectionHeader>Running Total</SectionHeader>
|
||||
<div className="col-span-full">
|
||||
<div className="mb-2 flex flex-wrap gap-1" role="group" aria-label="Running total metric">
|
||||
{runningTotalOptions.map((metric) => (
|
||||
<button
|
||||
key={metric}
|
||||
type="button"
|
||||
aria-pressed={metric === activeRunningTotal}
|
||||
className={`rounded-md border px-2 py-1 text-[11px] transition ${
|
||||
metric === activeRunningTotal
|
||||
? 'border-ctp-surface2 bg-ctp-surface2 text-ctp-text'
|
||||
: 'border-ctp-surface1 text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
onClick={() => updateRunningTotalMetric(metric)}
|
||||
>
|
||||
{RUNNING_TOTAL_LABELS[metric]}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<TrendChart
|
||||
title={`${RUNNING_TOTAL_LABELS[activeRunningTotal]}, total since the start of this range`}
|
||||
data={startFromZero(
|
||||
isWatchTimeTotal
|
||||
? toHours(data.progress.watchTime)
|
||||
: data.progress[activeRunningTotal],
|
||||
)}
|
||||
color={RUNNING_TOTAL_COLORS[activeRunningTotal]}
|
||||
type="line"
|
||||
{...(isWatchTimeTotal
|
||||
? { tickFormatter: formatHourTick, valueFormatter: formatHourValue }
|
||||
: {})}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<SectionHeader>Patterns</SectionHeader>
|
||||
<TrendChart
|
||||
title="Watch Time by Day of Week (min)"
|
||||
data={data.patterns.watchTimeByDayOfWeek}
|
||||
color="#8aadf4"
|
||||
type="bar"
|
||||
/>
|
||||
<TrendChart
|
||||
title="Watch Time by Hour (min)"
|
||||
data={data.patterns.watchTimeByHour}
|
||||
color="#c6a0f6"
|
||||
type="bar"
|
||||
/>
|
||||
<DayHourHeatmap grid={data.watchTimeByDayHour} />
|
||||
|
||||
<SectionHeader>Library — Cumulative</SectionHeader>
|
||||
<AnimeVisibilityFilter
|
||||
animeTitles={animeTitles}
|
||||
hiddenAnime={activeHiddenAnime}
|
||||
maxTitles={maxTitles}
|
||||
maxTitlesMode={maxTitlesMode}
|
||||
onShowAll={() => updateHiddenAnime(new Set())}
|
||||
onHideAll={() => updateHiddenAnime(new Set(animeTitles))}
|
||||
onToggleAnime={(title) => {
|
||||
const next = new Set(hiddenAnime);
|
||||
if (next.has(title)) {
|
||||
next.delete(title);
|
||||
} else {
|
||||
next.add(title);
|
||||
}
|
||||
updateHiddenAnime(next);
|
||||
}}
|
||||
onMaxTitlesChange={updateMaxTitles}
|
||||
onMaxTitlesModeChange={updateMaxTitlesMode}
|
||||
<SectionHeader
|
||||
hint={`${formatNumber(data.librarySummary.length)} titles. Click a title to filter the page, ↗ to open it`}
|
||||
>
|
||||
Titles
|
||||
</SectionHeader>
|
||||
<TitleTimeline
|
||||
rows={visibleTitleRows}
|
||||
points={data.titleTimeline}
|
||||
window={data.window}
|
||||
selectedTitle={selectedTitle}
|
||||
onSelectTitle={toggleSelectedTitle}
|
||||
onOpenTitle={openTitle}
|
||||
/>
|
||||
<StackedTrendChart
|
||||
title="Watch Time Progress (min)"
|
||||
data={filteredWatchTimeProgress}
|
||||
maxSeries={maxTitles}
|
||||
maxSeriesMode={maxTitlesMode}
|
||||
<LibrarySummarySection
|
||||
rows={visibleTitleRows}
|
||||
selectedTitle={selectedTitle}
|
||||
onSelectTitle={toggleSelectedTitle}
|
||||
onOpenTitle={openTitle}
|
||||
/>
|
||||
<StackedTrendChart
|
||||
title="Episodes Progress"
|
||||
data={filteredAnimeProgress}
|
||||
maxSeries={maxTitles}
|
||||
maxSeriesMode={maxTitlesMode}
|
||||
/>
|
||||
<StackedTrendChart
|
||||
title="Cards Mined Progress"
|
||||
data={filteredCardsProgress}
|
||||
colorPalette={cardsMinedStackedColors}
|
||||
maxSeries={maxTitles}
|
||||
maxSeriesMode={maxTitlesMode}
|
||||
/>
|
||||
<StackedTrendChart
|
||||
title="Words Seen Progress"
|
||||
data={filteredWordsProgress}
|
||||
maxSeries={maxTitles}
|
||||
maxSeriesMode={maxTitlesMode}
|
||||
/>
|
||||
|
||||
<SectionHeader>Library — Summary</SectionHeader>
|
||||
<LibrarySummarySection rows={data.librarySummary} hiddenTitles={activeHiddenAnime} />
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import { buildCumulativePerAnime } from './anime-cumulative';
|
||||
|
||||
test('buildCumulativePerAnime carries each title forward to the last active day', () => {
|
||||
const result = buildCumulativePerAnime([
|
||||
{ epochDay: 103, animeTitle: 'B', value: 4 },
|
||||
{ epochDay: 100, animeTitle: 'A', value: 10 },
|
||||
{ epochDay: 102, animeTitle: 'A', value: 5 },
|
||||
{ epochDay: 102, animeTitle: 'A', value: 1 },
|
||||
]);
|
||||
|
||||
assert.deepEqual(
|
||||
result
|
||||
.filter((point) => point.animeTitle === 'A')
|
||||
.map((point) => [point.epochDay, point.value]),
|
||||
[
|
||||
[100, 10],
|
||||
[101, 10],
|
||||
[102, 16],
|
||||
[103, 16],
|
||||
],
|
||||
);
|
||||
// A title starts at its own first active day, not the earliest day overall.
|
||||
assert.deepEqual(
|
||||
result
|
||||
.filter((point) => point.animeTitle === 'B')
|
||||
.map((point) => [point.epochDay, point.value]),
|
||||
[[103, 4]],
|
||||
);
|
||||
});
|
||||
|
||||
test('buildCumulativePerAnime returns nothing for no points', () => {
|
||||
assert.deepEqual(buildCumulativePerAnime([]), []);
|
||||
});
|
||||
@@ -1,41 +0,0 @@
|
||||
import type { TrendsDashboardData } from '../../types/stats';
|
||||
import type { PerAnimeDataPoint } from './StackedTrendChart';
|
||||
|
||||
type AnimeSeries = TrendsDashboardData['animeDaily'];
|
||||
|
||||
/**
|
||||
* Turns sparse per-title daily values into running totals: one point per
|
||||
* title for every calendar day from its first active day to the last active
|
||||
* day of any title, so the stacked area charts carry each total forward.
|
||||
*/
|
||||
export function buildCumulativePerAnime(points: PerAnimeDataPoint[]): PerAnimeDataPoint[] {
|
||||
const byAnime = new Map<string, Map<number, number>>();
|
||||
let maxDay = Number.NEGATIVE_INFINITY;
|
||||
|
||||
for (const point of points) {
|
||||
const dayMap = byAnime.get(point.animeTitle) ?? new Map<number, number>();
|
||||
dayMap.set(point.epochDay, (dayMap.get(point.epochDay) ?? 0) + point.value);
|
||||
byAnime.set(point.animeTitle, dayMap);
|
||||
maxDay = Math.max(maxDay, point.epochDay);
|
||||
}
|
||||
|
||||
const result: PerAnimeDataPoint[] = [];
|
||||
for (const [animeTitle, dayMap] of byAnime) {
|
||||
const firstDay = Math.min(...dayMap.keys());
|
||||
let cumulative = 0;
|
||||
for (let epochDay = firstDay; epochDay <= maxDay; epochDay += 1) {
|
||||
cumulative += dayMap.get(epochDay) ?? 0;
|
||||
result.push({ epochDay, animeTitle, value: cumulative });
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export function buildAnimeCumulative(daily: AnimeSeries): AnimeSeries {
|
||||
return {
|
||||
watchTime: buildCumulativePerAnime(daily.watchTime),
|
||||
episodes: buildCumulativePerAnime(daily.episodes),
|
||||
cards: buildCumulativePerAnime(daily.cards),
|
||||
words: buildCumulativePerAnime(daily.words),
|
||||
};
|
||||
}
|
||||
@@ -1,161 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import type { PerAnimeDataPoint } from './StackedTrendChart';
|
||||
import {
|
||||
buildAnimeVisibilityOptions,
|
||||
filterHiddenAnimeData,
|
||||
loadHiddenTitles,
|
||||
loadMaxTitles,
|
||||
loadMaxTitlesMode,
|
||||
loadShowEmptyDays,
|
||||
pruneHiddenAnime,
|
||||
saveHiddenTitles,
|
||||
saveMaxTitles,
|
||||
saveMaxTitlesMode,
|
||||
saveShowEmptyDays,
|
||||
} from './anime-visibility';
|
||||
|
||||
function installLocalStorage(initial: Record<string, string> = {}) {
|
||||
const previous = Object.getOwnPropertyDescriptor(globalThis, 'localStorage');
|
||||
const values = new Map(Object.entries(initial));
|
||||
Object.defineProperty(globalThis, 'localStorage', {
|
||||
configurable: true,
|
||||
value: {
|
||||
getItem: (key: string) => values.get(key) ?? null,
|
||||
setItem: (key: string, value: string) => values.set(key, value),
|
||||
removeItem: (key: string) => values.delete(key),
|
||||
},
|
||||
});
|
||||
return {
|
||||
values,
|
||||
restore: () => {
|
||||
if (previous) {
|
||||
Object.defineProperty(globalThis, 'localStorage', previous);
|
||||
} else {
|
||||
delete (globalThis as { localStorage?: unknown }).localStorage;
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const SAMPLE_POINTS: PerAnimeDataPoint[] = [
|
||||
{ epochDay: 1, animeTitle: 'KonoSuba', value: 5 },
|
||||
{ epochDay: 2, animeTitle: 'KonoSuba', value: 10 },
|
||||
{ epochDay: 1, animeTitle: 'Little Witch Academia', value: 6 },
|
||||
{ epochDay: 1, animeTitle: 'Trapped in a Dating Sim', value: 20 },
|
||||
];
|
||||
|
||||
test('buildAnimeVisibilityOptions sorts anime by combined contribution', () => {
|
||||
const titles = buildAnimeVisibilityOptions([
|
||||
SAMPLE_POINTS,
|
||||
[
|
||||
{ epochDay: 1, animeTitle: 'Little Witch Academia', value: 8 },
|
||||
{ epochDay: 1, animeTitle: 'KonoSuba', value: 1 },
|
||||
],
|
||||
]);
|
||||
|
||||
assert.deepEqual(titles, ['Trapped in a Dating Sim', 'KonoSuba', 'Little Witch Academia']);
|
||||
});
|
||||
|
||||
test('filterHiddenAnimeData removes globally hidden anime from chart data', () => {
|
||||
const filtered = filterHiddenAnimeData(SAMPLE_POINTS, new Set(['KonoSuba']));
|
||||
|
||||
assert.equal(
|
||||
filtered.some((point) => point.animeTitle === 'KonoSuba'),
|
||||
false,
|
||||
);
|
||||
assert.equal(filtered.length, 2);
|
||||
});
|
||||
|
||||
test('pruneHiddenAnime drops titles that are no longer available', () => {
|
||||
const hidden = pruneHiddenAnime(new Set(['KonoSuba', 'Ghost in the Shell']), [
|
||||
'KonoSuba',
|
||||
'Little Witch Academia',
|
||||
]);
|
||||
|
||||
assert.deepEqual([...hidden], ['KonoSuba']);
|
||||
});
|
||||
|
||||
test('hidden titles round-trip through localStorage', () => {
|
||||
const { restore } = installLocalStorage();
|
||||
try {
|
||||
saveHiddenTitles(new Set(['KonoSuba', 'One Piece']));
|
||||
assert.deepEqual([...loadHiddenTitles()], ['KonoSuba', 'One Piece']);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('loadHiddenTitles tolerates missing or malformed stored values', () => {
|
||||
const { restore } = installLocalStorage({
|
||||
'subminer-stats-trends-hidden-titles': '{"not":"an array"}',
|
||||
});
|
||||
try {
|
||||
assert.deepEqual([...loadHiddenTitles()], []);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('max titles preference defaults to 7 and round-trips including explicit All', () => {
|
||||
const { values, restore } = installLocalStorage();
|
||||
try {
|
||||
// First run (nothing stored) defaults to the 7-title cap, not "All".
|
||||
assert.equal(loadMaxTitles(), 7);
|
||||
saveMaxTitles(5);
|
||||
assert.equal(loadMaxTitles(), 5);
|
||||
// "All" persists explicitly instead of collapsing back to the default.
|
||||
saveMaxTitles(null);
|
||||
assert.equal(loadMaxTitles(), null);
|
||||
assert.equal(values.get('subminer-stats-trends-max-titles'), 'all');
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('loadMaxTitles falls back to the default for unsupported stored values', () => {
|
||||
for (const storedValue of ['8', '0', 'banana']) {
|
||||
const { restore } = installLocalStorage({ 'subminer-stats-trends-max-titles': storedValue });
|
||||
try {
|
||||
assert.equal(loadMaxTitles(), 7);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('max titles mode defaults to recent and round-trips', () => {
|
||||
const { restore } = installLocalStorage();
|
||||
try {
|
||||
assert.equal(loadMaxTitlesMode(), 'recent');
|
||||
saveMaxTitlesMode('total');
|
||||
assert.equal(loadMaxTitlesMode(), 'total');
|
||||
saveMaxTitlesMode('recent');
|
||||
assert.equal(loadMaxTitlesMode(), 'recent');
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('loadMaxTitlesMode falls back to recent for unknown stored values', () => {
|
||||
const { restore } = installLocalStorage({ 'subminer-stats-trends-max-titles-mode': 'sideways' });
|
||||
try {
|
||||
assert.equal(loadMaxTitlesMode(), 'recent');
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('show empty days preference defaults to true and round-trips', () => {
|
||||
const { restore } = installLocalStorage();
|
||||
try {
|
||||
assert.equal(loadShowEmptyDays(), true);
|
||||
saveShowEmptyDays(false);
|
||||
assert.equal(loadShowEmptyDays(), false);
|
||||
saveShowEmptyDays(true);
|
||||
assert.equal(loadShowEmptyDays(), true);
|
||||
} finally {
|
||||
restore();
|
||||
}
|
||||
});
|
||||
@@ -1,136 +0,0 @@
|
||||
import type { PerAnimeDataPoint } from './StackedTrendChart';
|
||||
|
||||
const HIDDEN_TITLES_KEY = 'subminer-stats-trends-hidden-titles';
|
||||
const MAX_TITLES_KEY = 'subminer-stats-trends-max-titles';
|
||||
const MAX_TITLES_MODE_KEY = 'subminer-stats-trends-max-titles-mode';
|
||||
const SHOW_EMPTY_DAYS_KEY = 'subminer-stats-trends-show-empty-days';
|
||||
|
||||
export const MAX_TITLES_OPTIONS = [3, 5, 7, 10] as const;
|
||||
|
||||
// How the per-chart limit picks which titles survive: 'recent' keeps the most
|
||||
// recently active titles, 'total' keeps the highest cumulative totals. 'recent'
|
||||
// is listed first so it heads the dropdown.
|
||||
export type MaxTitlesMode = 'total' | 'recent';
|
||||
export const MAX_TITLES_MODES: readonly MaxTitlesMode[] = ['recent', 'total'];
|
||||
|
||||
// First-run defaults: show the 7 most recently active titles per chart.
|
||||
const DEFAULT_MAX_TITLES = 7;
|
||||
const DEFAULT_MAX_TITLES_MODE: MaxTitlesMode = 'recent';
|
||||
const ALL_TITLES_STORED_VALUE = 'all';
|
||||
|
||||
function getStorage(): Storage | null {
|
||||
try {
|
||||
return globalThis.localStorage ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function loadHiddenTitles(): Set<string> {
|
||||
try {
|
||||
const raw = getStorage()?.getItem(HIDDEN_TITLES_KEY);
|
||||
const parsed: unknown = raw ? JSON.parse(raw) : [];
|
||||
if (!Array.isArray(parsed)) return new Set();
|
||||
return new Set(parsed.filter((value): value is string => typeof value === 'string'));
|
||||
} catch {
|
||||
return new Set();
|
||||
}
|
||||
}
|
||||
|
||||
export function saveHiddenTitles(hidden: ReadonlySet<string>): void {
|
||||
try {
|
||||
getStorage()?.setItem(HIDDEN_TITLES_KEY, JSON.stringify([...hidden]));
|
||||
} catch {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
|
||||
// Returns the persisted per-chart limit: a specific count, or null for "All".
|
||||
// Absent/garbage storage falls back to the default count (never null), so a
|
||||
// first run shows the default cap rather than every title. "All" is stored
|
||||
// explicitly so it round-trips instead of collapsing back to the default.
|
||||
export function loadMaxTitles(): number | null {
|
||||
try {
|
||||
const raw = getStorage()?.getItem(MAX_TITLES_KEY);
|
||||
if (raw === ALL_TITLES_STORED_VALUE) return null;
|
||||
const value = Number(raw);
|
||||
return (MAX_TITLES_OPTIONS as readonly number[]).includes(value) ? value : DEFAULT_MAX_TITLES;
|
||||
} catch {
|
||||
return DEFAULT_MAX_TITLES;
|
||||
}
|
||||
}
|
||||
|
||||
export function saveMaxTitles(value: number | null): void {
|
||||
try {
|
||||
getStorage()?.setItem(MAX_TITLES_KEY, value === null ? ALL_TITLES_STORED_VALUE : String(value));
|
||||
} catch {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
|
||||
export function loadMaxTitlesMode(): MaxTitlesMode {
|
||||
try {
|
||||
const raw = getStorage()?.getItem(MAX_TITLES_MODE_KEY);
|
||||
if (raw === 'recent' || raw === 'total') return raw;
|
||||
return DEFAULT_MAX_TITLES_MODE;
|
||||
} catch {
|
||||
return DEFAULT_MAX_TITLES_MODE;
|
||||
}
|
||||
}
|
||||
|
||||
export function saveMaxTitlesMode(mode: MaxTitlesMode): void {
|
||||
try {
|
||||
getStorage()?.setItem(MAX_TITLES_MODE_KEY, mode);
|
||||
} catch {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
|
||||
// Whether trend charts zero-fill empty calendar buckets. Defaults to true so a
|
||||
// first run shows the calendar-accurate view.
|
||||
export function loadShowEmptyDays(): boolean {
|
||||
try {
|
||||
return getStorage()?.getItem(SHOW_EMPTY_DAYS_KEY) !== 'false';
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
export function saveShowEmptyDays(show: boolean): void {
|
||||
try {
|
||||
getStorage()?.setItem(SHOW_EMPTY_DAYS_KEY, show ? 'true' : 'false');
|
||||
} catch {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
|
||||
export function buildAnimeVisibilityOptions(datasets: PerAnimeDataPoint[][]): string[] {
|
||||
const totals = new Map<string, number>();
|
||||
for (const dataset of datasets) {
|
||||
for (const point of dataset) {
|
||||
totals.set(point.animeTitle, (totals.get(point.animeTitle) ?? 0) + point.value);
|
||||
}
|
||||
}
|
||||
|
||||
return [...totals.entries()]
|
||||
.sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
|
||||
.map(([title]) => title);
|
||||
}
|
||||
|
||||
export function filterHiddenAnimeData(
|
||||
data: PerAnimeDataPoint[],
|
||||
hiddenAnime: ReadonlySet<string>,
|
||||
): PerAnimeDataPoint[] {
|
||||
if (hiddenAnime.size === 0) {
|
||||
return data;
|
||||
}
|
||||
return data.filter((point) => !hiddenAnime.has(point.animeTitle));
|
||||
}
|
||||
|
||||
export function pruneHiddenAnime(
|
||||
hiddenAnime: ReadonlySet<string>,
|
||||
availableAnime: readonly string[],
|
||||
): Set<string> {
|
||||
const availableSet = new Set(availableAnime);
|
||||
return new Set([...hiddenAnime].filter((title) => availableSet.has(title)));
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { buildTitleTimeline } from './title-timeline';
|
||||
|
||||
const point = (epochDay: number, animeTitle: string, value: number) => ({
|
||||
epochDay,
|
||||
animeTitle,
|
||||
value,
|
||||
});
|
||||
|
||||
test('buildTitleTimeline positions one column per day for short ranges', () => {
|
||||
const model = buildTitleTimeline([point(20_002, 'A', 30), point(20_000, 'A', 10)], {
|
||||
startDay: 20_000,
|
||||
endDay: 20_009,
|
||||
});
|
||||
|
||||
assert.equal(model.columnDays, 1);
|
||||
assert.deepEqual(
|
||||
model.cellsByTitle.get('A')?.map((cell) => [cell.startDay, cell.leftPercent, cell.minutes]),
|
||||
[
|
||||
[20_000, 0, 10],
|
||||
[20_002, 20, 30],
|
||||
],
|
||||
);
|
||||
assert.equal(model.maxCellMinutes, 30);
|
||||
});
|
||||
|
||||
test('buildTitleTimeline merges days into weekly columns for long ranges', () => {
|
||||
const model = buildTitleTimeline(
|
||||
[point(20_000, 'A', 10), point(20_006, 'A', 5), point(20_007, 'A', 1)],
|
||||
{ startDay: 20_000, endDay: 20_199 },
|
||||
);
|
||||
|
||||
assert.equal(model.columnDays, 7);
|
||||
assert.deepEqual(
|
||||
model.cellsByTitle.get('A')?.map((cell) => [cell.startDay, cell.minutes]),
|
||||
[
|
||||
[20_000, 15],
|
||||
[20_007, 1],
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test('buildTitleTimeline ignores points outside the range', () => {
|
||||
const model = buildTitleTimeline([point(19_999, 'A', 10), point(20_010, 'B', 10)], {
|
||||
startDay: 20_000,
|
||||
endDay: 20_009,
|
||||
});
|
||||
assert.equal(model.cellsByTitle.size, 0);
|
||||
});
|
||||
|
||||
test('buildTitleTimeline labels the start of the range unless a month tick is already there', () => {
|
||||
// 2024-10-05 to 2024-11-03: only November starts inside the range.
|
||||
const midMonth = buildTitleTimeline([], { startDay: 20_001, endDay: 20_030 });
|
||||
assert.equal(midMonth.ticks.length, 2);
|
||||
assert.equal(midMonth.ticks[0]?.leftPercent, 0);
|
||||
|
||||
// 2024-10-01 onward: October's own tick already marks the start.
|
||||
const monthStart = buildTitleTimeline([], { startDay: 19_997, endDay: 20_030 });
|
||||
assert.equal(monthStart.ticks.length, 2);
|
||||
assert.equal(monthStart.ticks[0]?.leftPercent, 0);
|
||||
assert.notEqual(monthStart.ticks[1]?.leftPercent, 0);
|
||||
});
|
||||
@@ -0,0 +1,101 @@
|
||||
import type { TrendPerAnimePoint } from '../../types/stats';
|
||||
import { epochDayToDate } from '../../lib/formatters';
|
||||
|
||||
// Past this span a daily column is too thin to see, so columns become weeks.
|
||||
const MAX_DAILY_COLUMN_SPAN = 120;
|
||||
const WEEK_DAYS = 7;
|
||||
// Past this span a tick per month no longer fits, so ticks mark quarters.
|
||||
const MAX_MONTHLY_TICK_SPAN = 400;
|
||||
// The start-date tick is dropped when a month tick would sit on top of it.
|
||||
const MIN_START_TICK_CLEARANCE_PERCENT = 8;
|
||||
|
||||
export interface TimelineCell {
|
||||
startDay: number;
|
||||
/** Position and size along the row, as percentages of its width. */
|
||||
leftPercent: number;
|
||||
widthPercent: number;
|
||||
minutes: number;
|
||||
}
|
||||
|
||||
export interface TimelineTick {
|
||||
label: string;
|
||||
leftPercent: number;
|
||||
}
|
||||
|
||||
export interface TitleTimelineModel {
|
||||
columnDays: number;
|
||||
ticks: TimelineTick[];
|
||||
cellsByTitle: Map<string, TimelineCell[]>;
|
||||
maxCellMinutes: number;
|
||||
}
|
||||
|
||||
function buildTicks(startDay: number, endDay: number): TimelineTick[] {
|
||||
const span = endDay - startDay + 1;
|
||||
const monthStep = span > MAX_MONTHLY_TICK_SPAN ? 3 : 1;
|
||||
const monthTicks: TimelineTick[] = [];
|
||||
for (let day = startDay; day <= endDay; day += 1) {
|
||||
const date = epochDayToDate(day);
|
||||
if (date.getDate() !== 1 || date.getMonth() % monthStep !== 0) continue;
|
||||
monthTicks.push({
|
||||
label: date.toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
...(date.getMonth() === 0 ? { year: 'numeric' } : {}),
|
||||
}),
|
||||
leftPercent: ((day - startDay) / span) * 100,
|
||||
});
|
||||
}
|
||||
const firstTick = monthTicks[0];
|
||||
if (firstTick && firstTick.leftPercent < MIN_START_TICK_CLEARANCE_PERCENT) return monthTicks;
|
||||
const startTick = {
|
||||
label: epochDayToDate(startDay).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
}),
|
||||
leftPercent: 0,
|
||||
};
|
||||
return [startTick, ...monthTicks];
|
||||
}
|
||||
|
||||
/** Lays per-title daily minutes out as positioned cells along the dashboard's date range. */
|
||||
export function buildTitleTimeline(
|
||||
points: readonly TrendPerAnimePoint[],
|
||||
window: { startDay: number; endDay: number },
|
||||
): TitleTimelineModel {
|
||||
const span = window.endDay - window.startDay + 1;
|
||||
const columnDays = span > MAX_DAILY_COLUMN_SPAN ? WEEK_DAYS : 1;
|
||||
const columnCount = Math.ceil(span / columnDays);
|
||||
const widthPercent = 100 / columnCount;
|
||||
|
||||
const minutesByTitle = new Map<string, Map<number, number>>();
|
||||
for (const point of points) {
|
||||
const column = Math.floor((point.epochDay - window.startDay) / columnDays);
|
||||
if (column < 0 || column >= columnCount) continue;
|
||||
const columns = minutesByTitle.get(point.animeTitle) ?? new Map<number, number>();
|
||||
columns.set(column, (columns.get(column) ?? 0) + point.value);
|
||||
minutesByTitle.set(point.animeTitle, columns);
|
||||
}
|
||||
|
||||
let maxCellMinutes = 0;
|
||||
const cellsByTitle = new Map<string, TimelineCell[]>();
|
||||
for (const [title, columns] of minutesByTitle) {
|
||||
const cells = [...columns.entries()]
|
||||
.sort(([left], [right]) => left - right)
|
||||
.map(([column, minutes]) => {
|
||||
maxCellMinutes = Math.max(maxCellMinutes, minutes);
|
||||
return {
|
||||
startDay: window.startDay + column * columnDays,
|
||||
leftPercent: column * widthPercent,
|
||||
widthPercent,
|
||||
minutes,
|
||||
};
|
||||
});
|
||||
cellsByTitle.set(title, cells);
|
||||
}
|
||||
|
||||
return {
|
||||
columnDays,
|
||||
ticks: buildTicks(window.startDay, window.endDay),
|
||||
cellsByTitle,
|
||||
maxCellMinutes,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {
|
||||
loadGroupBy,
|
||||
loadRange,
|
||||
loadRunningTotalMetric,
|
||||
saveGroupBy,
|
||||
saveRange,
|
||||
saveRunningTotalMetric,
|
||||
} from './trends-preferences';
|
||||
|
||||
function withStorage(run: () => void, initial: Record<string, string> = {}) {
|
||||
const values = new Map(Object.entries(initial));
|
||||
const previous = Object.getOwnPropertyDescriptor(globalThis, 'localStorage');
|
||||
Object.defineProperty(globalThis, 'localStorage', {
|
||||
configurable: true,
|
||||
value: {
|
||||
getItem: (key: string) => values.get(key) ?? null,
|
||||
setItem: (key: string, value: string) => values.set(key, value),
|
||||
},
|
||||
});
|
||||
try {
|
||||
run();
|
||||
} finally {
|
||||
if (previous) Object.defineProperty(globalThis, 'localStorage', previous);
|
||||
else Reflect.deleteProperty(globalThis, 'localStorage');
|
||||
}
|
||||
}
|
||||
|
||||
test('range and grouping survive a reload, including custom ranges', () => {
|
||||
withStorage(() => {
|
||||
saveRange('90d');
|
||||
assert.equal(loadRange(), '90d');
|
||||
saveRange({ startDay: 20_000, endDay: 20_030 });
|
||||
assert.deepEqual(loadRange(), { startDay: 20_000, endDay: 20_030 });
|
||||
saveGroupBy('week');
|
||||
assert.equal(loadGroupBy(), 'week');
|
||||
saveRunningTotalMetric('cards');
|
||||
assert.equal(loadRunningTotalMetric(), 'cards');
|
||||
});
|
||||
});
|
||||
|
||||
test('unreadable stored values fall back to defaults', () => {
|
||||
withStorage(
|
||||
() => {
|
||||
assert.equal(loadRange(), '30d');
|
||||
assert.equal(loadGroupBy(), 'day');
|
||||
assert.equal(loadRunningTotalMetric(), 'watchTime');
|
||||
},
|
||||
{
|
||||
'subminer-stats-trends-range': '9:1',
|
||||
'subminer-stats-trends-group-by': 'year',
|
||||
'subminer-stats-trends-running-total-metric': 'nope',
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,79 @@
|
||||
import type { GroupBy, PresetTimeRange, TimeRange } from '../../hooks/useTrends';
|
||||
import {
|
||||
readBooleanPreference,
|
||||
readStringPreference,
|
||||
writeBooleanPreference,
|
||||
writeStringPreference,
|
||||
} from '../../lib/preference-storage';
|
||||
|
||||
const RANGE_KEY = 'subminer-stats-trends-range';
|
||||
const GROUP_BY_KEY = 'subminer-stats-trends-group-by';
|
||||
const SHOW_EMPTY_DAYS_KEY = 'subminer-stats-trends-show-empty-days';
|
||||
const RUNNING_TOTAL_METRIC_KEY = 'subminer-stats-trends-running-total-metric';
|
||||
|
||||
const PRESET_RANGES: readonly string[] = [
|
||||
'7d',
|
||||
'30d',
|
||||
'90d',
|
||||
'365d',
|
||||
'all',
|
||||
] satisfies readonly PresetTimeRange[];
|
||||
|
||||
export const RUNNING_TOTAL_METRICS = [
|
||||
'watchTime',
|
||||
'words',
|
||||
'newWords',
|
||||
'cards',
|
||||
'episodes',
|
||||
'lookups',
|
||||
] as const;
|
||||
export type RunningTotalMetric = (typeof RUNNING_TOTAL_METRICS)[number];
|
||||
|
||||
function isPresetRange(value: string): value is PresetTimeRange {
|
||||
return PRESET_RANGES.includes(value);
|
||||
}
|
||||
|
||||
/** Custom ranges are stored as `startDay:endDay`; anything unreadable falls back to 30d. */
|
||||
export function loadRange(): TimeRange {
|
||||
const raw = readStringPreference(RANGE_KEY);
|
||||
if (raw === null) return '30d';
|
||||
if (isPresetRange(raw)) return raw;
|
||||
const match = /^(\d+):(\d+)$/.exec(raw);
|
||||
if (!match) return '30d';
|
||||
const startDay = Number(match[1]);
|
||||
const endDay = Number(match[2]);
|
||||
return startDay <= endDay ? { startDay, endDay } : '30d';
|
||||
}
|
||||
|
||||
export function saveRange(range: TimeRange): void {
|
||||
writeStringPreference(
|
||||
RANGE_KEY,
|
||||
typeof range === 'string' ? range : `${range.startDay}:${range.endDay}`,
|
||||
);
|
||||
}
|
||||
|
||||
export function loadGroupBy(): GroupBy {
|
||||
const raw = readStringPreference(GROUP_BY_KEY);
|
||||
return raw === 'week' || raw === 'month' ? raw : 'day';
|
||||
}
|
||||
|
||||
export function saveGroupBy(groupBy: GroupBy): void {
|
||||
writeStringPreference(GROUP_BY_KEY, groupBy);
|
||||
}
|
||||
|
||||
export function loadShowEmptyDays(): boolean {
|
||||
return readBooleanPreference(SHOW_EMPTY_DAYS_KEY, true);
|
||||
}
|
||||
|
||||
export function saveShowEmptyDays(show: boolean): void {
|
||||
writeBooleanPreference(SHOW_EMPTY_DAYS_KEY, show);
|
||||
}
|
||||
|
||||
export function loadRunningTotalMetric(): RunningTotalMetric {
|
||||
const raw = readStringPreference(RUNNING_TOTAL_METRIC_KEY);
|
||||
return RUNNING_TOTAL_METRICS.find((metric) => metric === raw) ?? 'watchTime';
|
||||
}
|
||||
|
||||
export function saveRunningTotalMetric(metric: RunningTotalMetric): void {
|
||||
writeStringPreference(RUNNING_TOTAL_METRIC_KEY, metric);
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { buildCrossAnimeWordRows, CrossAnimeWordsTable } from './CrossAnimeWordsTable';
|
||||
import type { VocabularyEntry } from '../../types/stats';
|
||||
import { withLocalStorage } from '../../test-utils/dom';
|
||||
|
||||
function makeEntry(over: Partial<VocabularyEntry>): VocabularyEntry {
|
||||
return {
|
||||
wordId: 1,
|
||||
headword: '日本語',
|
||||
word: '日本語',
|
||||
reading: 'にほんご',
|
||||
frequency: 5,
|
||||
frequencyRank: 100,
|
||||
animeCount: 2,
|
||||
partOfSpeech: null,
|
||||
firstSeen: 0,
|
||||
lastSeen: 0,
|
||||
...over,
|
||||
} as VocabularyEntry;
|
||||
}
|
||||
|
||||
test('cross-title rows can hide kana-only headwords', () => {
|
||||
const rows = buildCrossAnimeWordRows(
|
||||
[
|
||||
makeEntry({ wordId: 1, headword: 'さらに', word: 'さらに', reading: 'さらに' }),
|
||||
makeEntry({ wordId: 2, headword: '前に', word: '前に', reading: 'まえに' }),
|
||||
makeEntry({ wordId: 3, headword: 'バカ', word: 'バカ', reading: 'バカ' }),
|
||||
],
|
||||
new Set(),
|
||||
{ hideKnown: false, hideKanaOnly: true },
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.headword),
|
||||
['前に'],
|
||||
);
|
||||
});
|
||||
|
||||
test('cross-title table uses saved Hide Kana preference on first render', () => {
|
||||
const markup = withLocalStorage({ 'subminer.stats.crossAnimeWords.hideKanaOnly': 'true' }, () =>
|
||||
renderToStaticMarkup(
|
||||
<CrossAnimeWordsTable
|
||||
words={[
|
||||
makeEntry({ wordId: 1, headword: 'さらに', word: 'さらに', reading: 'さらに' }),
|
||||
makeEntry({ wordId: 2, headword: '前に', word: '前に', reading: 'まえに' }),
|
||||
]}
|
||||
knownWords={new Set()}
|
||||
/>,
|
||||
),
|
||||
);
|
||||
|
||||
assert.doesNotMatch(markup, />さらに</);
|
||||
assert.match(markup, />前に</);
|
||||
});
|
||||
@@ -1,219 +0,0 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import { PosBadge } from './pos-helpers';
|
||||
import { isKanaOnlyTokenText } from '../../lib/kana-token';
|
||||
import { readBooleanPreference, writeBooleanPreference } from '../../lib/preference-storage';
|
||||
import { fullReading } from '../../lib/reading-utils';
|
||||
import type { VocabularyEntry } from '../../types/stats';
|
||||
|
||||
interface CrossAnimeWordsTableProps {
|
||||
words: VocabularyEntry[];
|
||||
knownWords: Set<string>;
|
||||
onSelectWord?: (word: VocabularyEntry) => void;
|
||||
}
|
||||
|
||||
const PAGE_SIZE = 25;
|
||||
const HIDE_KANA_ONLY_STORAGE_KEY = 'subminer.stats.crossAnimeWords.hideKanaOnly';
|
||||
|
||||
interface CrossAnimeWordsOptions {
|
||||
hideKnown: boolean;
|
||||
hideKanaOnly: boolean;
|
||||
}
|
||||
|
||||
function isWordKnown(w: VocabularyEntry, knownWords: Set<string>): boolean {
|
||||
return knownWords.has(w.headword) || knownWords.has(w.word);
|
||||
}
|
||||
|
||||
function isKanaOnlyWord(w: VocabularyEntry): boolean {
|
||||
return isKanaOnlyTokenText(w.headword || w.word);
|
||||
}
|
||||
|
||||
export function buildCrossAnimeWordRows(
|
||||
words: VocabularyEntry[],
|
||||
knownWords: Set<string>,
|
||||
options: CrossAnimeWordsOptions,
|
||||
): VocabularyEntry[] {
|
||||
const hasKnownData = knownWords.size > 0;
|
||||
let filtered = words.filter((w) => w.animeCount >= 2);
|
||||
if (options.hideKnown && hasKnownData) {
|
||||
filtered = filtered.filter((w) => !isWordKnown(w, knownWords));
|
||||
}
|
||||
if (options.hideKanaOnly) {
|
||||
filtered = filtered.filter((w) => !isKanaOnlyWord(w));
|
||||
}
|
||||
|
||||
const byHeadword = new Map<string, VocabularyEntry>();
|
||||
for (const w of filtered) {
|
||||
const existing = byHeadword.get(w.headword);
|
||||
if (!existing) {
|
||||
byHeadword.set(w.headword, { ...w });
|
||||
} else {
|
||||
existing.frequency += w.frequency;
|
||||
existing.animeCount = Math.max(existing.animeCount, w.animeCount);
|
||||
if (
|
||||
w.frequencyRank != null &&
|
||||
(existing.frequencyRank == null || w.frequencyRank < existing.frequencyRank)
|
||||
) {
|
||||
existing.frequencyRank = w.frequencyRank;
|
||||
}
|
||||
if (!existing.reading && w.reading) existing.reading = w.reading;
|
||||
if (!existing.partOfSpeech && w.partOfSpeech) existing.partOfSpeech = w.partOfSpeech;
|
||||
}
|
||||
}
|
||||
|
||||
return [...byHeadword.values()].sort((a, b) => {
|
||||
if (b.animeCount !== a.animeCount) return b.animeCount - a.animeCount;
|
||||
return b.frequency - a.frequency;
|
||||
});
|
||||
}
|
||||
|
||||
export function CrossAnimeWordsTable({
|
||||
words,
|
||||
knownWords,
|
||||
onSelectWord,
|
||||
}: CrossAnimeWordsTableProps) {
|
||||
const [page, setPage] = useState(0);
|
||||
const [hideKnown, setHideKnown] = useState(true);
|
||||
const [hideKanaOnly, setHideKanaOnly] = useState(() =>
|
||||
readBooleanPreference(HIDE_KANA_ONLY_STORAGE_KEY, false),
|
||||
);
|
||||
const [collapsed, setCollapsed] = useState(false);
|
||||
|
||||
const hasKnownData = knownWords.size > 0;
|
||||
|
||||
const ranked = useMemo(() => {
|
||||
return buildCrossAnimeWordRows(words, knownWords, { hideKnown, hideKanaOnly });
|
||||
}, [words, knownWords, hideKnown, hideKanaOnly]);
|
||||
|
||||
const hasMultiAnimeWords = words.some((w) => w.animeCount >= 2);
|
||||
if (!hasMultiAnimeWords) return null;
|
||||
|
||||
const totalPages = Math.ceil(ranked.length / PAGE_SIZE);
|
||||
const paged = ranked.slice(page * PAGE_SIZE, (page + 1) * PAGE_SIZE);
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setCollapsed(!collapsed)}
|
||||
className="flex items-center gap-2 text-sm font-semibold text-ctp-text hover:text-ctp-subtext1 transition-colors"
|
||||
>
|
||||
<span
|
||||
className={`text-xs text-ctp-overlay2 transition-transform ${collapsed ? '' : 'rotate-90'}`}
|
||||
>
|
||||
{'\u25B6'}
|
||||
</span>
|
||||
Words Across Multiple Titles
|
||||
</button>
|
||||
<div className="flex flex-wrap items-center justify-end gap-2">
|
||||
{hasKnownData && (
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={hideKnown}
|
||||
onClick={() => {
|
||||
setHideKnown(!hideKnown);
|
||||
setPage(0);
|
||||
}}
|
||||
className={`px-2.5 py-1 rounded-lg text-xs transition-colors border ${
|
||||
hideKnown
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-blue/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Hide Known
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={hideKanaOnly}
|
||||
onClick={() => {
|
||||
const next = !hideKanaOnly;
|
||||
setHideKanaOnly(next);
|
||||
writeBooleanPreference(HIDE_KANA_ONLY_STORAGE_KEY, next);
|
||||
setPage(0);
|
||||
}}
|
||||
className={`px-2.5 py-1 rounded-lg text-xs transition-colors border ${
|
||||
hideKanaOnly
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-blue/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Hide Kana
|
||||
</button>
|
||||
<span className="text-xs text-ctp-overlay2">{ranked.length} words</span>
|
||||
</div>
|
||||
</div>
|
||||
{collapsed ? null : ranked.length === 0 ? (
|
||||
<div className="text-xs text-ctp-overlay2 mt-3">
|
||||
{hideKnown && hasKnownData && !hideKanaOnly
|
||||
? 'All words that span multiple titles are already known!'
|
||||
: (hideKnown && hasKnownData) || hideKanaOnly
|
||||
? 'No words across multiple titles match the active filters.'
|
||||
: 'No words found across multiple titles.'}
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="overflow-x-auto mt-3">
|
||||
<table className="w-full text-sm">
|
||||
<thead>
|
||||
<tr className="text-xs text-ctp-overlay2 border-b border-ctp-surface1">
|
||||
<th className="text-left py-2 pr-3 font-medium">Word</th>
|
||||
<th className="text-left py-2 pr-3 font-medium">Reading</th>
|
||||
<th className="text-left py-2 pr-3 font-medium w-20">POS</th>
|
||||
<th className="text-right py-2 pr-3 font-medium w-16">Titles</th>
|
||||
<th className="text-right py-2 font-medium w-16">Seen</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{paged.map((w) => (
|
||||
<tr
|
||||
key={w.wordId}
|
||||
onClick={() => onSelectWord?.(w)}
|
||||
className="border-b border-ctp-surface1 last:border-0 cursor-pointer hover:bg-ctp-surface1/50 transition-colors"
|
||||
>
|
||||
<td className="py-1.5 pr-3 text-ctp-text font-medium">{w.headword}</td>
|
||||
<td className="py-1.5 pr-3 text-ctp-subtext0">
|
||||
{fullReading(w.headword, w.reading) || w.headword}
|
||||
</td>
|
||||
<td className="py-1.5 pr-3">
|
||||
{w.partOfSpeech && <PosBadge pos={w.partOfSpeech} />}
|
||||
</td>
|
||||
<td className="py-1.5 pr-3 text-right font-mono tabular-nums text-ctp-green text-xs">
|
||||
{w.animeCount}
|
||||
</td>
|
||||
<td className="py-1.5 text-right font-mono tabular-nums text-ctp-blue text-xs">
|
||||
{w.frequency}x
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{totalPages > 1 && (
|
||||
<div className="flex items-center justify-center gap-3 mt-3 text-xs">
|
||||
<button
|
||||
type="button"
|
||||
disabled={page === 0}
|
||||
onClick={() => setPage(page - 1)}
|
||||
className="px-2 py-1 rounded bg-ctp-surface1 text-ctp-text disabled:opacity-30 hover:bg-ctp-surface2 transition-colors"
|
||||
>
|
||||
Prev
|
||||
</button>
|
||||
<span className="text-ctp-overlay2">
|
||||
{page + 1} / {totalPages}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
disabled={page >= totalPages - 1}
|
||||
onClick={() => setPage(page + 1)}
|
||||
className="px-2 py-1 rounded bg-ctp-surface1 text-ctp-text disabled:opacity-30 hover:bg-ctp-surface2 transition-colors"
|
||||
>
|
||||
Next
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import {
|
||||
buildFrequencyRankRows,
|
||||
FrequencyRankTable,
|
||||
isKanaOnlyTokenText,
|
||||
} from './FrequencyRankTable';
|
||||
import type { VocabularyEntry } from '../../types/stats';
|
||||
import { withLocalStorage } from '../../test-utils/dom';
|
||||
|
||||
function makeEntry(over: Partial<VocabularyEntry>): VocabularyEntry {
|
||||
return {
|
||||
wordId: 1,
|
||||
headword: '日本語',
|
||||
word: '日本語',
|
||||
reading: 'にほんご',
|
||||
frequency: 5,
|
||||
frequencyRank: 100,
|
||||
animeCount: 1,
|
||||
partOfSpeech: null,
|
||||
firstSeen: 0,
|
||||
lastSeen: 0,
|
||||
...over,
|
||||
} as VocabularyEntry;
|
||||
}
|
||||
|
||||
test('renders headword and reading inline in a single column (no separate Reading header)', () => {
|
||||
const entry = makeEntry({});
|
||||
const markup = renderToStaticMarkup(
|
||||
<FrequencyRankTable words={[entry]} knownWords={new Set()} />,
|
||||
);
|
||||
assert.ok(!markup.includes('>Reading<'), 'should not have a Reading column header');
|
||||
assert.ok(markup.includes('日本語'), 'should include the headword');
|
||||
assert.ok(markup.includes('にほんご'), 'should include the reading inline');
|
||||
});
|
||||
|
||||
test('omits reading when reading equals headword', () => {
|
||||
const entry = makeEntry({ headword: 'カレー', word: 'カレー', reading: 'カレー' });
|
||||
const markup = renderToStaticMarkup(
|
||||
<FrequencyRankTable words={[entry]} knownWords={new Set()} />,
|
||||
);
|
||||
assert.ok(markup.includes('カレー'), 'should include the headword');
|
||||
assert.ok(
|
||||
!markup.includes('【'),
|
||||
'should not render any bracketed reading when equal to headword',
|
||||
);
|
||||
});
|
||||
|
||||
test('identifies kana-only token text without hiding mixed kanji words', () => {
|
||||
assert.equal(isKanaOnlyTokenText('さらに'), true);
|
||||
assert.equal(isKanaOnlyTokenText('バカ'), true);
|
||||
assert.equal(isKanaOnlyTokenText('カレー'), true);
|
||||
assert.equal(isKanaOnlyTokenText('前に'), false);
|
||||
assert.equal(isKanaOnlyTokenText('間違いない'), false);
|
||||
});
|
||||
|
||||
test('frequency rows can hide kana-only headwords', () => {
|
||||
const rows = buildFrequencyRankRows(
|
||||
[
|
||||
makeEntry({ wordId: 1, headword: 'さらに', word: 'さらに', frequencyRank: 10 }),
|
||||
makeEntry({
|
||||
wordId: 2,
|
||||
headword: '前に',
|
||||
word: '前に',
|
||||
reading: 'まえに',
|
||||
frequencyRank: 20,
|
||||
}),
|
||||
makeEntry({ wordId: 3, headword: 'バカ', word: 'バカ', reading: 'バカ', frequencyRank: 30 }),
|
||||
],
|
||||
new Set(),
|
||||
{ hideKnown: false, hideKanaOnly: true },
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
rows.map((row) => row.headword),
|
||||
['前に'],
|
||||
);
|
||||
});
|
||||
|
||||
test('uses saved Hide Kana preference on first render', () => {
|
||||
const markup = withLocalStorage({ 'subminer.stats.frequencyRank.hideKanaOnly': 'true' }, () =>
|
||||
renderToStaticMarkup(
|
||||
<FrequencyRankTable
|
||||
words={[
|
||||
makeEntry({ wordId: 1, headword: 'さらに', word: 'さらに', frequencyRank: 10 }),
|
||||
makeEntry({
|
||||
wordId: 2,
|
||||
headword: '前に',
|
||||
word: '前に',
|
||||
reading: 'まえに',
|
||||
frequencyRank: 20,
|
||||
}),
|
||||
]}
|
||||
knownWords={new Set()}
|
||||
/>,
|
||||
),
|
||||
);
|
||||
|
||||
assert.doesNotMatch(markup, />さらに</);
|
||||
assert.match(markup, />前に</);
|
||||
});
|
||||
|
||||
test('uses saved Hide Known preference on first render', () => {
|
||||
const markup = withLocalStorage({ 'subminer.stats.frequencyRank.hideKnown': 'false' }, () =>
|
||||
renderToStaticMarkup(
|
||||
<FrequencyRankTable
|
||||
words={[makeEntry({ headword: '日本語', word: '日本語', frequencyRank: 10 })]}
|
||||
knownWords={new Set(['日本語'])}
|
||||
/>,
|
||||
),
|
||||
);
|
||||
|
||||
assert.match(markup, />Most Common Words Seen</);
|
||||
assert.match(markup, />日本語</);
|
||||
});
|
||||
@@ -1,238 +0,0 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import { PosBadge } from './pos-helpers';
|
||||
import { fullReading } from '../../lib/reading-utils';
|
||||
import { isKanaOnlyTokenText } from '../../lib/kana-token';
|
||||
import { readBooleanPreference, writeBooleanPreference } from '../../lib/preference-storage';
|
||||
import type { VocabularyEntry } from '../../types/stats';
|
||||
|
||||
interface FrequencyRankTableProps {
|
||||
words: VocabularyEntry[];
|
||||
knownWords: Set<string>;
|
||||
onSelectWord?: (word: VocabularyEntry) => void;
|
||||
}
|
||||
|
||||
const PAGE_SIZE = 25;
|
||||
const HIDE_KNOWN_STORAGE_KEY = 'subminer.stats.frequencyRank.hideKnown';
|
||||
const HIDE_KANA_ONLY_STORAGE_KEY = 'subminer.stats.frequencyRank.hideKanaOnly';
|
||||
|
||||
interface FrequencyRankOptions {
|
||||
hideKnown: boolean;
|
||||
hideKanaOnly: boolean;
|
||||
}
|
||||
|
||||
export { isKanaOnlyTokenText };
|
||||
|
||||
function isWordKnown(w: VocabularyEntry, knownWords: Set<string>): boolean {
|
||||
return knownWords.has(w.headword) || knownWords.has(w.word);
|
||||
}
|
||||
|
||||
function isKanaOnlyWord(w: VocabularyEntry): boolean {
|
||||
return isKanaOnlyTokenText(w.headword || w.word);
|
||||
}
|
||||
|
||||
export function buildFrequencyRankRows(
|
||||
words: VocabularyEntry[],
|
||||
knownWords: Set<string>,
|
||||
options: FrequencyRankOptions,
|
||||
): VocabularyEntry[] {
|
||||
const hasKnownData = knownWords.size > 0;
|
||||
let filtered = words.filter((w) => w.frequencyRank != null && w.frequencyRank > 0);
|
||||
if (options.hideKnown && hasKnownData) {
|
||||
filtered = filtered.filter((w) => !isWordKnown(w, knownWords));
|
||||
}
|
||||
if (options.hideKanaOnly) {
|
||||
filtered = filtered.filter((w) => !isKanaOnlyWord(w));
|
||||
}
|
||||
|
||||
const byHeadword = new Map<string, VocabularyEntry>();
|
||||
for (const w of filtered) {
|
||||
const existing = byHeadword.get(w.headword);
|
||||
if (!existing) {
|
||||
byHeadword.set(w.headword, { ...w });
|
||||
} else {
|
||||
existing.frequency += w.frequency;
|
||||
existing.animeCount = Math.max(existing.animeCount, w.animeCount);
|
||||
if (w.frequencyRank! < existing.frequencyRank!) {
|
||||
existing.frequencyRank = w.frequencyRank;
|
||||
}
|
||||
if (!existing.reading && w.reading) {
|
||||
existing.reading = w.reading;
|
||||
}
|
||||
if (!existing.partOfSpeech && w.partOfSpeech) {
|
||||
existing.partOfSpeech = w.partOfSpeech;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [...byHeadword.values()].sort((a, b) => a.frequencyRank! - b.frequencyRank!);
|
||||
}
|
||||
|
||||
export function FrequencyRankTable({ words, knownWords, onSelectWord }: FrequencyRankTableProps) {
|
||||
const [page, setPage] = useState(0);
|
||||
const [hideKnown, setHideKnown] = useState(() =>
|
||||
readBooleanPreference(HIDE_KNOWN_STORAGE_KEY, true),
|
||||
);
|
||||
const [hideKanaOnly, setHideKanaOnly] = useState(() =>
|
||||
readBooleanPreference(HIDE_KANA_ONLY_STORAGE_KEY, false),
|
||||
);
|
||||
const [collapsed, setCollapsed] = useState(false);
|
||||
|
||||
const hasKnownData = knownWords.size > 0;
|
||||
|
||||
const ranked = useMemo(() => {
|
||||
return buildFrequencyRankRows(words, knownWords, { hideKnown, hideKanaOnly });
|
||||
}, [words, knownWords, hideKnown, hideKanaOnly]);
|
||||
|
||||
if (words.every((w) => w.frequencyRank == null)) {
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<h3 className="text-sm font-semibold text-ctp-text mb-2">Most Common Words Seen</h3>
|
||||
<div className="text-xs text-ctp-overlay2">
|
||||
No frequency rank data available. Run the frequency backfill script or install a frequency
|
||||
dictionary.
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const totalPages = Math.ceil(ranked.length / PAGE_SIZE);
|
||||
const paged = ranked.slice(page * PAGE_SIZE, (page + 1) * PAGE_SIZE);
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setCollapsed(!collapsed)}
|
||||
className="flex items-center gap-2 text-sm font-semibold text-ctp-text hover:text-ctp-subtext1 transition-colors"
|
||||
>
|
||||
<span
|
||||
className={`text-xs text-ctp-overlay2 transition-transform ${collapsed ? '' : 'rotate-90'}`}
|
||||
>
|
||||
{'\u25B6'}
|
||||
</span>
|
||||
{hideKnown && hasKnownData ? 'Common Words Not Yet Mined' : 'Most Common Words Seen'}
|
||||
</button>
|
||||
<div className="flex flex-wrap items-center justify-end gap-2">
|
||||
{hasKnownData && (
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={hideKnown}
|
||||
onClick={() => {
|
||||
const next = !hideKnown;
|
||||
setHideKnown(next);
|
||||
writeBooleanPreference(HIDE_KNOWN_STORAGE_KEY, next);
|
||||
setPage(0);
|
||||
}}
|
||||
className={`px-2.5 py-1 rounded-lg text-xs transition-colors border ${
|
||||
hideKnown
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-blue/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Hide Known
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={hideKanaOnly}
|
||||
onClick={() => {
|
||||
const next = !hideKanaOnly;
|
||||
setHideKanaOnly(next);
|
||||
writeBooleanPreference(HIDE_KANA_ONLY_STORAGE_KEY, next);
|
||||
setPage(0);
|
||||
}}
|
||||
className={`px-2.5 py-1 rounded-lg text-xs transition-colors border ${
|
||||
hideKanaOnly
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-blue/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Hide Kana
|
||||
</button>
|
||||
<span className="text-xs text-ctp-overlay2">{ranked.length} words</span>
|
||||
</div>
|
||||
</div>
|
||||
{collapsed ? null : ranked.length === 0 ? (
|
||||
<div className="text-xs text-ctp-overlay2 mt-3">
|
||||
{hideKnown && hasKnownData && !hideKanaOnly
|
||||
? 'All ranked words are already in Anki!'
|
||||
: (hideKnown && hasKnownData) || hideKanaOnly
|
||||
? 'No ranked words match the active filters.'
|
||||
: 'No words with frequency data.'}
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="overflow-x-auto mt-3">
|
||||
<table className="w-full text-sm">
|
||||
<thead>
|
||||
<tr className="text-xs text-ctp-overlay2 border-b border-ctp-surface1">
|
||||
<th className="text-left py-2 pr-3 font-medium w-16">Rank</th>
|
||||
<th className="text-left py-2 pr-3 font-medium">Word</th>
|
||||
<th className="text-left py-2 pr-3 font-medium w-20">POS</th>
|
||||
<th className="text-right py-2 font-medium w-20">Seen</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{paged.map((w) => (
|
||||
<tr
|
||||
key={w.wordId}
|
||||
onClick={() => onSelectWord?.(w)}
|
||||
className="border-b border-ctp-surface1 last:border-0 cursor-pointer hover:bg-ctp-surface1/50 transition-colors"
|
||||
>
|
||||
<td className="py-1.5 pr-3 font-mono tabular-nums text-ctp-peach text-xs">
|
||||
#{w.frequencyRank!.toLocaleString()}
|
||||
</td>
|
||||
<td className="py-1.5 pr-3">
|
||||
<span className="text-ctp-text font-medium">{w.headword}</span>
|
||||
{(() => {
|
||||
const reading = fullReading(w.headword, w.reading);
|
||||
// `fullReading` normalizes katakana to hiragana, so we normalize the
|
||||
// headword the same way before comparing — otherwise katakana-only
|
||||
// entries like `カレー` would render `【かれー】`.
|
||||
const normalizedHeadword = fullReading(w.headword, w.headword);
|
||||
if (!reading || reading === normalizedHeadword) return null;
|
||||
return (
|
||||
<span className="text-ctp-subtext0 text-xs ml-1.5">【{reading}】</span>
|
||||
);
|
||||
})()}
|
||||
</td>
|
||||
<td className="py-1.5 pr-3">
|
||||
{w.partOfSpeech && <PosBadge pos={w.partOfSpeech} />}
|
||||
</td>
|
||||
<td className="py-1.5 text-right font-mono tabular-nums text-ctp-blue text-xs">
|
||||
{w.frequency}x
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{totalPages > 1 && (
|
||||
<div className="flex items-center justify-center gap-3 mt-3 text-xs">
|
||||
<button
|
||||
type="button"
|
||||
disabled={page === 0}
|
||||
onClick={() => setPage(page - 1)}
|
||||
className="px-2 py-1 rounded bg-ctp-surface1 text-ctp-text disabled:opacity-30 hover:bg-ctp-surface2 transition-colors"
|
||||
>
|
||||
Prev
|
||||
</button>
|
||||
<span className="text-ctp-overlay2">
|
||||
{page + 1} / {totalPages}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
disabled={page >= totalPages - 1}
|
||||
onClick={() => setPage(page + 1)}
|
||||
className="px-2 py-1 rounded bg-ctp-surface1 text-ctp-text disabled:opacity-30 hover:bg-ctp-surface2 transition-colors"
|
||||
>
|
||||
Next
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,89 +1,139 @@
|
||||
import { useMemo } from 'react';
|
||||
import type { KanjiEntry } from '../../types/stats';
|
||||
import { useMemo, useState } from 'react';
|
||||
import { formatNumber } from '../../lib/formatters';
|
||||
import type { KanjiEntry } from '../../types/stats';
|
||||
|
||||
interface KanjiBreakdownProps {
|
||||
kanji: KanjiEntry[];
|
||||
knownWords: ReadonlySet<string>;
|
||||
selectedKanjiId?: number | null;
|
||||
onSelectKanji?: (entry: KanjiEntry) => void;
|
||||
}
|
||||
|
||||
// Heat scale from rare (cool) to very frequent (warm). Catppuccin Macchiato.
|
||||
const FREQ_TIERS = [
|
||||
{ min: 0.85, color: 'text-ctp-peach', swatch: 'bg-ctp-peach', label: 'Very frequent' },
|
||||
{ min: 0.6, color: 'text-ctp-yellow', swatch: 'bg-ctp-yellow', label: 'Frequent' },
|
||||
{ min: 0.35, color: 'text-ctp-green', swatch: 'bg-ctp-green', label: 'Common' },
|
||||
{ min: 0.15, color: 'text-ctp-teal', swatch: 'bg-ctp-teal', label: 'Occasional' },
|
||||
{ min: 0, color: 'text-ctp-subtext0', swatch: 'bg-ctp-subtext0', label: 'Rare' },
|
||||
] as const;
|
||||
type KanjiFilter = 'all' | 'unknown';
|
||||
|
||||
function tierFor(intensity: number) {
|
||||
return FREQ_TIERS.find((tier) => intensity >= tier.min) ?? FREQ_TIERS[FREQ_TIERS.length - 1]!;
|
||||
// Rendering every kanji at once is slow to lay out and rarely what you want first.
|
||||
const INITIAL_VISIBLE = 180;
|
||||
const MIN_CELL_ALPHA = 0.08;
|
||||
const MAX_CELL_ALPHA = 0.6;
|
||||
const KANJI_CHARACTER = /\p{Script=Han}/u;
|
||||
|
||||
/** Every kanji appearing in at least one known word. */
|
||||
export function kanjiInKnownWords(knownWords: Iterable<string>): Set<string> {
|
||||
const found = new Set<string>();
|
||||
for (const word of knownWords) {
|
||||
for (const character of word) {
|
||||
if (KANJI_CHARACTER.test(character)) found.add(character);
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
/** Square root, so common kanji still differ while rare ones stay visible. */
|
||||
export function kanjiCellAlpha(frequency: number, maxFrequency: number): number {
|
||||
const intensity = maxFrequency > 0 ? Math.sqrt(frequency / maxFrequency) : 0;
|
||||
return MIN_CELL_ALPHA + (MAX_CELL_ALPHA - MIN_CELL_ALPHA) * intensity;
|
||||
}
|
||||
|
||||
export function KanjiBreakdown({
|
||||
kanji,
|
||||
knownWords,
|
||||
selectedKanjiId = null,
|
||||
onSelectKanji,
|
||||
}: KanjiBreakdownProps) {
|
||||
const { totalOccurrences, maxLogFreq } = useMemo(() => {
|
||||
let total = 0;
|
||||
let maxFreq = 1;
|
||||
for (const entry of kanji) {
|
||||
total += entry.frequency;
|
||||
if (entry.frequency > maxFreq) maxFreq = entry.frequency;
|
||||
}
|
||||
return { totalOccurrences: total, maxLogFreq: Math.log(maxFreq + 1) };
|
||||
}, [kanji]);
|
||||
const [filter, setFilter] = useState<KanjiFilter>('all');
|
||||
const [showAll, setShowAll] = useState(false);
|
||||
const hasKnownData = knownWords.size > 0;
|
||||
const knownKanji = useMemo(() => kanjiInKnownWords(knownWords), [knownWords]);
|
||||
const unknownKanji = useMemo(
|
||||
() => kanji.filter((entry) => !knownKanji.has(entry.kanji)),
|
||||
[kanji, knownKanji],
|
||||
);
|
||||
const maxFrequency = useMemo(
|
||||
() => kanji.reduce((max, entry) => Math.max(max, entry.frequency), 1),
|
||||
[kanji],
|
||||
);
|
||||
|
||||
if (kanji.length === 0) return null;
|
||||
|
||||
const matching = filter === 'unknown' ? unknownKanji : kanji;
|
||||
const visible = showAll ? matching : matching.slice(0, INITIAL_VISIBLE);
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<div className="mb-3 flex flex-wrap items-center justify-between gap-2">
|
||||
<h3 className="text-sm font-semibold text-ctp-text">
|
||||
Kanji Encountered
|
||||
<span className="ml-2 font-normal text-ctp-subtext0">
|
||||
{formatNumber(kanji.length)} unique · {formatNumber(totalOccurrences)} seen
|
||||
</span>
|
||||
</h3>
|
||||
<div className="flex items-center gap-1.5 text-[11px] text-ctp-subtext0">
|
||||
<span>rare</span>
|
||||
<div className="flex items-center gap-1">
|
||||
{[...FREQ_TIERS].reverse().map((tier) => (
|
||||
<span
|
||||
key={tier.label}
|
||||
className={`h-2 w-2 rounded-full ${tier.swatch}`}
|
||||
title={tier.label}
|
||||
/>
|
||||
<section
|
||||
aria-label="Kanji"
|
||||
className="rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4"
|
||||
>
|
||||
<div className="mb-3 flex flex-wrap items-start justify-between gap-2">
|
||||
<div>
|
||||
<h3 className="text-xs font-semibold text-ctp-text">Kanji</h3>
|
||||
<p className="mt-0.5 text-[11px] text-ctp-overlay2">
|
||||
Shaded by how often you have seen it.
|
||||
{hasKnownData ? ' Dashed means it is not in any word you know.' : ''}
|
||||
</p>
|
||||
</div>
|
||||
{hasKnownData ? (
|
||||
<div
|
||||
role="group"
|
||||
aria-label="Kanji filter"
|
||||
className="flex rounded-lg border border-ctp-surface1 p-0.5 text-[11px]"
|
||||
>
|
||||
{(
|
||||
[
|
||||
['all', `All ${formatNumber(kanji.length)}`],
|
||||
['unknown', `Not in a known word ${formatNumber(unknownKanji.length)}`],
|
||||
] as const
|
||||
).map(([value, label]) => (
|
||||
<button
|
||||
key={value}
|
||||
type="button"
|
||||
aria-pressed={filter === value}
|
||||
onClick={() => setFilter(value)}
|
||||
className={`rounded-md px-2 py-0.5 transition-colors ${
|
||||
filter === value
|
||||
? 'bg-ctp-surface2 text-ctp-text'
|
||||
: 'text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<span>frequent</span>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-1">
|
||||
{kanji.map((k) => {
|
||||
// Log scale keeps the heavily-skewed frequency distribution readable.
|
||||
const intensity = maxLogFreq > 0 ? Math.log(k.frequency + 1) / maxLogFreq : 0;
|
||||
const tier = tierFor(intensity);
|
||||
const selected = selectedKanjiId === k.kanjiId;
|
||||
<div className="grid grid-cols-[repeat(auto-fill,minmax(2.25rem,1fr))] gap-1">
|
||||
{visible.map((entry) => {
|
||||
const selected = selectedKanjiId === entry.kanjiId;
|
||||
const outsideKnown = hasKnownData && !knownKanji.has(entry.kanji);
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
key={k.kanji}
|
||||
className={`cursor-pointer rounded-md px-1.5 py-0.5 text-xl leading-none font-medium transition-colors duration-150 ${tier.color} ${
|
||||
selected ? 'bg-ctp-surface2 ring-1 ring-ctp-lavender' : 'hover:bg-ctp-surface1'
|
||||
}`}
|
||||
title={`${k.kanji} — seen ${formatNumber(k.frequency)}×`}
|
||||
aria-label={`${k.kanji} — seen ${k.frequency} times`}
|
||||
key={entry.kanji}
|
||||
lang="ja"
|
||||
title={`${entry.kanji}: seen ${formatNumber(entry.frequency)} times`}
|
||||
aria-label={`${entry.kanji}, seen ${entry.frequency} times`}
|
||||
aria-pressed={selected}
|
||||
onClick={() => onSelectKanji?.(k)}
|
||||
onClick={() => onSelectKanji?.(entry)}
|
||||
style={{
|
||||
backgroundColor: `rgb(138 173 244 / ${kanjiCellAlpha(entry.frequency, maxFrequency).toFixed(2)})`,
|
||||
}}
|
||||
className={`flex aspect-square items-center justify-center rounded text-lg leading-none text-ctp-text transition hover:ring-1 hover:ring-ctp-blue ${
|
||||
selected ? 'ring-1 ring-ctp-lavender' : ''
|
||||
} ${outsideKnown ? 'outline-1 -outline-offset-1 outline-ctp-blue/70 outline-dashed' : ''}`}
|
||||
>
|
||||
{k.kanji}
|
||||
{entry.kanji}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
{matching.length > INITIAL_VISIBLE ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowAll((value) => !value)}
|
||||
className="mt-2 text-[11px] text-ctp-overlay2 transition-colors hover:text-ctp-blue"
|
||||
>
|
||||
{showAll ? 'Show fewer' : `Show all ${formatNumber(matching.length)}`}
|
||||
</button>
|
||||
) : null}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import { formatNumber } from '../../lib/formatters';
|
||||
import type { VocabularyCoverageBand, VocabularyCoverageBandId } from '../../types/stats';
|
||||
|
||||
const BAND_LABELS: Record<VocabularyCoverageBandId, string> = {
|
||||
top1k: 'Top 1k',
|
||||
top5k: '1k to 5k',
|
||||
top10k: '5k to 10k',
|
||||
top20k: '10k to 20k',
|
||||
};
|
||||
|
||||
/** How much of each frequency band you know, which points at where to mine next. */
|
||||
export function VocabularyCoverage({ bands }: { bands: VocabularyCoverageBand[] }) {
|
||||
return (
|
||||
<div className="h-full rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4">
|
||||
<h3 className="text-xs font-semibold text-ctp-text">Known Words by Frequency</h3>
|
||||
<p className="mb-3 mt-0.5 text-[11px] text-ctp-overlay2">
|
||||
Words you have seen in each band that are already in Anki.
|
||||
</p>
|
||||
<div className="space-y-3">
|
||||
{bands.map(({ band, seen, known }) => {
|
||||
const percent = seen > 0 ? Math.round((known / seen) * 100) : 0;
|
||||
return (
|
||||
<div
|
||||
key={band}
|
||||
className="grid grid-cols-[4.5rem_minmax(0,1fr)_auto] items-center gap-2 text-xs"
|
||||
>
|
||||
<span className="text-ctp-subtext0">{BAND_LABELS[band]}</span>
|
||||
<div
|
||||
role="img"
|
||||
aria-label={`${percent}% known`}
|
||||
className="h-2 overflow-hidden rounded-full bg-ctp-surface1/60"
|
||||
>
|
||||
<div
|
||||
className="h-full rounded-full bg-ctp-blue"
|
||||
style={{ width: `${seen > 0 ? (known / seen) * 100 : 0}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="whitespace-nowrap text-right text-[11px] tabular-nums text-ctp-overlay2">
|
||||
{formatNumber(known)} of {formatNumber(seen)}, {percent}%
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { KanjiBreakdown, kanjiCellAlpha, kanjiInKnownWords } from './KanjiBreakdown';
|
||||
import { VocabularyCoverage } from './VocabularyCoverage';
|
||||
import { highlightWord } from './WordDetailPanel';
|
||||
import { buildVocabularyGrowth } from './VocabularyTab';
|
||||
import {
|
||||
buildWordQuery,
|
||||
defaultSortForView,
|
||||
nextSort,
|
||||
type WordExplorerState,
|
||||
} from './word-explorer-query';
|
||||
|
||||
const baseState: WordExplorerState = {
|
||||
view: 'all',
|
||||
search: '',
|
||||
hideKnown: false,
|
||||
hideNames: true,
|
||||
hideKana: true,
|
||||
partOfSpeech: null,
|
||||
sort: defaultSortForView('all'),
|
||||
page: 2,
|
||||
};
|
||||
|
||||
test('Worth mining means common and unknown, whatever the Hide known chip says', () => {
|
||||
const query = buildWordQuery({ ...baseState, view: 'worth', sort: defaultSortForView('worth') });
|
||||
assert.equal(query.hideKnown, true);
|
||||
assert.equal(query.maxRank, 10_000);
|
||||
assert.equal(query.sort, 'seen');
|
||||
assert.equal(query.offset, 50);
|
||||
});
|
||||
|
||||
test('Across titles needs two titles and sorts by them', () => {
|
||||
const query = buildWordQuery({
|
||||
...baseState,
|
||||
view: 'titles',
|
||||
sort: defaultSortForView('titles'),
|
||||
});
|
||||
assert.equal(query.minTitles, 2);
|
||||
assert.equal(query.sort, 'titles');
|
||||
});
|
||||
|
||||
test('searching ignores the view and filters so any word can be found', () => {
|
||||
const query = buildWordQuery({
|
||||
...baseState,
|
||||
view: 'worth',
|
||||
search: ' 猫 ',
|
||||
hideKnown: true,
|
||||
partOfSpeech: 'verb',
|
||||
});
|
||||
assert.equal(query.search, '猫');
|
||||
assert.equal(query.hideKnown, false);
|
||||
assert.equal(query.hideNames, false);
|
||||
assert.equal(query.hideKana, false);
|
||||
assert.equal(query.partOfSpeech, undefined);
|
||||
assert.equal(query.maxRank, undefined);
|
||||
});
|
||||
|
||||
test('nextSort flips the active column and starts new ones on their useful end', () => {
|
||||
assert.deepEqual(nextSort({ sort: 'seen', descending: true }, 'seen'), {
|
||||
sort: 'seen',
|
||||
descending: false,
|
||||
});
|
||||
assert.deepEqual(nextSort({ sort: 'seen', descending: true }, 'rank'), {
|
||||
sort: 'rank',
|
||||
descending: false,
|
||||
});
|
||||
assert.deepEqual(nextSort({ sort: 'rank', descending: false }, 'titles'), {
|
||||
sort: 'titles',
|
||||
descending: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('vocabulary growth is a running total', () => {
|
||||
const growth = buildVocabularyGrowth([
|
||||
{ epochDay: 20_000, wordCount: 5 },
|
||||
{ epochDay: 20_010, wordCount: 2 },
|
||||
]);
|
||||
assert.deepEqual(
|
||||
growth.map((point) => point.value),
|
||||
[5, 7],
|
||||
);
|
||||
});
|
||||
|
||||
test('kanjiInKnownWords collects kanji and skips kana and Latin', () => {
|
||||
assert.deepEqual(
|
||||
[...kanjiInKnownWords(['見る', 'さん (used with adults)', '大丈夫'])].sort(),
|
||||
['丈', '夫', '大', '見'].sort(),
|
||||
);
|
||||
});
|
||||
|
||||
test('kanjiCellAlpha keeps rare kanji visible and scales up to the most seen', () => {
|
||||
assert.ok(kanjiCellAlpha(1, 7_000) > 0.08);
|
||||
assert.ok(kanjiCellAlpha(900, 7_000) < kanjiCellAlpha(4_000, 7_000));
|
||||
assert.equal(kanjiCellAlpha(7_000, 7_000), 0.6);
|
||||
});
|
||||
|
||||
const KANJI = [
|
||||
{ kanjiId: 1, kanji: '人', frequency: 100, firstSeen: 0, lastSeen: 0 },
|
||||
{ kanjiId: 2, kanji: '鈴', frequency: 50, firstSeen: 0, lastSeen: 0 },
|
||||
];
|
||||
|
||||
test('KanjiBreakdown reports the real totals and flags kanji outside known words', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<KanjiBreakdown kanji={KANJI} knownWords={new Set(['人間'])} />,
|
||||
);
|
||||
assert.match(markup, /All 2/);
|
||||
assert.match(markup, /Not in a known word 1/);
|
||||
assert.equal(markup.match(/outline-dashed/g)?.length, 1);
|
||||
assert.match(markup, /lang="ja"/);
|
||||
});
|
||||
|
||||
test('KanjiBreakdown drops known-word features when there is no known-word data', () => {
|
||||
const markup = renderToStaticMarkup(<KanjiBreakdown kanji={KANJI} knownWords={new Set()} />);
|
||||
assert.doesNotMatch(markup, /Not in a known word/);
|
||||
assert.doesNotMatch(markup, /outline-dashed/);
|
||||
});
|
||||
|
||||
test('VocabularyCoverage shows each band as a share of the words seen in it', () => {
|
||||
const markup = renderToStaticMarkup(
|
||||
<VocabularyCoverage
|
||||
bands={[
|
||||
{ band: 'top1k', seen: 910, known: 690 },
|
||||
{ band: 'top5k', seen: 0, known: 0 },
|
||||
]}
|
||||
/>,
|
||||
);
|
||||
assert.match(markup, /690 of 910, 76%/);
|
||||
assert.match(markup, /0 of 0, 0%/);
|
||||
});
|
||||
|
||||
test('highlightWord marks the longest form, whatever order forms arrive in', () => {
|
||||
const markup = renderToStaticMarkup(<p>{highlightWord('食べたい', ['食べ', '食べた'])}</p>);
|
||||
assert.match(markup, />食べた<\/mark>い/);
|
||||
});
|
||||
@@ -1,45 +1,60 @@
|
||||
import { useState, useMemo } from 'react';
|
||||
import { useMemo, useState } from 'react';
|
||||
import type { ExcludedWord } from '../../hooks/useExcludedWords';
|
||||
import { useVocabulary } from '../../hooks/useVocabulary';
|
||||
import { StatCard } from '../layout/StatCard';
|
||||
import { WordList } from './WordList';
|
||||
import { epochDayToDate, formatNumber } from '../../lib/formatters';
|
||||
import { writeBooleanPreference } from '../../lib/preference-storage';
|
||||
import type { DictionaryWordRow, KanjiEntry, StatsVocabularyCharts } from '../../types/stats';
|
||||
import { SummaryTile } from '../layout/SummaryTile';
|
||||
import { TrendChart } from '../trends/TrendChart';
|
||||
import { DuplicateLineCleanup } from './DuplicateLineCleanup';
|
||||
import { ExclusionManager } from './ExclusionManager';
|
||||
import { KanjiBreakdown } from './KanjiBreakdown';
|
||||
import { KanjiDetailPanel } from './KanjiDetailPanel';
|
||||
import { ExclusionManager } from './ExclusionManager';
|
||||
import { DuplicateLineCleanup } from './DuplicateLineCleanup';
|
||||
import { epochDayToDate, formatNumber } from '../../lib/formatters';
|
||||
import { TrendChart } from '../trends/TrendChart';
|
||||
import { FrequencyRankTable } from './FrequencyRankTable';
|
||||
import { CrossAnimeWordsTable } from './CrossAnimeWordsTable';
|
||||
import type { ExcludedWord } from '../../hooks/useExcludedWords';
|
||||
import type { KanjiEntry, VocabularyEntry } from '../../types/stats';
|
||||
import { VocabularyCoverage } from './VocabularyCoverage';
|
||||
import { WordExplorer } from './WordExplorer';
|
||||
import { HIDE_NAMES_KEY, loadHideNames } from './word-explorer-query';
|
||||
|
||||
interface VocabularyTabProps {
|
||||
onNavigateToAnime?: (animeId: number) => void;
|
||||
onOpenWordDetail?: (wordId: number) => void;
|
||||
excluded: ExcludedWord[];
|
||||
isExcluded: (w: { headword: string; word: string; reading: string }) => boolean;
|
||||
onToggleExclusion: (w: { headword: string; word: string; reading: string }) => void;
|
||||
onRemoveExclusion: (w: ExcludedWord) => void;
|
||||
onClearExclusions: () => void;
|
||||
}
|
||||
|
||||
function isProperNoun(w: VocabularyEntry): boolean {
|
||||
return w.pos2 === '固有名詞';
|
||||
const GROWTH_COLOR = '#8aadf4';
|
||||
|
||||
/** Running total of words seen, one point per day a new word appeared. */
|
||||
export function buildVocabularyGrowth(timeline: StatsVocabularyCharts['newWordsTimeline']) {
|
||||
let total = 0;
|
||||
return timeline.map((point) => {
|
||||
total += point.wordCount;
|
||||
return {
|
||||
label: epochDayToDate(point.epochDay).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
}),
|
||||
value: total,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export function VocabularyTab({
|
||||
onNavigateToAnime,
|
||||
onOpenWordDetail,
|
||||
excluded,
|
||||
isExcluded,
|
||||
onToggleExclusion,
|
||||
onRemoveExclusion,
|
||||
onClearExclusions,
|
||||
}: VocabularyTabProps) {
|
||||
const {
|
||||
words,
|
||||
kanji,
|
||||
knownWords,
|
||||
summary,
|
||||
charts,
|
||||
coverage,
|
||||
dataRevision,
|
||||
loading,
|
||||
error,
|
||||
aggregatesError,
|
||||
@@ -47,35 +62,15 @@ export function VocabularyTab({
|
||||
reload,
|
||||
} = useVocabulary();
|
||||
const [selectedKanjiId, setSelectedKanjiId] = useState<number | null>(null);
|
||||
const [hideNames, setHideNames] = useState(false);
|
||||
const [hideNames, setHideNames] = useState(() => loadHideNames());
|
||||
const [showExclusionManager, setShowExclusionManager] = useState(false);
|
||||
const [showDuplicateLineCleanup, setShowDuplicateLineCleanup] = useState(false);
|
||||
|
||||
const hasNames = useMemo(() => words.some(isProperNoun), [words]);
|
||||
const filteredWords = useMemo(() => {
|
||||
let result = words;
|
||||
if (hideNames) result = result.filter((w) => !isProperNoun(w));
|
||||
if (excluded.length > 0) result = result.filter((w) => !isExcluded(w));
|
||||
return result;
|
||||
}, [words, hideNames, excluded, isExcluded]);
|
||||
const chartData = useMemo(
|
||||
() => ({
|
||||
topWords: ((hideNames ? charts?.topWordsWithoutNames : charts?.topWords) ?? []).map(
|
||||
(word) => ({
|
||||
label: word.headword,
|
||||
value: word.frequency,
|
||||
}),
|
||||
const growth = useMemo(
|
||||
() =>
|
||||
buildVocabularyGrowth(
|
||||
(hideNames ? charts?.newWordsTimelineWithoutNames : charts?.newWordsTimeline) ?? [],
|
||||
),
|
||||
newWordsTimeline: (
|
||||
(hideNames ? charts?.newWordsTimelineWithoutNames : charts?.newWordsTimeline) ?? []
|
||||
).map((point) => ({
|
||||
label: epochDayToDate(point.epochDay).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
}),
|
||||
value: point.wordCount,
|
||||
})),
|
||||
}),
|
||||
[charts, hideNames],
|
||||
);
|
||||
|
||||
@@ -94,61 +89,53 @@ export function VocabularyTab({
|
||||
);
|
||||
}
|
||||
|
||||
const handleSelectWord = (entry: VocabularyEntry): void => {
|
||||
onOpenWordDetail?.(entry.wordId);
|
||||
const updateHideNames = (hide: boolean) => {
|
||||
setHideNames(hide);
|
||||
writeBooleanPreference(HIDE_NAMES_KEY, hide);
|
||||
};
|
||||
|
||||
const handleBarClick = (headword: string): void => {
|
||||
const match = (hideNames ? charts?.topWordsWithoutNames : charts?.topWords)?.find(
|
||||
(word) => word.headword === headword,
|
||||
);
|
||||
if (match) onOpenWordDetail?.(match.wordId);
|
||||
const hideWord = (row: DictionaryWordRow) => {
|
||||
onToggleExclusion({ headword: row.headword, word: row.headword, reading: row.reading });
|
||||
};
|
||||
const openKanjiDetail = (entry: KanjiEntry) => setSelectedKanjiId(entry.kanjiId);
|
||||
|
||||
const openKanjiDetail = (entry: KanjiEntry): void => {
|
||||
setSelectedKanjiId(entry.kanjiId);
|
||||
};
|
||||
|
||||
const displayedSummary = hideNames
|
||||
? {
|
||||
uniqueWords: summary?.uniqueWordsWithoutNames ?? 0,
|
||||
newThisWeek: summary?.newThisWeekWithoutNames ?? 0,
|
||||
knownWordCount: summary?.knownWordCountWithoutNames ?? null,
|
||||
}
|
||||
: {
|
||||
uniqueWords: summary?.uniqueWords ?? 0,
|
||||
newThisWeek: summary?.newThisWeek ?? 0,
|
||||
knownWordCount: summary?.knownWordCount ?? null,
|
||||
};
|
||||
const uniqueWords = (hideNames ? summary?.uniqueWordsWithoutNames : summary?.uniqueWords) ?? 0;
|
||||
const newThisWeek = (hideNames ? summary?.newThisWeekWithoutNames : summary?.newThisWeek) ?? 0;
|
||||
const knownWordCount =
|
||||
(hideNames ? summary?.knownWordCountWithoutNames : summary?.knownWordCount) ?? null;
|
||||
const knownPercent =
|
||||
knownWordCount !== null && uniqueWords > 0
|
||||
? Math.round((knownWordCount / uniqueWords) * 100)
|
||||
: 0;
|
||||
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
<div className="grid grid-cols-2 xl:grid-cols-4 gap-3">
|
||||
<StatCard
|
||||
label="Unique Words"
|
||||
value={summary ? formatNumber(displayedSummary.uniqueWords) : '…'}
|
||||
color="text-ctp-blue"
|
||||
<section
|
||||
aria-label="Vocabulary summary"
|
||||
className="grid grid-cols-2 gap-4 rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4 lg:grid-cols-4"
|
||||
>
|
||||
<SummaryTile
|
||||
label="Words seen"
|
||||
value={summary ? formatNumber(uniqueWords) : '…'}
|
||||
detail={hideNames ? 'Dictionary forms, names excluded' : 'Dictionary forms'}
|
||||
/>
|
||||
{displayedSummary.knownWordCount !== null ? (
|
||||
<StatCard
|
||||
label="Known Words"
|
||||
value={`${formatNumber(displayedSummary.knownWordCount)} (${displayedSummary.uniqueWords > 0 ? Math.round((displayedSummary.knownWordCount / displayedSummary.uniqueWords) * 100) : 0}%)`}
|
||||
color="text-ctp-green"
|
||||
{knownWordCount !== null ? (
|
||||
<SummaryTile
|
||||
label="Known in Anki"
|
||||
value={formatNumber(knownWordCount)}
|
||||
detail={`${knownPercent}% of words seen`}
|
||||
/>
|
||||
) : knownWords.size > 0 ? (
|
||||
<StatCard label="Known Words" value="…" color="text-ctp-green" />
|
||||
) : null}
|
||||
<StatCard
|
||||
label="Unique Kanji"
|
||||
<SummaryTile
|
||||
label="New in the last 7 days"
|
||||
value={summary ? `+${formatNumber(newThisWeek)}` : '…'}
|
||||
detail={null}
|
||||
/>
|
||||
<SummaryTile
|
||||
label="Kanji seen"
|
||||
value={summary ? formatNumber(summary.uniqueKanji) : '…'}
|
||||
color="text-ctp-teal"
|
||||
detail={null}
|
||||
/>
|
||||
<StatCard
|
||||
label="New This Week"
|
||||
value={summary ? `+${formatNumber(displayedSummary.newThisWeek)}` : '…'}
|
||||
color="text-ctp-mauve"
|
||||
/>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{aggregatesError && (
|
||||
<p className="text-xs text-ctp-red" role="alert">
|
||||
@@ -163,54 +150,11 @@ export function VocabularyTab({
|
||||
</p>
|
||||
)}
|
||||
|
||||
<div className="flex items-center justify-end gap-3">
|
||||
{hasNames && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setHideNames(!hideNames)}
|
||||
className={`shrink-0 px-3 py-2 rounded-lg text-xs transition-colors border ${
|
||||
hideNames
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-blue/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Hide Names
|
||||
</button>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowDuplicateLineCleanup(true)}
|
||||
className="shrink-0 rounded-lg border border-ctp-surface1 bg-ctp-surface0 px-3 py-2 text-xs text-ctp-overlay2 transition-colors hover:text-ctp-subtext0"
|
||||
>
|
||||
Duplicates
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowExclusionManager(true)}
|
||||
className={`shrink-0 px-3 py-2 rounded-lg text-xs transition-colors border ${
|
||||
excluded.length > 0
|
||||
? 'bg-ctp-surface2 text-ctp-text border-ctp-red/50'
|
||||
: 'bg-ctp-surface0 text-ctp-overlay2 border-ctp-surface1 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
Exclusions{excluded.length > 0 && ` (${excluded.length})`}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 xl:grid-cols-2 gap-4">
|
||||
<TrendChart
|
||||
title="Top Repeated Words"
|
||||
data={chartData.topWords}
|
||||
color="#8aadf4"
|
||||
type="bar"
|
||||
onBarClick={handleBarClick}
|
||||
/>
|
||||
<TrendChart
|
||||
title="New Words by Day"
|
||||
data={chartData.newWordsTimeline}
|
||||
color="#c6a0f6"
|
||||
type="line"
|
||||
/>
|
||||
<div className="grid grid-cols-1 gap-4 xl:grid-cols-2">
|
||||
{coverage.length > 0 ? <VocabularyCoverage bands={coverage} /> : null}
|
||||
<div className={coverage.length > 0 ? '' : 'xl:col-span-2'}>
|
||||
<TrendChart title="Vocabulary Growth" data={growth} color={GROWTH_COLOR} type="line" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{charts && !charts.ready && (
|
||||
@@ -219,26 +163,39 @@ export function VocabularyTab({
|
||||
</p>
|
||||
)}
|
||||
|
||||
<FrequencyRankTable
|
||||
words={filteredWords}
|
||||
knownWords={knownWords}
|
||||
onSelectWord={handleSelectWord}
|
||||
<WordExplorer
|
||||
hasKnownData={knownWords.size > 0}
|
||||
hideNames={hideNames}
|
||||
onHideNamesChange={updateHideNames}
|
||||
refreshKey={dataRevision}
|
||||
onOpenWord={(wordId) => onOpenWordDetail?.(wordId)}
|
||||
onHideWord={hideWord}
|
||||
/>
|
||||
|
||||
<CrossAnimeWordsTable
|
||||
words={filteredWords}
|
||||
knownWords={knownWords}
|
||||
onSelectWord={handleSelectWord}
|
||||
/>
|
||||
|
||||
<WordList words={filteredWords} selectedKey={null} onSelectWord={handleSelectWord} />
|
||||
|
||||
<KanjiBreakdown
|
||||
kanji={kanji}
|
||||
knownWords={knownWords}
|
||||
selectedKanjiId={selectedKanjiId}
|
||||
onSelectKanji={openKanjiDetail}
|
||||
/>
|
||||
|
||||
<div className="flex justify-end gap-4 text-[11px] text-ctp-overlay2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowExclusionManager(true)}
|
||||
className="transition-colors hover:text-ctp-text"
|
||||
>
|
||||
Exclusions{excluded.length > 0 && ` (${excluded.length})`}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowDuplicateLineCleanup(true)}
|
||||
className="transition-colors hover:text-ctp-text"
|
||||
>
|
||||
Clean up duplicates
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<KanjiDetailPanel
|
||||
kanjiId={selectedKanjiId}
|
||||
onClose={() => setSelectedKanjiId(null)}
|
||||
|
||||
@@ -4,11 +4,14 @@ import { apiClient } from '../../lib/api-client';
|
||||
import { epochMsFromDbTimestamp, formatNumber, formatRelativeDate } from '../../lib/formatters';
|
||||
import {
|
||||
buildStatsMineCardParams,
|
||||
describeStreamedLines,
|
||||
getStatsMineCardError,
|
||||
getStatsMineCardUnavailableReason,
|
||||
} from '../../lib/mining';
|
||||
import { fullReading } from '../../lib/reading-utils';
|
||||
import type { VocabularyOccurrenceEntry } from '../../types/stats';
|
||||
import { MineUnavailableNote } from '../common/MineUnavailableNote';
|
||||
import { SummaryTile } from '../layout/SummaryTile';
|
||||
import { PosBadge } from './pos-helpers';
|
||||
|
||||
const INITIAL_PAGE_SIZE = 5;
|
||||
@@ -26,8 +29,9 @@ interface WordDetailPanelProps {
|
||||
onToggleExclusion?: (w: { headword: string; word: string; reading: string }) => void;
|
||||
}
|
||||
|
||||
function highlightWord(text: string, words: string[]): React.ReactNode {
|
||||
const needles = words.filter(Boolean);
|
||||
export function highlightWord(text: string, words: string[]): React.ReactNode {
|
||||
// Longest first: regex alternation takes the first match, so 食べ would cut 食べた short.
|
||||
const needles = words.filter(Boolean).sort((left, right) => right.length - left.length);
|
||||
if (needles.length === 0) return text;
|
||||
|
||||
const escaped = needles.map((w) => w.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'));
|
||||
@@ -49,6 +53,11 @@ function highlightWord(text: string, words: string[]): React.ReactNode {
|
||||
);
|
||||
}
|
||||
|
||||
function displayReading(detail: { headword: string; dictionaryReading: string }): string {
|
||||
const reading = fullReading(detail.headword, detail.dictionaryReading);
|
||||
return reading === fullReading(detail.headword, detail.headword) ? '' : reading;
|
||||
}
|
||||
|
||||
function formatSegment(ms: number | null): string {
|
||||
if (ms == null || !Number.isFinite(ms)) return '--:--';
|
||||
const totalSeconds = Math.max(0, Math.floor(ms / 1000));
|
||||
@@ -88,8 +97,6 @@ export function WordDetailPanel({
|
||||
requestIdRef.current++;
|
||||
}, [wordId]);
|
||||
|
||||
if (wordId === null) return null;
|
||||
|
||||
const loadOccurrences = async (
|
||||
detail: NonNullable<typeof data>['detail'],
|
||||
offset: number,
|
||||
@@ -110,6 +117,7 @@ export function WordDetailPanel({
|
||||
detail.reading,
|
||||
limit,
|
||||
offset,
|
||||
true,
|
||||
);
|
||||
if (reqId !== requestIdRef.current) return;
|
||||
setOccurrences((prev) => (append ? [...prev, ...rows] : rows));
|
||||
@@ -129,10 +137,16 @@ export function WordDetailPanel({
|
||||
}
|
||||
};
|
||||
|
||||
const handleShowOccurrences = () => {
|
||||
if (!data) return;
|
||||
// The examples are where mining happens, so they load with the word. The detail
|
||||
// may still be the previous word's while the new one loads.
|
||||
const loadedWordId = data?.detail.wordId ?? null;
|
||||
useEffect(() => {
|
||||
if (!data || loadedWordId !== wordId) return;
|
||||
void loadOccurrences(data.detail, 0, INITIAL_PAGE_SIZE, false);
|
||||
};
|
||||
// Only a newly opened word should reload examples.
|
||||
}, [loadedWordId, wordId]);
|
||||
|
||||
if (wordId === null) return null;
|
||||
|
||||
const handleLoadMore = () => {
|
||||
if (!data || occLoadingMore || !hasMore) return;
|
||||
@@ -182,6 +196,11 @@ export function WordDetailPanel({
|
||||
}
|
||||
};
|
||||
|
||||
const dictionaryReading = data ? displayReading(data.detail) : '';
|
||||
const streamedLinesNotice = data
|
||||
? describeStreamedLines(data.detail.lineCount, data.detail.streamLineCount)
|
||||
: null;
|
||||
|
||||
return (
|
||||
<div className="fixed inset-0 z-40 flex items-center justify-center p-4">
|
||||
<button
|
||||
@@ -201,30 +220,19 @@ export function WordDetailPanel({
|
||||
{error && <div className="mt-2 text-sm text-ctp-red">Error: {error}</div>}
|
||||
{data && (
|
||||
<>
|
||||
<h2 className="mt-1 truncate text-3xl font-semibold text-ctp-text">
|
||||
<h2 lang="ja" className="mt-1 truncate text-3xl font-semibold text-ctp-text">
|
||||
{data.detail.headword}
|
||||
</h2>
|
||||
<div className="mt-1 text-sm text-ctp-subtext0">
|
||||
{fullReading(data.detail.headword, data.detail.reading) || data.detail.word}
|
||||
</div>
|
||||
<div className="mt-2 flex flex-wrap gap-1.5">
|
||||
{data.detail.partOfSpeech && <PosBadge pos={data.detail.partOfSpeech} />}
|
||||
{data.detail.pos1 && data.detail.pos1 !== data.detail.partOfSpeech && (
|
||||
<span className="rounded-full bg-ctp-surface1 px-2 py-0.5 text-[11px] text-ctp-subtext0">
|
||||
{data.detail.pos1}
|
||||
</span>
|
||||
)}
|
||||
{data.detail.pos2 && (
|
||||
<span className="rounded-full bg-ctp-surface1 px-2 py-0.5 text-[11px] text-ctp-subtext0">
|
||||
{data.detail.pos2}
|
||||
</span>
|
||||
)}
|
||||
{data.detail.pos3 && (
|
||||
<span className="rounded-full bg-ctp-surface1 px-2 py-0.5 text-[11px] text-ctp-subtext0">
|
||||
{data.detail.pos3}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
{dictionaryReading && (
|
||||
<div lang="ja" className="mt-1 text-sm text-ctp-subtext0">
|
||||
{dictionaryReading}
|
||||
</div>
|
||||
)}
|
||||
{data.detail.partOfSpeech && (
|
||||
<div className="mt-2">
|
||||
<PosBadge pos={data.detail.partOfSpeech} />
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
@@ -255,27 +263,62 @@ export function WordDetailPanel({
|
||||
<div className="flex-1 overflow-y-auto px-5 py-4 space-y-5">
|
||||
{data && (
|
||||
<>
|
||||
<div className="grid grid-cols-3 gap-3">
|
||||
<div className="rounded-lg bg-ctp-surface0 p-3 text-center">
|
||||
<div className="text-lg font-bold text-ctp-blue">
|
||||
{formatNumber(data.detail.frequency)}
|
||||
</div>
|
||||
<div className="text-[11px] text-ctp-overlay1 uppercase">Frequency</div>
|
||||
</div>
|
||||
<div className="rounded-lg bg-ctp-surface0 p-3 text-center">
|
||||
<div className="text-sm font-medium text-ctp-green">
|
||||
{formatRelativeDate(epochMsFromDbTimestamp(data.detail.firstSeen))}
|
||||
</div>
|
||||
<div className="text-[11px] text-ctp-overlay1 uppercase">First Seen</div>
|
||||
</div>
|
||||
<div className="rounded-lg bg-ctp-surface0 p-3 text-center">
|
||||
<div className="text-sm font-medium text-ctp-mauve">
|
||||
{formatRelativeDate(epochMsFromDbTimestamp(data.detail.lastSeen))}
|
||||
</div>
|
||||
<div className="text-[11px] text-ctp-overlay1 uppercase">Last Seen</div>
|
||||
</div>
|
||||
<div className="grid grid-cols-2 gap-4 sm:grid-cols-4">
|
||||
<SummaryTile
|
||||
label="Seen"
|
||||
value={formatNumber(data.detail.frequency)}
|
||||
detail={
|
||||
data.detail.formCount > 1
|
||||
? `across ${formatNumber(data.detail.formCount)} forms`
|
||||
: null
|
||||
}
|
||||
/>
|
||||
<SummaryTile
|
||||
label="Frequency rank"
|
||||
value={
|
||||
data.detail.frequencyRank === null
|
||||
? '—'
|
||||
: formatNumber(data.detail.frequencyRank)
|
||||
}
|
||||
detail={null}
|
||||
/>
|
||||
<SummaryTile
|
||||
label="First seen"
|
||||
value={formatRelativeDate(epochMsFromDbTimestamp(data.detail.firstSeen))}
|
||||
detail={null}
|
||||
compact
|
||||
/>
|
||||
<SummaryTile
|
||||
label="Last seen"
|
||||
value={formatRelativeDate(epochMsFromDbTimestamp(data.detail.lastSeen))}
|
||||
detail={null}
|
||||
compact
|
||||
/>
|
||||
</div>
|
||||
|
||||
{data.detail.formCount > 1 && (
|
||||
<section>
|
||||
<h3 className="text-xs font-semibold uppercase tracking-wide text-ctp-overlay1 mb-2">
|
||||
Forms
|
||||
</h3>
|
||||
<div lang="ja" className="flex flex-wrap gap-x-3 gap-y-1 text-sm">
|
||||
{data.detail.forms.map((form) => (
|
||||
<span key={form.wordId} className="text-ctp-text">
|
||||
{form.word}
|
||||
<span lang="en" className="ml-1 text-xs text-ctp-overlay2 tabular-nums">
|
||||
{formatNumber(form.seen)}
|
||||
</span>
|
||||
</span>
|
||||
))}
|
||||
{data.detail.formCount > data.detail.forms.length && (
|
||||
<span lang="en" className="text-xs text-ctp-overlay2">
|
||||
+{formatNumber(data.detail.formCount - data.detail.forms.length)} more
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</section>
|
||||
)}
|
||||
|
||||
{data.animeAppearances.length > 0 && (
|
||||
<section>
|
||||
<h3 className="text-xs font-semibold uppercase tracking-wide text-ctp-overlay1 mb-2">
|
||||
@@ -328,11 +371,13 @@ export function WordDetailPanel({
|
||||
<button
|
||||
key={sw.wordId}
|
||||
type="button"
|
||||
className="inline-flex items-center gap-1 rounded px-2 py-1 text-xs text-ctp-blue bg-ctp-blue/10 transition hover:ring-1 hover:ring-ctp-blue"
|
||||
className="inline-flex items-center gap-1.5 rounded px-2 py-1 text-xs text-ctp-text bg-ctp-surface0 transition hover:ring-1 hover:ring-ctp-blue"
|
||||
onClick={() => onSelectWord?.(sw.wordId)}
|
||||
>
|
||||
{sw.headword}
|
||||
<span className="opacity-60">({formatNumber(sw.frequency)})</span>
|
||||
<span lang="ja">{sw.headword}</span>
|
||||
<span className="text-ctp-overlay2 tabular-nums">
|
||||
{formatNumber(sw.frequency)}
|
||||
</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
@@ -343,19 +388,15 @@ export function WordDetailPanel({
|
||||
<h3 className="text-xs font-semibold uppercase tracking-wide text-ctp-overlay1 mb-2">
|
||||
Example Lines
|
||||
</h3>
|
||||
{!occLoaded && !occLoading && (
|
||||
<button
|
||||
type="button"
|
||||
className="w-full rounded-lg border border-ctp-surface2 bg-ctp-surface0 px-4 py-2 text-sm font-medium text-ctp-text transition hover:border-ctp-blue hover:text-ctp-blue"
|
||||
onClick={handleShowOccurrences}
|
||||
>
|
||||
Load example lines
|
||||
</button>
|
||||
)}
|
||||
{occLoading && (
|
||||
<div className="text-sm text-ctp-overlay2">Loading occurrences...</div>
|
||||
)}
|
||||
{occError && <div className="text-sm text-ctp-red">Error: {occError}</div>}
|
||||
{streamedLinesNotice && (
|
||||
<p className="mb-2 text-xs text-ctp-peach" role="note">
|
||||
{streamedLinesNotice}
|
||||
</p>
|
||||
)}
|
||||
{occLoaded && !occLoading && occurrences.length === 0 && (
|
||||
<div className="text-sm text-ctp-overlay2">
|
||||
No example lines tracked yet. Lines are stored for sessions recorded after the
|
||||
@@ -393,54 +434,49 @@ export function WordDetailPanel({
|
||||
const wordStatus = mineStatus[`${baseKey}-word`];
|
||||
const sentenceStatus = mineStatus[`${baseKey}-sentence`];
|
||||
const audioStatus = mineStatus[`${baseKey}-audio`];
|
||||
if (unavailableReason) {
|
||||
return <MineUnavailableNote reason={unavailableReason} />;
|
||||
}
|
||||
return (
|
||||
<>
|
||||
<button
|
||||
type="button"
|
||||
title={unavailableReason ?? 'Mine this word from video clip'}
|
||||
title="Mine this word from video clip"
|
||||
className="rounded border border-ctp-surface2 px-1.5 py-0.5 text-[10px] font-medium text-ctp-subtext0 transition hover:border-ctp-mauve hover:text-ctp-mauve disabled:cursor-not-allowed disabled:opacity-60"
|
||||
disabled={wordStatus?.loading || !!unavailableReason}
|
||||
disabled={wordStatus?.loading}
|
||||
onClick={() => void handleMine(occ, 'word')}
|
||||
>
|
||||
{wordStatus?.loading
|
||||
? 'Mining...'
|
||||
: wordStatus?.success
|
||||
? 'Mined!'
|
||||
: unavailableReason
|
||||
? 'Unavailable'
|
||||
: 'Mine Word'}
|
||||
: 'Mine Word'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
title={
|
||||
unavailableReason ?? 'Mine this sentence from video clip'
|
||||
}
|
||||
title="Mine this sentence from video clip"
|
||||
className="rounded border border-ctp-surface2 px-1.5 py-0.5 text-[10px] font-medium text-ctp-subtext0 transition hover:border-ctp-green hover:text-ctp-green disabled:cursor-not-allowed disabled:opacity-60"
|
||||
disabled={sentenceStatus?.loading || !!unavailableReason}
|
||||
disabled={sentenceStatus?.loading}
|
||||
onClick={() => void handleMine(occ, 'sentence')}
|
||||
>
|
||||
{sentenceStatus?.loading
|
||||
? 'Mining...'
|
||||
: sentenceStatus?.success
|
||||
? 'Mined!'
|
||||
: unavailableReason
|
||||
? 'Unavailable'
|
||||
: 'Mine Sentence'}
|
||||
: 'Mine Sentence'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
title={unavailableReason ?? 'Mine this line as audio-only card'}
|
||||
title="Mine this line as audio-only card"
|
||||
className="rounded border border-ctp-surface2 px-1.5 py-0.5 text-[10px] font-medium text-ctp-subtext0 transition hover:border-ctp-blue hover:text-ctp-blue disabled:cursor-not-allowed disabled:opacity-60"
|
||||
disabled={audioStatus?.loading || !!unavailableReason}
|
||||
disabled={audioStatus?.loading}
|
||||
onClick={() => void handleMine(occ, 'audio')}
|
||||
>
|
||||
{audioStatus?.loading
|
||||
? 'Mining...'
|
||||
: audioStatus?.success
|
||||
? 'Mined!'
|
||||
: unavailableReason
|
||||
? 'Unavailable'
|
||||
: 'Mine Audio'}
|
||||
: 'Mine Audio'}
|
||||
</button>
|
||||
</>
|
||||
);
|
||||
@@ -455,8 +491,14 @@ export function WordDetailPanel({
|
||||
<div className="mt-1 text-[10px] text-ctp-red">{errors[0]}</div>
|
||||
) : null;
|
||||
})()}
|
||||
<p className="mt-3 rounded-lg bg-ctp-base/70 px-3 py-3 text-sm leading-6 text-ctp-text">
|
||||
{highlightWord(occ.text, [data!.detail.headword, data!.detail.word])}
|
||||
<p
|
||||
lang="ja"
|
||||
className="mt-3 rounded-lg bg-ctp-base/70 px-3 py-3 text-sm leading-6 text-ctp-text"
|
||||
>
|
||||
{highlightWord(occ.text, [
|
||||
data!.detail.headword,
|
||||
...data!.detail.forms.map((form) => form.word),
|
||||
])}
|
||||
</p>
|
||||
</article>
|
||||
))}
|
||||
|
||||
@@ -0,0 +1,463 @@
|
||||
import { Fragment, useEffect, useMemo, useState } from 'react';
|
||||
import { useDictionaryWords } from '../../hooks/useDictionaryWords';
|
||||
import {
|
||||
epochDayToDate,
|
||||
epochMsFromDbTimestamp,
|
||||
formatNumber,
|
||||
localDayFromMs,
|
||||
} from '../../lib/formatters';
|
||||
import {
|
||||
readBooleanPreference,
|
||||
writeBooleanPreference,
|
||||
writeStringPreference,
|
||||
} from '../../lib/preference-storage';
|
||||
import { fullReading } from '../../lib/reading-utils';
|
||||
import type {
|
||||
DictionaryWordRow,
|
||||
StatsDictionaryWordPos,
|
||||
StatsDictionaryWordSort,
|
||||
} from '../../types/stats';
|
||||
import {
|
||||
HIDE_KANA_KEY,
|
||||
HIDE_KNOWN_KEY,
|
||||
WORD_PAGE_SIZE,
|
||||
WORD_VIEW_KEY,
|
||||
WORD_VIEWS,
|
||||
buildWordQuery,
|
||||
defaultSortForView,
|
||||
loadWordView,
|
||||
nextSort,
|
||||
type WordSort,
|
||||
type WordView,
|
||||
} from './word-explorer-query';
|
||||
|
||||
interface WordExplorerProps {
|
||||
hasKnownData: boolean;
|
||||
hideNames: boolean;
|
||||
onHideNamesChange: (hide: boolean) => void;
|
||||
/** Changes when server-side word data may have changed. */
|
||||
refreshKey: number;
|
||||
onOpenWord: (wordId: number) => void;
|
||||
onHideWord: (word: DictionaryWordRow) => void;
|
||||
}
|
||||
|
||||
const VIEW_LABELS: Record<WordView, string> = {
|
||||
worth: 'Worth mining',
|
||||
all: 'All words',
|
||||
titles: 'Across titles',
|
||||
newest: 'Newest',
|
||||
};
|
||||
|
||||
const POS_FILTERS: Array<{ value: StatsDictionaryWordPos; label: string }> = [
|
||||
{ value: 'noun', label: 'Nouns' },
|
||||
{ value: 'verb', label: 'Verbs' },
|
||||
{ value: 'adjective', label: 'Adjectives' },
|
||||
];
|
||||
|
||||
const POS_LABELS: Record<string, string> = {
|
||||
i_adjective: 'adjective',
|
||||
na_adjective: 'adjective',
|
||||
auxiliary_verb: 'auxiliary',
|
||||
bound_auxiliary: 'auxiliary',
|
||||
};
|
||||
|
||||
const SEARCH_DEBOUNCE_MS = 250;
|
||||
|
||||
const COLUMNS: Array<{ id: StatsDictionaryWordSort; label: string; align: 'left' | 'right' }> = [
|
||||
{ id: 'headword', label: 'Word', align: 'left' },
|
||||
{ id: 'rank', label: 'Rank', align: 'right' },
|
||||
{ id: 'seen', label: 'Seen', align: 'right' },
|
||||
{ id: 'titles', label: 'Titles', align: 'right' },
|
||||
{ id: 'firstSeen', label: 'First seen', align: 'right' },
|
||||
];
|
||||
|
||||
function ToggleChip({
|
||||
label,
|
||||
pressed,
|
||||
onToggle,
|
||||
}: {
|
||||
label: string;
|
||||
pressed: boolean;
|
||||
onToggle: () => void;
|
||||
}) {
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
aria-pressed={pressed}
|
||||
onClick={onToggle}
|
||||
className={`rounded-full border px-2.5 py-0.5 text-[11px] transition-colors ${
|
||||
pressed
|
||||
? 'border-ctp-blue/70 text-ctp-text'
|
||||
: 'border-ctp-surface1 text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
function formatFirstSeen(timestamp: number): string {
|
||||
const day = localDayFromMs(epochMsFromDbTimestamp(timestamp));
|
||||
return epochDayToDate(day).toLocaleDateString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
year: 'numeric',
|
||||
});
|
||||
}
|
||||
|
||||
function rowReading(row: DictionaryWordRow): string {
|
||||
const reading = fullReading(row.headword, row.reading);
|
||||
// `fullReading` normalizes katakana, so カレー must not render its own hiragana.
|
||||
return reading === fullReading(row.headword, row.headword) ? '' : reading;
|
||||
}
|
||||
|
||||
function describeResults(view: WordView, searching: boolean, hasKnownData: boolean): string {
|
||||
if (searching) return 'matching words';
|
||||
if (view === 'worth') return hasKnownData ? 'common words not yet mined' : 'common words';
|
||||
if (view === 'titles') return 'words across titles';
|
||||
return 'words';
|
||||
}
|
||||
|
||||
/** The one word list: server-filtered, so it reaches every word you have seen. */
|
||||
export function WordExplorer({
|
||||
hasKnownData,
|
||||
hideNames,
|
||||
onHideNamesChange,
|
||||
refreshKey,
|
||||
onOpenWord,
|
||||
onHideWord,
|
||||
}: WordExplorerProps) {
|
||||
const [view, setView] = useState<WordView>(() => loadWordView());
|
||||
const [sort, setSort] = useState<WordSort>(() => defaultSortForView(loadWordView()));
|
||||
const [searchDraft, setSearchDraft] = useState('');
|
||||
const [search, setSearch] = useState('');
|
||||
const [hideKnown, setHideKnown] = useState(() => readBooleanPreference(HIDE_KNOWN_KEY, true));
|
||||
const [hideKana, setHideKana] = useState(() => readBooleanPreference(HIDE_KANA_KEY, true));
|
||||
const [partOfSpeech, setPartOfSpeech] = useState<StatsDictionaryWordPos | null>(null);
|
||||
const [page, setPage] = useState(0);
|
||||
const [expandedWord, setExpandedWord] = useState<string | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
const timer = setTimeout(() => {
|
||||
setSearch(searchDraft);
|
||||
setPage(0);
|
||||
}, SEARCH_DEBOUNCE_MS);
|
||||
return () => clearTimeout(timer);
|
||||
}, [searchDraft]);
|
||||
|
||||
const query = useMemo(
|
||||
() =>
|
||||
buildWordQuery({
|
||||
view,
|
||||
search,
|
||||
hideKnown: hasKnownData && hideKnown,
|
||||
hideNames,
|
||||
hideKana,
|
||||
partOfSpeech,
|
||||
sort,
|
||||
page,
|
||||
}),
|
||||
[view, search, hasKnownData, hideKnown, hideNames, hideKana, partOfSpeech, sort, page],
|
||||
);
|
||||
const { page: results, loading, error } = useDictionaryWords(query, refreshKey);
|
||||
|
||||
const selectView = (next: WordView) => {
|
||||
setView(next);
|
||||
setSort(defaultSortForView(next));
|
||||
setPage(0);
|
||||
writeStringPreference(WORD_VIEW_KEY, next);
|
||||
};
|
||||
const applyFilter = (update: () => void) => {
|
||||
update();
|
||||
setPage(0);
|
||||
};
|
||||
|
||||
const total = results?.total ?? 0;
|
||||
const rows = results?.rows ?? [];
|
||||
const firstRow = total === 0 ? 0 : page * WORD_PAGE_SIZE + 1;
|
||||
const lastRow = Math.min((page + 1) * WORD_PAGE_SIZE, total);
|
||||
const searching = search.trim() !== '';
|
||||
|
||||
return (
|
||||
<section
|
||||
aria-label="Words"
|
||||
className="rounded-lg border border-ctp-surface1 bg-ctp-surface0 p-4"
|
||||
>
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<div
|
||||
role="group"
|
||||
aria-label="Word list view"
|
||||
className="flex rounded-lg border border-ctp-surface1 p-0.5"
|
||||
>
|
||||
{WORD_VIEWS.map((option) => (
|
||||
<button
|
||||
key={option}
|
||||
type="button"
|
||||
aria-pressed={view === option}
|
||||
onClick={() => selectView(option)}
|
||||
className={`rounded-md px-2.5 py-1 text-xs transition-colors ${
|
||||
view === option
|
||||
? 'bg-ctp-surface2 text-ctp-text'
|
||||
: 'text-ctp-overlay2 hover:text-ctp-subtext0'
|
||||
}`}
|
||||
>
|
||||
{VIEW_LABELS[option]}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<input
|
||||
type="search"
|
||||
aria-label="Search all words"
|
||||
placeholder="Search all words..."
|
||||
value={searchDraft}
|
||||
onChange={(event) => setSearchDraft(event.target.value)}
|
||||
className="min-w-40 flex-1 rounded-lg border border-ctp-surface1 bg-transparent px-2.5 py-1 text-xs text-ctp-text placeholder:text-ctp-overlay2 focus:border-ctp-blue focus:outline-none"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{searching ? null : (
|
||||
<div className="mt-2 flex flex-wrap items-center gap-1.5">
|
||||
{hasKnownData && view !== 'worth' ? (
|
||||
<ToggleChip
|
||||
label="Hide known"
|
||||
pressed={hideKnown}
|
||||
onToggle={() =>
|
||||
applyFilter(() => {
|
||||
setHideKnown(!hideKnown);
|
||||
writeBooleanPreference(HIDE_KNOWN_KEY, !hideKnown);
|
||||
})
|
||||
}
|
||||
/>
|
||||
) : null}
|
||||
<ToggleChip
|
||||
label="Hide names"
|
||||
pressed={hideNames}
|
||||
onToggle={() => applyFilter(() => onHideNamesChange(!hideNames))}
|
||||
/>
|
||||
<ToggleChip
|
||||
label="Hide kana"
|
||||
pressed={hideKana}
|
||||
onToggle={() =>
|
||||
applyFilter(() => {
|
||||
setHideKana(!hideKana);
|
||||
writeBooleanPreference(HIDE_KANA_KEY, !hideKana);
|
||||
})
|
||||
}
|
||||
/>
|
||||
<span aria-hidden="true" className="mx-1 h-3 w-px bg-ctp-surface1" />
|
||||
{POS_FILTERS.map(({ value, label }) => (
|
||||
<ToggleChip
|
||||
key={value}
|
||||
label={label}
|
||||
pressed={partOfSpeech === value}
|
||||
onToggle={() =>
|
||||
applyFilter(() => setPartOfSpeech(partOfSpeech === value ? null : value))
|
||||
}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{error ? (
|
||||
<div role="alert" className="mt-3 text-xs text-ctp-red">
|
||||
Words failed to load: {error}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<div
|
||||
aria-busy={loading}
|
||||
className={`mt-3 overflow-x-auto transition-opacity ${loading && results ? 'opacity-60' : ''}`}
|
||||
>
|
||||
<table className="w-full text-xs">
|
||||
<thead>
|
||||
<tr className="border-b border-ctp-surface1 text-ctp-overlay2">
|
||||
{COLUMNS.map((column) => {
|
||||
const active = sort.sort === column.id;
|
||||
return (
|
||||
<th
|
||||
key={column.id}
|
||||
scope="col"
|
||||
aria-sort={active ? (sort.descending ? 'descending' : 'ascending') : undefined}
|
||||
className={`pb-2 pl-3 font-medium first:pl-0 ${
|
||||
column.align === 'right' ? 'text-right' : 'text-left'
|
||||
}`}
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => {
|
||||
setSort(nextSort(sort, column.id));
|
||||
setPage(0);
|
||||
}}
|
||||
className={`whitespace-nowrap transition-colors hover:text-ctp-text ${
|
||||
active ? 'text-ctp-text' : ''
|
||||
}`}
|
||||
>
|
||||
{column.label}
|
||||
{active ? (sort.descending ? ' ▼' : ' ▲') : ''}
|
||||
</button>
|
||||
</th>
|
||||
);
|
||||
})}
|
||||
<th scope="col" className="pb-2 pl-3">
|
||||
<span className="sr-only">Actions</span>
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{rows.map((row) => {
|
||||
const expanded = expandedWord === row.headword;
|
||||
const reading = rowReading(row);
|
||||
return (
|
||||
<Fragment key={row.headword}>
|
||||
<tr className="border-b border-ctp-surface1/50 transition-colors last:border-0 hover:bg-ctp-surface1/30">
|
||||
<td className="py-1.5">
|
||||
<div className="flex min-w-0 items-baseline gap-2">
|
||||
{row.formCount > 1 ? (
|
||||
<button
|
||||
type="button"
|
||||
aria-expanded={expanded}
|
||||
aria-label={`Show forms of ${row.headword}`}
|
||||
onClick={() => setExpandedWord(expanded ? null : row.headword)}
|
||||
className={`w-3 shrink-0 text-[9px] text-ctp-overlay2 transition-transform hover:text-ctp-text ${
|
||||
expanded ? 'rotate-90' : ''
|
||||
}`}
|
||||
>
|
||||
▶
|
||||
</button>
|
||||
) : (
|
||||
<span aria-hidden="true" className="w-3 shrink-0" />
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
lang="ja"
|
||||
onClick={() => onOpenWord(row.wordId)}
|
||||
className="text-left text-[15px] font-medium text-ctp-text transition-colors hover:text-ctp-blue"
|
||||
>
|
||||
{row.headword}
|
||||
</button>
|
||||
{reading ? (
|
||||
<span lang="ja" className="text-[11px] text-ctp-overlay2">
|
||||
{reading}
|
||||
</span>
|
||||
) : null}
|
||||
<span className="text-[11px] text-ctp-overlay1">
|
||||
{row.partOfSpeech
|
||||
? (POS_LABELS[row.partOfSpeech] ?? row.partOfSpeech.replace(/_/g, ' '))
|
||||
: ''}
|
||||
</span>
|
||||
{row.known ? (
|
||||
<span className="text-[11px] text-ctp-green">Known</span>
|
||||
) : null}
|
||||
{row.isName ? (
|
||||
<span className="text-[11px] text-ctp-overlay1">Name</span>
|
||||
) : null}
|
||||
{row.streamOnly ? (
|
||||
<span
|
||||
title="Every line with this word came from a stream, so it can't be mined yet."
|
||||
className="text-[11px] whitespace-nowrap text-ctp-peach"
|
||||
>
|
||||
Stream only
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
</td>
|
||||
<td className="py-1.5 pl-3 text-right tabular-nums text-ctp-subtext0">
|
||||
{row.rank === null ? '—' : formatNumber(row.rank)}
|
||||
</td>
|
||||
<td className="py-1.5 pl-3 text-right tabular-nums text-ctp-text">
|
||||
<div className="flex items-center justify-end gap-2">
|
||||
{formatNumber(row.seen)}
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className="block h-1 w-12 overflow-hidden rounded-full bg-ctp-surface1/60"
|
||||
>
|
||||
<span
|
||||
className="block h-full rounded-full bg-ctp-blue"
|
||||
style={{
|
||||
width: `${results && results.maxSeen > 0 ? (row.seen / results.maxSeen) * 100 : 0}%`,
|
||||
}}
|
||||
/>
|
||||
</span>
|
||||
</div>
|
||||
</td>
|
||||
<td className="py-1.5 pl-3 text-right tabular-nums text-ctp-subtext0">
|
||||
{formatNumber(row.titles)}
|
||||
</td>
|
||||
<td className="whitespace-nowrap py-1.5 pl-3 text-right tabular-nums text-ctp-overlay2">
|
||||
{formatFirstSeen(row.firstSeen)}
|
||||
</td>
|
||||
<td className="py-1.5 pl-3 text-right">
|
||||
<button
|
||||
type="button"
|
||||
title="Exclude this word from stats. Restore it from Exclusions."
|
||||
aria-label={`Hide ${row.headword}`}
|
||||
onClick={() => onHideWord(row)}
|
||||
className="text-[11px] text-ctp-overlay2 transition-colors hover:text-ctp-text"
|
||||
>
|
||||
Hide
|
||||
</button>
|
||||
</td>
|
||||
</tr>
|
||||
{expanded ? (
|
||||
<tr className="border-b border-ctp-surface1/50">
|
||||
<td colSpan={6} className="pb-2 pl-5">
|
||||
<div lang="ja" className="flex flex-wrap gap-x-3 gap-y-1">
|
||||
{row.forms.map((form) => (
|
||||
<span key={form.wordId} className="text-ctp-subtext0">
|
||||
{form.word}
|
||||
<span lang="en" className="ml-1 tabular-nums text-ctp-overlay2">
|
||||
{formatNumber(form.seen)}
|
||||
</span>
|
||||
</span>
|
||||
))}
|
||||
{row.formCount > row.forms.length ? (
|
||||
<span lang="en" className="text-ctp-overlay2">
|
||||
+{formatNumber(row.formCount - row.forms.length)} more
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
) : null}
|
||||
</Fragment>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
{results && rows.length === 0 ? (
|
||||
<div className="py-6 text-center text-xs text-ctp-overlay2">
|
||||
{searching
|
||||
? 'No words match that search. Hidden words are managed under Exclusions.'
|
||||
: 'No words match these filters.'}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="mt-3 flex items-center justify-between text-[11px] text-ctp-overlay2">
|
||||
<span role="status">
|
||||
{results
|
||||
? `${formatNumber(firstRow)} to ${formatNumber(lastRow)} of ${formatNumber(total)} ${describeResults(view, searching, hasKnownData)}`
|
||||
: 'Loading...'}
|
||||
</span>
|
||||
<span className="flex items-center gap-3">
|
||||
<button
|
||||
type="button"
|
||||
disabled={page === 0}
|
||||
onClick={() => setPage(page - 1)}
|
||||
className="transition-colors hover:text-ctp-text disabled:opacity-40"
|
||||
>
|
||||
Prev
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={lastRow >= total}
|
||||
onClick={() => setPage(page + 1)}
|
||||
className="transition-colors hover:text-ctp-text disabled:opacity-40"
|
||||
>
|
||||
Next
|
||||
</button>
|
||||
</span>
|
||||
</div>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -1,130 +0,0 @@
|
||||
import { useMemo, useState } from 'react';
|
||||
import type { VocabularyEntry } from '../../types/stats';
|
||||
import { PosBadge } from './pos-helpers';
|
||||
|
||||
interface WordListProps {
|
||||
words: VocabularyEntry[];
|
||||
selectedKey?: string | null;
|
||||
onSelectWord?: (word: VocabularyEntry) => void;
|
||||
search?: string;
|
||||
}
|
||||
|
||||
type SortKey = 'frequency' | 'lastSeen' | 'firstSeen';
|
||||
|
||||
function toWordKey(word: VocabularyEntry): string {
|
||||
return `${word.headword}\u0000${word.word}\u0000${word.reading}`;
|
||||
}
|
||||
|
||||
const PAGE_SIZE = 100;
|
||||
|
||||
export function WordList({ words, selectedKey = null, onSelectWord, search = '' }: WordListProps) {
|
||||
const [sortBy, setSortBy] = useState<SortKey>('frequency');
|
||||
const [page, setPage] = useState(0);
|
||||
|
||||
const titleBySort: Record<SortKey, string> = {
|
||||
frequency: 'Most Seen Words',
|
||||
lastSeen: 'Recently Seen Words',
|
||||
firstSeen: 'First Seen Words',
|
||||
};
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
const needle = search.trim().toLowerCase();
|
||||
if (!needle) return words;
|
||||
return words.filter(
|
||||
(w) =>
|
||||
w.headword.toLowerCase().includes(needle) ||
|
||||
w.word.toLowerCase().includes(needle) ||
|
||||
w.reading.toLowerCase().includes(needle),
|
||||
);
|
||||
}, [words, search]);
|
||||
|
||||
const sorted = useMemo(() => {
|
||||
const copy = [...filtered];
|
||||
if (sortBy === 'frequency') copy.sort((a, b) => b.frequency - a.frequency);
|
||||
else if (sortBy === 'lastSeen') copy.sort((a, b) => b.lastSeen - a.lastSeen);
|
||||
else copy.sort((a, b) => b.firstSeen - a.firstSeen);
|
||||
return copy;
|
||||
}, [filtered, sortBy]);
|
||||
|
||||
const totalPages = Math.ceil(sorted.length / PAGE_SIZE);
|
||||
const paged = sorted.slice(page * PAGE_SIZE, (page + 1) * PAGE_SIZE);
|
||||
const maxFreq = words.reduce((max, word) => Math.max(max, word.frequency), 1);
|
||||
|
||||
const getFrequencyColor = (freq: number) => {
|
||||
const ratio = freq / maxFreq;
|
||||
if (ratio > 0.5) return 'text-ctp-blue bg-ctp-blue/10';
|
||||
if (ratio > 0.2) return 'text-ctp-green bg-ctp-green/10';
|
||||
return 'text-ctp-mauve bg-ctp-mauve/10';
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="bg-ctp-surface0 border border-ctp-surface1 rounded-lg p-4">
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
<h3 className="text-sm font-semibold text-ctp-text">
|
||||
{titleBySort[sortBy]}
|
||||
{search && (
|
||||
<span className="ml-2 text-ctp-overlay1 font-normal">({filtered.length} matches)</span>
|
||||
)}
|
||||
</h3>
|
||||
<select
|
||||
value={sortBy}
|
||||
onChange={(e) => {
|
||||
setSortBy(e.target.value as SortKey);
|
||||
setPage(0);
|
||||
}}
|
||||
className="text-xs bg-ctp-surface1 text-ctp-subtext0 border border-ctp-surface2 rounded px-2 py-1"
|
||||
>
|
||||
<option value="frequency">Frequency</option>
|
||||
<option value="lastSeen">Last Seen</option>
|
||||
<option value="firstSeen">First Seen</option>
|
||||
</select>
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{paged.map((w) => (
|
||||
<button
|
||||
type="button"
|
||||
key={toWordKey(w)}
|
||||
className={`inline-flex items-center gap-1 rounded px-2 py-0.5 text-xs transition ${getFrequencyColor(
|
||||
w.frequency,
|
||||
)} ${
|
||||
selectedKey === toWordKey(w)
|
||||
? 'ring-1 ring-ctp-blue ring-offset-1 ring-offset-ctp-surface0'
|
||||
: 'hover:ring-1 hover:ring-ctp-surface2'
|
||||
}`}
|
||||
title={`${w.word} (${w.reading}) — seen ${w.frequency}x`}
|
||||
onClick={() => onSelectWord?.(w)}
|
||||
>
|
||||
{w.headword}
|
||||
{w.partOfSpeech && <PosBadge pos={w.partOfSpeech} />}
|
||||
<span className="opacity-60">({w.frequency})</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
{totalPages > 1 && (
|
||||
<div className="flex items-center justify-center gap-2 mt-3">
|
||||
<button
|
||||
type="button"
|
||||
disabled={page === 0}
|
||||
className="rounded border border-ctp-surface2 px-2 py-0.5 text-xs text-ctp-subtext0 transition hover:border-ctp-blue hover:text-ctp-blue disabled:opacity-40 disabled:cursor-not-allowed"
|
||||
onClick={() => setPage((p) => p - 1)}
|
||||
>
|
||||
Prev
|
||||
</button>
|
||||
<span className="text-xs text-ctp-overlay1">
|
||||
{page + 1} / {totalPages}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
disabled={page >= totalPages - 1}
|
||||
className="rounded border border-ctp-surface2 px-2 py-0.5 text-xs text-ctp-subtext0 transition hover:border-ctp-blue hover:text-ctp-blue disabled:opacity-40 disabled:cursor-not-allowed"
|
||||
onClick={() => setPage((p) => p + 1)}
|
||||
>
|
||||
Next
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export { toWordKey };
|
||||
@@ -0,0 +1,88 @@
|
||||
import { readBooleanPreference, readStringPreference } from '../../lib/preference-storage';
|
||||
import type {
|
||||
StatsDictionaryWordPos,
|
||||
StatsDictionaryWordQuery,
|
||||
StatsDictionaryWordSort,
|
||||
} from '../../types/stats';
|
||||
|
||||
export const WORD_VIEWS = ['worth', 'all', 'titles', 'newest'] as const;
|
||||
export type WordView = (typeof WORD_VIEWS)[number];
|
||||
|
||||
export const WORD_PAGE_SIZE = 25;
|
||||
// Beyond this rank a word is rarely worth prioritizing, and sorting by rank alone
|
||||
// surfaces tokenizer artifacts, so Worth mining ranks by how often you meet a word.
|
||||
const WORTH_MINING_MAX_RANK = 10_000;
|
||||
const ACROSS_TITLES_MIN_TITLES = 2;
|
||||
|
||||
export const HIDE_KNOWN_KEY = 'subminer-stats-vocabulary-hide-known';
|
||||
export const HIDE_KANA_KEY = 'subminer-stats-vocabulary-hide-kana';
|
||||
export const HIDE_NAMES_KEY = 'subminer-stats-vocabulary-hide-names';
|
||||
export const WORD_VIEW_KEY = 'subminer-stats-vocabulary-view';
|
||||
|
||||
export function loadWordView(): WordView {
|
||||
const raw = readStringPreference(WORD_VIEW_KEY);
|
||||
return WORD_VIEWS.find((view) => view === raw) ?? 'worth';
|
||||
}
|
||||
|
||||
export function loadHideNames(): boolean {
|
||||
return readBooleanPreference(HIDE_NAMES_KEY, true);
|
||||
}
|
||||
|
||||
export interface WordSort {
|
||||
sort: StatsDictionaryWordSort;
|
||||
descending: boolean;
|
||||
}
|
||||
|
||||
const VIEW_DEFAULT_SORT: Record<WordView, WordSort> = {
|
||||
worth: { sort: 'seen', descending: true },
|
||||
all: { sort: 'seen', descending: true },
|
||||
titles: { sort: 'titles', descending: true },
|
||||
newest: { sort: 'firstSeen', descending: true },
|
||||
};
|
||||
|
||||
export function defaultSortForView(view: WordView): WordSort {
|
||||
return VIEW_DEFAULT_SORT[view];
|
||||
}
|
||||
|
||||
/** Clicking a sorted column flips it; a new column starts on its most useful end. */
|
||||
export function nextSort(current: WordSort, column: StatsDictionaryWordSort): WordSort {
|
||||
if (current.sort === column) return { sort: column, descending: !current.descending };
|
||||
return { sort: column, descending: column !== 'rank' && column !== 'headword' };
|
||||
}
|
||||
|
||||
export interface WordExplorerState {
|
||||
view: WordView;
|
||||
search: string;
|
||||
hideKnown: boolean;
|
||||
hideNames: boolean;
|
||||
hideKana: boolean;
|
||||
partOfSpeech: StatsDictionaryWordPos | null;
|
||||
sort: WordSort;
|
||||
page: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Turns the controls into a server query. Searching looks through everything, so
|
||||
* typing a word you know or a name still finds it whatever the view and filters say.
|
||||
*/
|
||||
export function buildWordQuery(state: WordExplorerState): StatsDictionaryWordQuery {
|
||||
const paging = {
|
||||
...state.sort,
|
||||
limit: WORD_PAGE_SIZE,
|
||||
offset: state.page * WORD_PAGE_SIZE,
|
||||
};
|
||||
const search = state.search.trim();
|
||||
if (search) {
|
||||
return { ...paging, search, hideKnown: false, hideNames: false, hideKana: false };
|
||||
}
|
||||
const isWorth = state.view === 'worth';
|
||||
return {
|
||||
...paging,
|
||||
hideKnown: isWorth || state.hideKnown,
|
||||
hideNames: state.hideNames,
|
||||
hideKana: state.hideKana,
|
||||
partOfSpeech: state.partOfSpeech ?? undefined,
|
||||
maxRank: isWorth ? WORTH_MINING_MAX_RANK : undefined,
|
||||
minTitles: state.view === 'titles' ? ACROSS_TITLES_MIN_TITLES : undefined,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import { useEffect, useState } from 'react';
|
||||
import { getStatsClient } from './useStatsApi';
|
||||
import type { DictionaryWordPage, StatsDictionaryWordQuery } from '../types/stats';
|
||||
|
||||
/**
|
||||
* Loads one page of the word list. The previous page stays in `page` while the
|
||||
* next one loads, so the table doesn't collapse between filters.
|
||||
*/
|
||||
export function useDictionaryWords(query: StatsDictionaryWordQuery, refreshKey: number) {
|
||||
const [page, setPage] = useState<DictionaryWordPage | null>(null);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const queryKey = JSON.stringify(query);
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
getStatsClient()
|
||||
.getDictionaryWords(query)
|
||||
.then((nextPage) => {
|
||||
if (!cancelled) setPage(nextPage);
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
if (!cancelled) setError(err instanceof Error ? err.message : String(err));
|
||||
})
|
||||
.finally(() => {
|
||||
if (!cancelled) setLoading(false);
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
// `queryKey` stands in for `query`, which is a new object each render.
|
||||
}, [queryKey, refreshKey]);
|
||||
|
||||
return { page, loading, error };
|
||||
}
|
||||
@@ -1,13 +1,20 @@
|
||||
import { useState, useEffect } from 'react';
|
||||
import { getStatsClient } from './useStatsApi';
|
||||
import { SESSION_CHART_EVENT_TYPES } from '../lib/session-events';
|
||||
import type { SessionSummary, SessionTimelinePoint, SessionEvent } from '../types/stats';
|
||||
import type {
|
||||
SessionSummary,
|
||||
SessionTimelinePoint,
|
||||
SessionEvent,
|
||||
StatsDayRange,
|
||||
} from '../types/stats';
|
||||
|
||||
export function toErrorMessage(err: unknown): string {
|
||||
return err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
|
||||
export function useSessions(limit = 50) {
|
||||
export function useSessions(limit = 50, dayRange?: StatsDayRange | null) {
|
||||
const startDay = dayRange?.startDay;
|
||||
const endDay = dayRange?.endDay;
|
||||
const [sessions, setSessions] = useState<SessionSummary[]>([]);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
@@ -18,7 +25,10 @@ export function useSessions(limit = 50) {
|
||||
setError(null);
|
||||
const client = getStatsClient();
|
||||
client
|
||||
.getSessions(limit)
|
||||
.getSessions(
|
||||
limit,
|
||||
startDay !== undefined && endDay !== undefined ? { startDay, endDay } : undefined,
|
||||
)
|
||||
.then((nextSessions) => {
|
||||
if (cancelled) return;
|
||||
setSessions(nextSessions);
|
||||
@@ -34,7 +44,7 @@ export function useSessions(limit = 50) {
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [limit]);
|
||||
}, [limit, startDay, endDay]);
|
||||
|
||||
return { sessions, loading, error };
|
||||
}
|
||||
|
||||
@@ -1,21 +1,38 @@
|
||||
import { useState, useEffect } from 'react';
|
||||
import { getStatsClient } from './useStatsApi';
|
||||
import type { TrendsDashboardData } from '../types/stats';
|
||||
import type {
|
||||
StatsTrendGroupBy,
|
||||
StatsTrendPresetRange,
|
||||
StatsTrendRange,
|
||||
TrendsDashboardData,
|
||||
} from '../types/stats';
|
||||
|
||||
export type TimeRange = '7d' | '30d' | '90d' | '365d' | 'all';
|
||||
export type GroupBy = 'day' | 'month';
|
||||
export type TimeRange = StatsTrendRange;
|
||||
export type PresetTimeRange = StatsTrendPresetRange;
|
||||
export type GroupBy = StatsTrendGroupBy;
|
||||
|
||||
export function useTrends(range: TimeRange, groupBy: GroupBy, fillEmpty = true) {
|
||||
/**
|
||||
* Loads the dashboard for a selection. The previous payload stays in `data` while
|
||||
* a new one loads, so controls and charts stay on screen; `refreshing` marks that
|
||||
* state and `loading` is only true before the first payload arrives.
|
||||
*/
|
||||
export function useTrends(
|
||||
range: TimeRange,
|
||||
groupBy: GroupBy,
|
||||
fillEmpty = true,
|
||||
title: string | null = null,
|
||||
) {
|
||||
const [data, setData] = useState<TrendsDashboardData | null>(null);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [refreshing, setRefreshing] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const rangeKey = typeof range === 'string' ? range : `${range.startDay}:${range.endDay}`;
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
setLoading(true);
|
||||
setRefreshing(true);
|
||||
setError(null);
|
||||
getStatsClient()
|
||||
.getTrendsDashboard(range, groupBy, fillEmpty)
|
||||
.getTrendsDashboard(range, groupBy, fillEmpty, title)
|
||||
.then((nextData) => {
|
||||
if (cancelled) return;
|
||||
setData(nextData);
|
||||
@@ -26,12 +43,13 @@ export function useTrends(range: TimeRange, groupBy: GroupBy, fillEmpty = true)
|
||||
})
|
||||
.finally(() => {
|
||||
if (cancelled) return;
|
||||
setLoading(false);
|
||||
setRefreshing(false);
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [range, groupBy, fillEmpty]);
|
||||
// `rangeKey` stands in for `range`, which is a new object for each custom selection.
|
||||
}, [rangeKey, groupBy, fillEmpty, title]);
|
||||
|
||||
return { data, loading, error };
|
||||
return { data, loading: refreshing && data === null, refreshing, error };
|
||||
}
|
||||
@@ -66,8 +66,6 @@ function summaryFixture(): StatsVocabularySummary {
|
||||
function chartsFixture(overrides: Partial<StatsVocabularyCharts> = {}): StatsVocabularyCharts {
|
||||
return {
|
||||
ready: true,
|
||||
topWords: [{ wordId: 1, headword: '猫', frequency: 5 }],
|
||||
topWordsWithoutNames: [{ wordId: 1, headword: '猫', frequency: 5 }],
|
||||
newWordsTimeline: [{ epochDay: 20_000, wordCount: 4 }],
|
||||
newWordsTimelineWithoutNames: [{ epochDay: 20_000, wordCount: 4 }],
|
||||
...overrides,
|
||||
@@ -147,14 +145,14 @@ function stubVocabularyClient(overrides: {
|
||||
getVocabularyCharts: () => Promise<StatsVocabularyCharts>;
|
||||
}): () => void {
|
||||
const original = {
|
||||
getVocabulary: apiClient.getVocabulary,
|
||||
getVocabularyCoverage: apiClient.getVocabularyCoverage,
|
||||
getKanji: apiClient.getKanji,
|
||||
getKnownWords: apiClient.getKnownWords,
|
||||
getVocabularySummary: apiClient.getVocabularySummary,
|
||||
getVocabularyCharts: apiClient.getVocabularyCharts,
|
||||
setExcludedWords: apiClient.setExcludedWords,
|
||||
};
|
||||
apiClient.getVocabulary = async () => [];
|
||||
apiClient.getVocabularyCoverage = async () => [];
|
||||
apiClient.getKanji = async () => [];
|
||||
apiClient.getKnownWords = async () => [];
|
||||
apiClient.setExcludedWords = async () => {};
|
||||
|
||||
@@ -2,10 +2,10 @@ import { useState, useEffect, useCallback } from 'react';
|
||||
import { getStatsClient } from './useStatsApi';
|
||||
import { subscribeExcludedWordsServerSync } from './useExcludedWords';
|
||||
import type {
|
||||
VocabularyEntry,
|
||||
KanjiEntry,
|
||||
StatsVocabularyCharts,
|
||||
StatsVocabularySummary,
|
||||
VocabularyCoverageBand,
|
||||
} from '../types/stats';
|
||||
|
||||
const AGGREGATE_RETRY_BASE_MS = 1_000;
|
||||
@@ -15,17 +15,19 @@ const CHART_BACKFILL_POLL_MS = 1_000;
|
||||
const CHART_BACKFILL_SLOW_POLL_MS = 5_000;
|
||||
const CHART_BACKFILL_FAST_POLLS = 30;
|
||||
const CHART_BACKFILL_POLL_LIMIT = 60;
|
||||
// Every kanji seen. The list stays small because the writing system does.
|
||||
const KANJI_LIMIT = 10_000;
|
||||
|
||||
function aggregateRetryDelayMs(attempt: number): number {
|
||||
return Math.min(AGGREGATE_RETRY_BASE_MS * 2 ** attempt, AGGREGATE_RETRY_MAX_MS);
|
||||
}
|
||||
|
||||
export function useVocabulary() {
|
||||
const [words, setWords] = useState<VocabularyEntry[]>([]);
|
||||
const [kanji, setKanji] = useState<KanjiEntry[]>([]);
|
||||
const [knownWords, setKnownWords] = useState<Set<string>>(new Set());
|
||||
const [summary, setSummary] = useState<StatsVocabularySummary | null>(null);
|
||||
const [charts, setCharts] = useState<StatsVocabularyCharts | null>(null);
|
||||
const [coverage, setCoverage] = useState<VocabularyCoverageBand[]>([]);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [aggregatesError, setAggregatesError] = useState<string | null>(null);
|
||||
@@ -47,17 +49,11 @@ export function useVocabulary() {
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
const client = getStatsClient();
|
||||
Promise.allSettled([client.getVocabulary(500), client.getKanji(200), client.getKnownWords()])
|
||||
.then(([wordsResult, kanjiResult, knownResult]) => {
|
||||
Promise.allSettled([client.getKanji(KANJI_LIMIT), client.getKnownWords()])
|
||||
.then(([kanjiResult, knownResult]) => {
|
||||
if (cancelled) return;
|
||||
const errors: string[] = [];
|
||||
|
||||
if (wordsResult.status === 'fulfilled') {
|
||||
setWords(wordsResult.value);
|
||||
} else {
|
||||
errors.push(wordsResult.reason.message);
|
||||
}
|
||||
|
||||
if (kanjiResult.status === 'fulfilled') {
|
||||
setKanji(kanjiResult.value);
|
||||
} else {
|
||||
@@ -145,6 +141,15 @@ export function useVocabulary() {
|
||||
});
|
||||
};
|
||||
|
||||
void client
|
||||
.getVocabularyCoverage()
|
||||
.then((nextCoverage) => {
|
||||
if (!cancelled) setCoverage(nextCoverage);
|
||||
})
|
||||
.catch((coverageError: unknown) => {
|
||||
// Coverage is a supplement, so losing it shouldn't flag the whole page.
|
||||
console.error('Failed to load vocabulary coverage', coverageError);
|
||||
});
|
||||
loadSummary(0);
|
||||
loadCharts(0, 0);
|
||||
return () => {
|
||||
@@ -154,11 +159,13 @@ export function useVocabulary() {
|
||||
}, [aggregatesToken]);
|
||||
|
||||
return {
|
||||
words,
|
||||
kanji,
|
||||
knownWords,
|
||||
summary,
|
||||
charts,
|
||||
coverage,
|
||||
/** Changes whenever server-side word data may have changed. */
|
||||
dataRevision: aggregatesToken,
|
||||
loading,
|
||||
error,
|
||||
aggregatesError,
|
||||
|
||||
@@ -89,6 +89,67 @@ test('deleteSession throws when the stats API delete request fails', async () =>
|
||||
);
|
||||
});
|
||||
|
||||
test('getTrendsDashboard encodes a custom range and title filter', async () => {
|
||||
await withFetchStub(
|
||||
() => jsonResponse({}),
|
||||
async (requests) => {
|
||||
await apiClient.getTrendsDashboard(
|
||||
{ startDay: 20_000, endDay: 20_010 },
|
||||
'week',
|
||||
true,
|
||||
'A & B',
|
||||
);
|
||||
assert.equal(
|
||||
requests[0]?.url,
|
||||
`${BASE_URL}/api/stats/trends/dashboard?range=custom&fromDay=20000&toDay=20010&groupBy=week&fillEmpty=true&title=A+%26+B`,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('getDictionaryWords sends only the filters in use', async () => {
|
||||
await withFetchStub(
|
||||
() => jsonResponse({ rows: [], total: 0, maxSeen: 0 }),
|
||||
async (requests) => {
|
||||
await apiClient.getDictionaryWords({
|
||||
search: '猫',
|
||||
sort: 'rank',
|
||||
descending: false,
|
||||
hideKnown: true,
|
||||
hideNames: false,
|
||||
hideKana: false,
|
||||
maxRank: 10_000,
|
||||
limit: 25,
|
||||
offset: 50,
|
||||
});
|
||||
const url = new URL(requests[0]!.url, 'http://stats.local');
|
||||
assert.equal(url.pathname, '/api/stats/vocabulary/words');
|
||||
assert.deepEqual(Object.fromEntries(url.searchParams), {
|
||||
sort: 'rank',
|
||||
order: 'asc',
|
||||
limit: '25',
|
||||
offset: '50',
|
||||
search: '猫',
|
||||
hideKnown: 'true',
|
||||
maxRank: '10000',
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('getSessions narrows to a span of days when given one', async () => {
|
||||
await withFetchStub(
|
||||
() => jsonResponse([]),
|
||||
async (requests) => {
|
||||
await apiClient.getSessions(500, { startDay: 20_000, endDay: 20_006 });
|
||||
assert.equal(
|
||||
requests[0]?.url,
|
||||
`${BASE_URL}/api/stats/sessions?limit=500&fromDay=20000&toDay=20006`,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('getTrendsDashboard accepts 365d range and builds correct URL', async () => {
|
||||
await withFetchStub(
|
||||
() =>
|
||||
|
||||
@@ -16,6 +16,8 @@ import type {
|
||||
StatsMoveVideoRequest,
|
||||
StatsMoveVideoResponse,
|
||||
StatsTmdbAssignment,
|
||||
StatsDayRange,
|
||||
StatsDictionaryWordQuery,
|
||||
StatsTrendGroupBy,
|
||||
StatsTrendRange,
|
||||
StatsVideoWatchedRequest,
|
||||
@@ -67,7 +69,13 @@ export const apiClient = {
|
||||
fetchJson('dailyRollups', `/api/stats/daily-rollups?limit=${limit}`),
|
||||
getMonthlyRollups: (limit = 24) =>
|
||||
fetchJson('monthlyRollups', `/api/stats/monthly-rollups?limit=${limit}`),
|
||||
getSessions: (limit = 50) => fetchJson('sessions', `/api/stats/sessions?limit=${limit}`),
|
||||
getSessions: (limit = 50, dayRange?: StatsDayRange) =>
|
||||
fetchJson(
|
||||
'sessions',
|
||||
dayRange
|
||||
? `/api/stats/sessions?limit=${limit}&fromDay=${dayRange.startDay}&toDay=${dayRange.endDay}`
|
||||
: `/api/stats/sessions?limit=${limit}`,
|
||||
),
|
||||
getSessionTimeline: (id: number, limit?: number) =>
|
||||
fetchJson(
|
||||
'sessionTimeline',
|
||||
@@ -87,6 +95,23 @@ export const apiClient = {
|
||||
getVocabulary: (limit = 100) => fetchJson('vocabulary', `/api/stats/vocabulary?limit=${limit}`),
|
||||
getVocabularySummary: () => fetchJson('vocabularySummary', '/api/stats/vocabulary/summary'),
|
||||
getVocabularyCharts: () => fetchJson('vocabularyCharts', '/api/stats/vocabulary/charts'),
|
||||
getDictionaryWords: (query: StatsDictionaryWordQuery) => {
|
||||
const params = new URLSearchParams({
|
||||
sort: query.sort,
|
||||
order: query.descending ? 'desc' : 'asc',
|
||||
limit: String(query.limit),
|
||||
offset: String(query.offset),
|
||||
});
|
||||
if (query.search) params.set('search', query.search);
|
||||
if (query.hideKnown) params.set('hideKnown', 'true');
|
||||
if (query.hideNames) params.set('hideNames', 'true');
|
||||
if (query.hideKana) params.set('hideKana', 'true');
|
||||
if (query.partOfSpeech) params.set('pos', query.partOfSpeech);
|
||||
if (query.maxRank !== undefined) params.set('maxRank', String(query.maxRank));
|
||||
if (query.minTitles !== undefined) params.set('minTitles', String(query.minTitles));
|
||||
return fetchJson('dictionaryWords', `/api/stats/vocabulary/words?${params.toString()}`);
|
||||
},
|
||||
getVocabularyCoverage: () => fetchJson('vocabularyCoverage', '/api/stats/vocabulary/coverage'),
|
||||
getExcludedWords: () => fetchJson('excludedWords', '/api/stats/excluded-words'),
|
||||
setExcludedWords: async (words: StatsExcludedWord[]): Promise<void> => {
|
||||
await fetchResponse('/api/stats/excluded-words', {
|
||||
@@ -108,10 +133,17 @@ export const apiClient = {
|
||||
});
|
||||
return res.json() as Promise<StatsDuplicateLineCleanupResult>;
|
||||
},
|
||||
getWordOccurrences: (headword: string, word: string, reading: string, limit = 50, offset = 0) =>
|
||||
getWordOccurrences: (
|
||||
headword: string,
|
||||
word: string,
|
||||
reading: string,
|
||||
limit = 50,
|
||||
offset = 0,
|
||||
allForms = false,
|
||||
) =>
|
||||
fetchJson(
|
||||
'wordOccurrences',
|
||||
`/api/stats/vocabulary/occurrences?headword=${encodeURIComponent(headword)}&word=${encodeURIComponent(word)}&reading=${encodeURIComponent(reading)}&limit=${limit}&offset=${offset}`,
|
||||
`/api/stats/vocabulary/occurrences?headword=${encodeURIComponent(headword)}&word=${encodeURIComponent(word)}&reading=${encodeURIComponent(reading)}&limit=${limit}&offset=${offset}${allForms ? '&allForms=true' : ''}`,
|
||||
),
|
||||
searchSentences: (query: string, limit = 50, searchByHeadword = true) =>
|
||||
fetchJson(
|
||||
@@ -159,11 +191,26 @@ export const apiClient = {
|
||||
fetchJson('newAnimePerDay', `/api/stats/trends/new-anime-per-day?limit=${limit}`),
|
||||
getWatchTimePerAnime: (limit = 90) =>
|
||||
fetchJson('watchTimePerAnime', `/api/stats/trends/watch-time-per-anime?limit=${limit}`),
|
||||
getTrendsDashboard: (range: StatsTrendRange, groupBy: StatsTrendGroupBy, fillEmpty = true) =>
|
||||
fetchJson(
|
||||
'trendsDashboard',
|
||||
`/api/stats/trends/dashboard?range=${encodeURIComponent(range)}&groupBy=${encodeURIComponent(groupBy)}&fillEmpty=${fillEmpty ? 'true' : 'false'}`,
|
||||
),
|
||||
getTrendsDashboard: (
|
||||
range: StatsTrendRange,
|
||||
groupBy: StatsTrendGroupBy,
|
||||
fillEmpty = true,
|
||||
title: string | null = null,
|
||||
) => {
|
||||
const params = new URLSearchParams(
|
||||
typeof range === 'string'
|
||||
? { range }
|
||||
: {
|
||||
range: 'custom',
|
||||
fromDay: String(range.startDay),
|
||||
toDay: String(range.endDay),
|
||||
},
|
||||
);
|
||||
params.set('groupBy', groupBy);
|
||||
params.set('fillEmpty', fillEmpty ? 'true' : 'false');
|
||||
if (title !== null) params.set('title', title);
|
||||
return fetchJson('trendsDashboard', `/api/stats/trends/dashboard?${params.toString()}`);
|
||||
},
|
||||
getWordDetail: (wordId: number) =>
|
||||
fetchJson('wordDetail', `/api/stats/vocabulary/${wordId}/detail`),
|
||||
getKanjiDetail: (kanjiId: number) =>
|
||||
|
||||
@@ -5,6 +5,8 @@ import {
|
||||
epochDayToDate,
|
||||
epochMsFromDbTimestamp,
|
||||
formatClockDuration,
|
||||
isoDateToEpochDay,
|
||||
localDayFromMs,
|
||||
formatRelativeDate,
|
||||
formatSessionDayLabel,
|
||||
} from './formatters';
|
||||
@@ -160,3 +162,20 @@ test('formatClockDuration keeps seconds for sub-hour spans', () => {
|
||||
assert.equal(formatClockDuration(372_000), '6m 12s');
|
||||
assert.equal(formatClockDuration(3_780_000), '1h 3m');
|
||||
});
|
||||
|
||||
test('isoDateToEpochDay reads date input values and rejects dates before 1970', () => {
|
||||
assert.equal(isoDateToEpochDay('1970-01-02'), 1);
|
||||
assert.equal(isoDateToEpochDay('2026-03-01'), 20_513);
|
||||
assert.equal(isoDateToEpochDay('1969-12-31'), null);
|
||||
// Date.UTC would read this year as 1999.
|
||||
assert.equal(isoDateToEpochDay('0099-01-01'), null);
|
||||
assert.equal(isoDateToEpochDay(''), null);
|
||||
});
|
||||
|
||||
test('localDayFromMs numbers the local calendar day in every time zone', () => {
|
||||
const justAfterMidnight = new Date(2026, 0, 2, 0, 30).getTime();
|
||||
const lateEvening = new Date(2026, 0, 2, 23, 30).getTime();
|
||||
const january2 = Date.UTC(2026, 0, 2) / 86_400_000;
|
||||
assert.equal(localDayFromMs(justAfterMidnight), january2);
|
||||
assert.equal(localDayFromMs(lateEvening), january2);
|
||||
});
|
||||
@@ -10,6 +10,21 @@ export function formatNumber(n: number): string {
|
||||
return n.toLocaleString();
|
||||
}
|
||||
|
||||
const COMPACT_NUMBER = new Intl.NumberFormat(undefined, {
|
||||
notation: 'compact',
|
||||
maximumFractionDigits: 1,
|
||||
});
|
||||
|
||||
/** Short form for tight spaces like chart axes: 24000 becomes 24K. */
|
||||
export function formatCompactNumber(n: number): string {
|
||||
return COMPACT_NUMBER.format(n);
|
||||
}
|
||||
|
||||
/** Watch time for headline numbers: minutes below an hour, then hours to one decimal. */
|
||||
export function formatHours(minutes: number): string {
|
||||
return minutes < 60 ? `${Math.round(minutes)}m` : `${(minutes / 60).toFixed(1)}h`;
|
||||
}
|
||||
|
||||
export function formatPercent(ratio: number | null): string {
|
||||
if (ratio == null) return '\u2014';
|
||||
return `${Math.round(ratio * 100)}%`;
|
||||
@@ -42,10 +57,25 @@ export function epochDayToDate(epochDay: number): Date {
|
||||
return new Date(utcDate.getUTCFullYear(), utcDate.getUTCMonth(), utcDate.getUTCDate());
|
||||
}
|
||||
|
||||
/** `YYYY-MM-DD` for a date input. */
|
||||
export function epochDayToIsoDate(epochDay: number): string {
|
||||
return new Date(epochDay * 86_400_000).toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
/** Epoch day for a date input value, or null when the input is empty, invalid, or before 1970. */
|
||||
export function isoDateToEpochDay(value: string): number | null {
|
||||
const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(value);
|
||||
if (!match) return null;
|
||||
const year = Number(match[1]);
|
||||
if (year < 1970) return null;
|
||||
const ms = Date.UTC(year, Number(match[2]) - 1, Number(match[3]));
|
||||
return Number.isNaN(ms) ? null : ms / 86_400_000;
|
||||
}
|
||||
|
||||
/** The local calendar day as an epoch day, the same numbering the stats server uses. */
|
||||
export function localDayFromMs(ms: number): number {
|
||||
const d = new Date(ms);
|
||||
const localMidnight = new Date(d.getFullYear(), d.getMonth(), d.getDate()).getTime();
|
||||
return Math.floor(localMidnight / 86_400_000);
|
||||
return Date.UTC(d.getFullYear(), d.getMonth(), d.getDate()) / 86_400_000;
|
||||
}
|
||||
|
||||
export function todayLocalDay(): number {
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
const KANA_ONLY_TEXT = /^[\p{Script=Hiragana}\p{Script=Katakana}\u30fc\u309d\u309e\u30fd\u30fe]+$/u;
|
||||
|
||||
export function isKanaOnlyTokenText(text: string): boolean {
|
||||
const trimmed = text.trim();
|
||||
return trimmed.length > 0 && KANA_ONLY_TEXT.test(trimmed);
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {
|
||||
buildStatsMineCardParams,
|
||||
describeStreamedLines,
|
||||
getStatsMineCardError,
|
||||
getStatsMineCardUnavailableReason,
|
||||
} from './mining';
|
||||
@@ -67,6 +68,24 @@ test('buildStatsMineCardParams returns null when stored timing has no positive d
|
||||
);
|
||||
});
|
||||
|
||||
test('getStatsMineCardUnavailableReason says a line without a file came from a stream', () => {
|
||||
assert.match(getStatsMineCardUnavailableReason(makeResult({ sourcePath: null })) ?? '', /stream/);
|
||||
assert.equal(getStatsMineCardUnavailableReason(makeResult()), null);
|
||||
});
|
||||
|
||||
for (const c of [
|
||||
{ name: 'no streamed lines', lines: 10, streamed: 0, expected: null },
|
||||
{ name: 'some streamed lines', lines: 2_863, streamed: 935, match: /935 of 2,863 lines/ },
|
||||
{ name: 'only streamed lines', lines: 4, streamed: 4, match: /All 4 lines.*can't be mined yet/ },
|
||||
{ name: 'a single streamed line', lines: 1, streamed: 1, match: /only line came from a stream/ },
|
||||
]) {
|
||||
test(`describeStreamedLines handles ${c.name}`, () => {
|
||||
const text = describeStreamedLines(c.lines, c.streamed);
|
||||
if (c.match) assert.match(text ?? '', c.match);
|
||||
else assert.equal(text, null);
|
||||
});
|
||||
}
|
||||
|
||||
test('getStatsMineCardError surfaces partial media failures', () => {
|
||||
assert.equal(
|
||||
getStatsMineCardError({ noteId: 1, errors: ['audio: ffmpeg failed'] }),
|
||||
|
||||
+17
-1
@@ -7,11 +7,27 @@ import type {
|
||||
|
||||
export type { StatsMineCardParams, StatsMineCardResponse, StatsMineMode } from '../types/stats';
|
||||
|
||||
// Cards cut their audio and image from the video file. Only streamed videos have no
|
||||
// local path, so a missing path means the line came from a stream.
|
||||
const STREAMED_LINE_REASON =
|
||||
"Can't mine: from a stream, so there's no video file to cut clips from.";
|
||||
|
||||
/** Explains how a word's lines affect mining, or null when every line can be mined. */
|
||||
export function describeStreamedLines(lineCount: number, streamLineCount: number): string | null {
|
||||
if (streamLineCount === 0) return null;
|
||||
if (streamLineCount >= lineCount) {
|
||||
return lineCount === 1
|
||||
? "This word's only line came from a stream, so it can't be mined yet."
|
||||
: `All ${lineCount.toLocaleString()} lines with this word came from streams, so it can't be mined yet.`;
|
||||
}
|
||||
return `${streamLineCount.toLocaleString()} of ${lineCount.toLocaleString()} lines with this word came from streams and can't be mined.`;
|
||||
}
|
||||
|
||||
export function getStatsMineCardUnavailableReason(
|
||||
result: Pick<SentenceSearchResult, 'sourcePath' | 'segmentStartMs' | 'segmentEndMs'>,
|
||||
): string | null {
|
||||
if (!result.sourcePath) {
|
||||
return 'This source has no local file path.';
|
||||
return STREAMED_LINE_REASON;
|
||||
}
|
||||
if (result.segmentStartMs == null || result.segmentEndMs == null) {
|
||||
return 'This line is missing segment timing.';
|
||||
|
||||
@@ -24,3 +24,19 @@ export function writeBooleanPreference(key: string, value: boolean): void {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
|
||||
export function readStringPreference(key: string): string | null {
|
||||
try {
|
||||
return getPreferenceStorage()?.getItem(key) ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function writeStringPreference(key: string, value: string): void {
|
||||
try {
|
||||
getPreferenceStorage()?.setItem(key, value);
|
||||
} catch {
|
||||
// Storage can be blocked in private/restricted contexts; keep the in-memory choice.
|
||||
}
|
||||
}
|
||||
@@ -61,7 +61,7 @@ test('getSentenceSearchMineAvailability disables every mining mode without sourc
|
||||
canMineSentence: false,
|
||||
canMineWordAudio: false,
|
||||
exactMatch: true,
|
||||
unavailableReason: 'This source has no local file path.',
|
||||
unavailableReason: "Can't mine: from a stream, so there's no video file to cut clips from.",
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -5,8 +5,11 @@ import {
|
||||
createInitialStatsView,
|
||||
getSessionNavigationTarget,
|
||||
navigateToAnime,
|
||||
navigateToSessionsDays,
|
||||
openAnimeEpisodeDetail,
|
||||
openOverviewMediaDetail,
|
||||
openSessionsMediaDetail,
|
||||
openTrendsMediaDetail,
|
||||
switchTab,
|
||||
type StatsViewState,
|
||||
} from './stats-navigation';
|
||||
@@ -18,6 +21,7 @@ test('openAnimeEpisodeDetail opens dedicated media detail from anime context', (
|
||||
activeTab: 'anime',
|
||||
selectedAnimeId: 42,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId: 7,
|
||||
initialSessionId: null,
|
||||
@@ -36,6 +40,7 @@ test('closeMediaDetail returns to originating anime detail state', () => {
|
||||
activeTab: 'anime',
|
||||
selectedAnimeId: 42,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
} satisfies StatsViewState);
|
||||
});
|
||||
@@ -45,6 +50,7 @@ test('openOverviewMediaDetail opens dedicated media detail from overview context
|
||||
activeTab: 'overview',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId: 9,
|
||||
initialSessionId: null,
|
||||
@@ -68,6 +74,7 @@ test('switchTab clears dedicated media detail state', () => {
|
||||
activeTab: 'sessions',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
} satisfies StatsViewState);
|
||||
});
|
||||
@@ -92,6 +99,7 @@ test('openOverviewMediaDetail can carry a target session id for auto-expansion',
|
||||
activeTab: 'overview',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId: 9,
|
||||
initialSessionId: 33,
|
||||
@@ -101,3 +109,25 @@ test('openOverviewMediaDetail can carry a target session id for auto-expansion',
|
||||
},
|
||||
} satisfies StatsViewState);
|
||||
});
|
||||
|
||||
const OCT_5 = { startDay: 20_366, endDay: 20_366, label: 'Oct 5' };
|
||||
|
||||
test('navigateToSessionsDays opens Sessions narrowed to those days', () => {
|
||||
const state = navigateToSessionsDays(createInitialStatsView(), OCT_5);
|
||||
assert.equal(state.activeTab, 'sessions');
|
||||
assert.deepEqual(state.sessionsDayFilter, OCT_5);
|
||||
});
|
||||
|
||||
test('the sessions day filter survives a media detail round trip but not a tab switch', () => {
|
||||
const filtered = navigateToSessionsDays(createInitialStatsView(), OCT_5);
|
||||
const back = closeMediaDetail(openSessionsMediaDetail(filtered, 3));
|
||||
assert.equal(back.activeTab, 'sessions');
|
||||
assert.deepEqual(back.sessionsDayFilter, OCT_5);
|
||||
assert.equal(switchTab(filtered, 'sessions').sessionsDayFilter, null);
|
||||
});
|
||||
|
||||
test('media detail opened from Trends returns to Trends', () => {
|
||||
const state = closeMediaDetail(openTrendsMediaDetail(createInitialStatsView(), 3));
|
||||
assert.equal(state.activeTab, 'trends');
|
||||
assert.equal(state.mediaDetail, null);
|
||||
});
|
||||
@@ -1,10 +1,11 @@
|
||||
import type { SessionSummary } from '../types/stats';
|
||||
import type { SessionSummary, StatsDayRange } from '../types/stats';
|
||||
import type { TabId } from '../components/layout/TabBar';
|
||||
|
||||
export type MediaDetailOrigin =
|
||||
| { type: 'anime'; animeId: number }
|
||||
| { type: 'overview' }
|
||||
| { type: 'sessions' };
|
||||
| { type: 'sessions' }
|
||||
| { type: 'trends' };
|
||||
|
||||
export interface MediaDetailState {
|
||||
videoId: number;
|
||||
@@ -12,10 +13,16 @@ export interface MediaDetailState {
|
||||
origin: MediaDetailOrigin;
|
||||
}
|
||||
|
||||
/** Narrows the Sessions tab to a span of days, set when arriving from a Trends chart. */
|
||||
export interface SessionsDayFilter extends StatsDayRange {
|
||||
label: string;
|
||||
}
|
||||
|
||||
export interface StatsViewState {
|
||||
activeTab: TabId;
|
||||
selectedAnimeId: number | null;
|
||||
focusedSessionId: number | null;
|
||||
sessionsDayFilter: SessionsDayFilter | null;
|
||||
mediaDetail: MediaDetailState | null;
|
||||
}
|
||||
|
||||
@@ -24,6 +31,7 @@ export function createInitialStatsView(): StatsViewState {
|
||||
activeTab: 'overview',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
@@ -33,6 +41,7 @@ export function switchTab(state: StatsViewState, tabId: TabId): StatsViewState {
|
||||
activeTab: tabId,
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: tabId === 'sessions' ? state.focusedSessionId : null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
@@ -51,6 +60,20 @@ export function navigateToSession(state: StatsViewState, sessionId: number): Sta
|
||||
...state,
|
||||
activeTab: 'sessions',
|
||||
focusedSessionId: sessionId,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
|
||||
export function navigateToSessionsDays(
|
||||
state: StatsViewState,
|
||||
sessionsDayFilter: SessionsDayFilter,
|
||||
): StatsViewState {
|
||||
return {
|
||||
...state,
|
||||
activeTab: 'sessions',
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
@@ -65,6 +88,7 @@ export function openAnimeEpisodeDetail(
|
||||
activeTab: 'anime',
|
||||
selectedAnimeId: animeId,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId,
|
||||
initialSessionId: sessionId,
|
||||
@@ -85,6 +109,7 @@ export function openOverviewMediaDetail(
|
||||
activeTab: 'overview',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId,
|
||||
initialSessionId: sessionId,
|
||||
@@ -100,6 +125,7 @@ export function openSessionsMediaDetail(state: StatsViewState, videoId: number):
|
||||
activeTab: 'sessions',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: state.sessionsDayFilter,
|
||||
mediaDetail: {
|
||||
videoId,
|
||||
initialSessionId: null,
|
||||
@@ -110,33 +136,43 @@ export function openSessionsMediaDetail(state: StatsViewState, videoId: number):
|
||||
};
|
||||
}
|
||||
|
||||
export function openTrendsMediaDetail(_state: StatsViewState, videoId: number): StatsViewState {
|
||||
return {
|
||||
activeTab: 'trends',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: {
|
||||
videoId,
|
||||
initialSessionId: null,
|
||||
origin: {
|
||||
type: 'trends',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function closeMediaDetail(state: StatsViewState): StatsViewState {
|
||||
if (!state.mediaDetail) {
|
||||
return state;
|
||||
}
|
||||
|
||||
if (state.mediaDetail.origin.type === 'overview') {
|
||||
const { origin } = state.mediaDetail;
|
||||
if (origin.type === 'anime') {
|
||||
return {
|
||||
activeTab: 'overview',
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
|
||||
if (state.mediaDetail.origin.type === 'sessions') {
|
||||
return {
|
||||
activeTab: 'sessions',
|
||||
selectedAnimeId: null,
|
||||
activeTab: 'anime',
|
||||
selectedAnimeId: origin.animeId,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
activeTab: 'anime',
|
||||
selectedAnimeId: state.mediaDetail.origin.animeId,
|
||||
activeTab: origin.type,
|
||||
selectedAnimeId: null,
|
||||
focusedSessionId: null,
|
||||
sessionsDayFilter: origin.type === 'sessions' ? state.sessionsDayFilter : null,
|
||||
mediaDetail: null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -30,7 +30,8 @@
|
||||
--color-ctp-pink: #f5bde6;
|
||||
|
||||
--font-sans:
|
||||
'Geist Variable', -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
|
||||
'Geist Variable', 'Noto Sans CJK JP', 'Noto Sans JP', 'Hiragino Sans', 'Yu Gothic UI',
|
||||
-apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
|
||||
--font-mono: 'Geist Mono Variable', 'JetBrains Mono', 'Fira Code', ui-monospace, monospace;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user