mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-10-27 23:35:17 -07:00
fix(plugins): delay card-mined until field grouping and YouTube media
- Hold a sentence card's ready report until field grouping finishes - Report cards waiting on queued YouTube media once it is applied or the download fails - Proxy reports added notes ready right away when auto-update is off - Polling keeps enriching the rest of a batch after one note fails - Update plugin and architecture docs
This commit is contained in:
10 files changed
+210
-24
No files matched your search
@@ -130,6 +130,6 @@ A panel's `signal` aborts when the user closes it or another panel replaces it.
|
|||||||
|
|
||||||
mpv events: `connection-change`, `subtitle-change`, `subtitle-timing`, `time-pos-change`, `duration-change`, `pause-change`, `fullscreen-change`, `media-path-change`, `media-title-change`, `subtitle-track-change`, `subtitle-track-list-change`, `secondary-subtitle-change`, `secondary-subtitle-track-change`, `secondary-subtitle-delay-change`, `secondary-subtitle-visibility`, `client-message`, and `end-file`. `end-file` gives `{ reason, playlistEntryId, error }`, where `reason` is `eof`, `stop`, `quit`, `error`, `redirect`, or `unknown`.
|
mpv events: `connection-change`, `subtitle-change`, `subtitle-timing`, `time-pos-change`, `duration-change`, `pause-change`, `fullscreen-change`, `media-path-change`, `media-title-change`, `subtitle-track-change`, `subtitle-track-list-change`, `secondary-subtitle-change`, `secondary-subtitle-track-change`, `secondary-subtitle-delay-change`, `secondary-subtitle-visibility`, `client-message`, and `end-file`. `end-file` gives `{ reason, playlistEntryId, error }`, where `reason` is `eof`, `stop`, `quit`, `error`, `redirect`, or `unknown`.
|
||||||
|
|
||||||
Anki events: `card-mined` gives `{ noteIds }` once SubMiner has finished filling in a mined card (sentence, audio, image). With `ankiConnect.behavior.autoUpdateNewCards` off, it fires as soon as polling sees the card, and not at all for cards added through the proxy. If field grouping merged the card into an older one, the id may no longer exist.
|
Anki events: `card-mined` gives `{ noteIds }` once SubMiner has finished filling in a mined card (sentence, audio, image, and field grouping). For YouTube cards whose media waits on the download, it fires once that media is added or the download fails. With `ankiConnect.behavior.autoUpdateNewCards` off, it fires as soon as SubMiner sees the card. If field grouping merged the card into an older one, the id may no longer exist.
|
||||||
|
|
||||||
`anki.invoke` takes `{ timeoutMs }` (default 10 seconds), rejects on an Anki error, and never retries, so an action like `sync` or `addNote` runs at most once.
|
`anki.invoke` takes `{ timeoutMs }` (default 10 seconds), rejects on an Anki error, and never retries, so an action like `sync` or `addNote` runs at most once.
|
||||||
@@ -37,7 +37,7 @@ Update checks and startup launcher migration share a serialized update-state sto
|
|||||||
- `subtitle-generation-reference.ts` ranks mpv's loaded text subtitle tracks, excludes signs/songs and forced references, and extracts timing hints with FFmpeg. The overlay and launcher snapshot references only for matching media, including active subtitle delays. Hints guide long-passage cuts with or without VAD; they never limit audio coverage or replace Whisper timestamps.
|
- `subtitle-generation-reference.ts` ranks mpv's loaded text subtitle tracks, excludes signs/songs and forced references, and extracts timing hints with FFmpeg. The overlay and launcher snapshot references only for matching media, including active subtitle delays. Hints guide long-passage cuts with or without VAD; they never limit audio coverage or replace Whisper timestamps.
|
||||||
- `src/main/runtime/subtitle-selection.ts` reads and validates mpv subtitle tracks and applies primary/secondary selections. Its opt-in session shortcut opens the shared overlay modal window, with renderer focus and subtitle suppression handled by the modal registry.
|
- `src/main/runtime/subtitle-selection.ts` reads and validates mpv subtitle tracks and applies primary/secondary selections. Its opt-in session shortcut opens the shared overlay modal window, with renderer focus and subtitle suppression handled by the modal registry.
|
||||||
- `src/shared/session-key-sequences.ts` rejects sequence prefixes reserved by single-key actions. The session-binding compiler reserves configured and built-in overlay keys; `src/main/runtime/session-bindings-runtime.ts` adds active mpv bindings and publishes the effective list to both the plugin artifact and the renderer through `session-bindings:changed`. mpv no-op `ignore` bindings do not reserve prefixes.
|
- `src/shared/session-key-sequences.ts` rejects sequence prefixes reserved by single-key actions. The session-binding compiler reserves configured and built-in overlay keys; `src/main/runtime/session-bindings-runtime.ts` adds active mpv bindings and publishes the effective list to both the plugin artifact and the renderer through `session-bindings:changed`. mpv no-op `ignore` bindings do not reserve prefixes.
|
||||||
- `src/core/services/plugins/` owns user plugins: discovery from `<config dir>/plugins/<id>/plugin.json` for ids enabled in `plugins`, the per-plugin `ctx`, guarded callbacks, and `exec` (no shell, AppImage `LD_LIBRARY_PATH` dropped). `main.ts` supplies the host's deps, attaches each new mpv client so plugin listeners survive client replacement, starts plugins from the app-ready path after logging is configured, and unloads them on `before-quit`. Plugins run in the main process with no sandbox. Plugin actions are the `runPluginAction` session action (keybinding command `__plugin-action:<id>/<name>`, CLI `--plugin-action <id>/<name>`); `runAction` waits for an in-progress load so a cold-start CLI action is not dropped. Each plugin gets `<config dir>/plugin-data/<id>/` as `ctx.dataDir`. `ctx.anki` carries the `card-mined` event, fed by `AnkiIntegration.setCardsReadyCallback`: polling, the AnkiConnect proxy, and sentence-card creation report a note once its enrichment finishes (or when they see it, if auto-update is off), which is later than `recordCardsMined` (stats counts at add time). Stats-dashboard mining does not emit it. `ctx.anki.invoke` sends one action to the effective `ankiConnect.url` with no retries. `src/main/runtime/plugin-window-runtime.ts` owns plugin windows (`ctx.ui.openWindow`): one window per plugin + HTML path, raised above the overlay like the settings windows, destroyed on unload. `src/main/runtime/plugin-panel-runtime.ts` owns the single `plugin-panel` modal: a new panel aborts the previous handle's signal, updates broadcast to both overlay windows keyed by `panelId`, and the latest content is re-sent once the renderer acknowledges the open.
|
- `src/core/services/plugins/` owns user plugins: discovery from `<config dir>/plugins/<id>/plugin.json` for ids enabled in `plugins`, the per-plugin `ctx`, guarded callbacks, and `exec` (no shell, AppImage `LD_LIBRARY_PATH` dropped). `main.ts` supplies the host's deps, attaches each new mpv client so plugin listeners survive client replacement, starts plugins from the app-ready path after logging is configured, and unloads them first in the `will-quit` cleanup, before any teardown. Plugins run in the main process with no sandbox. Plugin actions are the `runPluginAction` session action (keybinding command `__plugin-action:<id>/<name>`, CLI `--plugin-action <id>/<name>`); `runAction` waits for an in-progress load so a cold-start CLI action is not dropped. Each plugin gets `<config dir>/plugin-data/<id>/` as `ctx.dataDir`. `ctx.anki` carries the `card-mined` event, fed by `AnkiIntegration.setCardsReadyCallback`: polling, the AnkiConnect proxy, and sentence-card creation report a note once its enrichment finishes (or when they see it, if auto-update is off); `AnkiIntegration` holds a sentence card's report until field grouping finishes, and the pending YouTube media queue reports notes whose media it is still waiting on, which is later than `recordCardsMined` (stats counts at add time). Stats-dashboard mining does not emit it. `ctx.anki.invoke` sends one action to the effective `ankiConnect.url` with no retries. `src/main/runtime/plugin-window-runtime.ts` owns plugin windows (`ctx.ui.openWindow`): one window per plugin + HTML path, raised above the overlay like the settings windows, destroyed on unload. `src/main/runtime/plugin-panel-runtime.ts` owns the single `plugin-panel` modal: a new panel aborts the previous handle's signal, updates broadcast to both overlay windows keyed by `panelId`, and the latest content is re-sent once the renderer acknowledges the open.
|
||||||
- `src/renderer/` owns overlay rendering and input behavior.
|
- `src/renderer/` owns overlay rendering and input behavior.
|
||||||
- `src/config/` owns config definitions, defaults, loading, and resolution.
|
- `src/config/` owns config definitions, defaults, loading, and resolution.
|
||||||
- `src/types/` owns shared cross-runtime contracts via domain entrypoints; `src/types.ts` stays a compatibility barrel.
|
- `src/types/` owns shared cross-runtime contracts via domain entrypoints; `src/types.ts` stays a compatibility barrel.
|
||||||
|
|||||||
@@ -82,6 +82,7 @@ type IntegrationInternals = {
|
|||||||
fields: Record<string, { value: string }>;
|
fields: Record<string, { value: string }>;
|
||||||
}) => void;
|
}) => void;
|
||||||
rememberMergedNoteIds: (deletedNoteId: number, keptNoteId: number) => void;
|
rememberMergedNoteIds: (deletedNoteId: number, keptNoteId: number) => void;
|
||||||
|
reportCardsReady: (noteIds: number[]) => void;
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Named stand-ins for AnkiIntegration's long positional constructor arguments. */
|
/** Named stand-ins for AnkiIntegration's long positional constructor arguments. */
|
||||||
@@ -404,6 +405,55 @@ test('AnkiIntegration triggers field grouping after a local duplicate sentence c
|
|||||||
assert.equal(groupingTriggered, 1);
|
assert.equal(groupingTriggered, 1);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('AnkiIntegration reports a new sentence card ready only after field grouping', async () => {
|
||||||
|
const integration = createIntegration({
|
||||||
|
isKiku: { enabled: true, fieldGrouping: 'manual' },
|
||||||
|
});
|
||||||
|
const events: string[] = [];
|
||||||
|
integration.setCardsReadyCallback((noteIds) => events.push(`ready ${noteIds.join(',')}`));
|
||||||
|
const internals = internalsOf(integration);
|
||||||
|
internals.cardCreationService = {
|
||||||
|
createSentenceCard: async () => {
|
||||||
|
integration.trackDuplicateNoteIdsForNote(42, [7]);
|
||||||
|
internals.reportCardsReady([42]);
|
||||||
|
return true;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
internals.fieldGroupingService = {
|
||||||
|
triggerFieldGroupingForLastAddedCard: async () => {
|
||||||
|
events.push('group');
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
assert.equal(await integration.createSentenceCard('duplicate sentence', 0, 1), true);
|
||||||
|
assert.deepEqual(events, ['group', 'ready 42']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('AnkiIntegration reports a card with queued YouTube media ready once the media lands', async () => {
|
||||||
|
const events: string[] = [];
|
||||||
|
const integration = createIntegration({
|
||||||
|
fields: { audio: 'ExpressionAudio', image: 'Picture' },
|
||||||
|
media: { imageFormat: 'jpg' },
|
||||||
|
behavior: { notificationType: 'osd' },
|
||||||
|
});
|
||||||
|
installFakeAnki(integration, {
|
||||||
|
noteFieldNames: ['ExpressionAudio', 'Picture'],
|
||||||
|
mediaGenerator: {
|
||||||
|
generateAudio: async () => Buffer.from('audio'),
|
||||||
|
generateScreenshot: async () => Buffer.from('image'),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
integration.setCardsReadyCallback((noteIds) => events.push(`ready ${noteIds.join(',')}`));
|
||||||
|
const internals = internalsOf(integration);
|
||||||
|
internals.queuePendingYoutubeMediaUpdate(youtubeJob({ noteId: 101 }));
|
||||||
|
|
||||||
|
internals.reportCardsReady([101, 5]);
|
||||||
|
assert.deepEqual(events, ['ready 5']);
|
||||||
|
|
||||||
|
await integration.handleYoutubeMediaCacheReady('https://youtu.be/abc123', '/tmp/media.mkv');
|
||||||
|
assert.deepEqual(events, ['ready 5', 'ready 101']);
|
||||||
|
});
|
||||||
|
|
||||||
test('AnkiIntegration marks partial update notifications as failures in OSD mode', async () => {
|
test('AnkiIntegration marks partial update notifications as failures in OSD mode', async () => {
|
||||||
const osdMessages: string[] = [];
|
const osdMessages: string[] = [];
|
||||||
const integration = createIntegration(
|
const integration = createIntegration(
|
||||||
|
|||||||
+46
-19
@@ -241,6 +241,8 @@ export class AnkiIntegration {
|
|||||||
private aiConfig: AiConfig;
|
private aiConfig: AiConfig;
|
||||||
private recordCardsMinedCallback: ((count: number, noteIds?: number[]) => void) | null = null;
|
private recordCardsMinedCallback: ((count: number, noteIds?: number[]) => void) | null = null;
|
||||||
private cardsReadyCallback: ((noteIds: number[]) => void) | null = null;
|
private cardsReadyCallback: ((noteIds: number[]) => void) | null = null;
|
||||||
|
// Ready reports collected while createSentenceCard still has field grouping to run.
|
||||||
|
private heldReadyNoteIds: number[] | null = null;
|
||||||
private knownWordCacheUpdatedCallback: (() => void) | null = null;
|
private knownWordCacheUpdatedCallback: (() => void) | null = null;
|
||||||
private consumeSubtitleMiningContextCallback: (() => SubtitleMiningContext | null) | null = null;
|
private consumeSubtitleMiningContextCallback: (() => SubtitleMiningContext | null) | null = null;
|
||||||
private generateSentenceFuriganaCallback:
|
private generateSentenceFuriganaCallback:
|
||||||
@@ -338,6 +340,18 @@ export class AnkiIntegration {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private reportCardsReady(noteIds: number[]): void {
|
private reportCardsReady(noteIds: number[]): void {
|
||||||
|
if (this.heldReadyNoteIds) {
|
||||||
|
this.heldReadyNoteIds.push(...noteIds);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Notes still waiting on queued YouTube media are reported by the queue once it settles.
|
||||||
|
this.emitCardsReady(
|
||||||
|
noteIds.filter((noteId) => !this.pendingYoutubeMediaQueue.hasPendingNote(noteId)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
private emitCardsReady(noteIds: number[]): void {
|
||||||
|
if (noteIds.length === 0) return;
|
||||||
try {
|
try {
|
||||||
this.cardsReadyCallback?.(noteIds);
|
this.cardsReadyCallback?.(noteIds);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
@@ -423,6 +437,7 @@ export class AnkiIntegration {
|
|||||||
showStatusNotification: (message) => this.showStatusNotification(message),
|
showStatusNotification: (message) => this.showStatusNotification(message),
|
||||||
showNotification: (noteId, label, errorSuffix) =>
|
showNotification: (noteId, label, errorSuffix) =>
|
||||||
this.showNotification(noteId, label, errorSuffix),
|
this.showNotification(noteId, label, errorSuffix),
|
||||||
|
onUpdateSettled: (noteId) => this.emitCardsReady([noteId]),
|
||||||
logInfo: (...args) => log.info(args[0] as string, ...args.slice(1)),
|
logInfo: (...args) => log.info(args[0] as string, ...args.slice(1)),
|
||||||
logWarn: (...args) => log.warn(args[0] as string, ...args.slice(1)),
|
logWarn: (...args) => log.warn(args[0] as string, ...args.slice(1)),
|
||||||
logError: (...args) => log.error(args[0] as string, ...args.slice(1)),
|
logError: (...args) => log.error(args[0] as string, ...args.slice(1)),
|
||||||
@@ -1555,25 +1570,36 @@ export class AnkiIntegration {
|
|||||||
secondarySubText?: string,
|
secondarySubText?: string,
|
||||||
): Promise<boolean> {
|
): Promise<boolean> {
|
||||||
const trackedDuplicateNoteIdsBeforeCreate = new Set(this.trackedDuplicateNoteIds.keys());
|
const trackedDuplicateNoteIdsBeforeCreate = new Set(this.trackedDuplicateNoteIds.keys());
|
||||||
const created = await this.cardCreationService.createSentenceCard(
|
// Field grouping below can still rewrite the new card, so hold its ready report until then.
|
||||||
sentence,
|
const ownsHold = this.heldReadyNoteIds === null;
|
||||||
startTime,
|
if (ownsHold) this.heldReadyNoteIds = [];
|
||||||
endTime,
|
try {
|
||||||
secondarySubText,
|
const created = await this.cardCreationService.createSentenceCard(
|
||||||
);
|
sentence,
|
||||||
if (
|
startTime,
|
||||||
created &&
|
endTime,
|
||||||
this.shouldTriggerFieldGroupingAfterLocalSentenceCardCreate(
|
secondarySubText,
|
||||||
trackedDuplicateNoteIdsBeforeCreate,
|
);
|
||||||
)
|
if (
|
||||||
) {
|
created &&
|
||||||
try {
|
this.shouldTriggerFieldGroupingAfterLocalSentenceCardCreate(
|
||||||
await this.fieldGroupingService.triggerFieldGroupingForLastAddedCard();
|
trackedDuplicateNoteIdsBeforeCreate,
|
||||||
} catch (error) {
|
)
|
||||||
log.warn('Failed to trigger field grouping after sentence card creation:', error);
|
) {
|
||||||
|
try {
|
||||||
|
await this.fieldGroupingService.triggerFieldGroupingForLastAddedCard();
|
||||||
|
} catch (error) {
|
||||||
|
log.warn('Failed to trigger field grouping after sentence card creation:', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return created;
|
||||||
|
} finally {
|
||||||
|
if (ownsHold) {
|
||||||
|
const held = this.heldReadyNoteIds ?? [];
|
||||||
|
this.heldReadyNoteIds = null;
|
||||||
|
this.reportCardsReady(held);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return created;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
trackDuplicateNoteIdsForNote(noteId: number, duplicateNoteIds: number[]): void {
|
trackDuplicateNoteIdsForNote(noteId: number, duplicateNoteIds: number[]): void {
|
||||||
@@ -1793,8 +1819,9 @@ export class AnkiIntegration {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Called once SubMiner has finished writing a mined card (fields and media), or as soon as polling
|
* Called once SubMiner has finished writing a mined card (fields, media including queued YouTube
|
||||||
* sees it when auto-update is off. Field grouping may have merged the note away by then.
|
* media, and field grouping), or as soon as polling or the proxy sees it when auto-update is off.
|
||||||
|
* Field grouping may have merged the note away by then.
|
||||||
*/
|
*/
|
||||||
setCardsReadyCallback(callback: ((noteIds: number[]) => void) | null): void {
|
setCardsReadyCallback(callback: ((noteIds: number[]) => void) | null): void {
|
||||||
this.cardsReadyCallback = callback;
|
this.cardsReadyCallback = callback;
|
||||||
|
|||||||
@@ -54,6 +54,40 @@ test('proxy enqueues addNote result for enrichment and reports it ready afterwar
|
|||||||
assert.deepEqual(events, ['process 42', 'ready 42']);
|
assert.deepEqual(events, ['process 42', 'ready 42']);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('proxy reports an added note ready right away when auto-update is off', () => {
|
||||||
|
const processed: number[] = [];
|
||||||
|
const recordedCards: number[] = [];
|
||||||
|
const ready: number[][] = [];
|
||||||
|
const proxy = new AnkiConnectProxyServer({
|
||||||
|
shouldAutoUpdateNewCards: () => false,
|
||||||
|
processNewCard: async (noteId) => {
|
||||||
|
processed.push(noteId);
|
||||||
|
},
|
||||||
|
recordCardsAdded: (count) => {
|
||||||
|
recordedCards.push(count);
|
||||||
|
},
|
||||||
|
onCardsReady: (noteIds) => {
|
||||||
|
ready.push(noteIds);
|
||||||
|
},
|
||||||
|
logInfo: () => undefined,
|
||||||
|
logWarn: () => undefined,
|
||||||
|
logError: () => undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
(
|
||||||
|
proxy as unknown as {
|
||||||
|
maybeEnqueueFromRequest: (request: Record<string, unknown>, responseBody: Buffer) => void;
|
||||||
|
}
|
||||||
|
).maybeEnqueueFromRequest(
|
||||||
|
{ action: 'addNote' },
|
||||||
|
Buffer.from(JSON.stringify({ result: 42, error: null }), 'utf8'),
|
||||||
|
);
|
||||||
|
|
||||||
|
assert.deepEqual(ready, [[42]]);
|
||||||
|
assert.deepEqual(processed, []);
|
||||||
|
assert.deepEqual(recordedCards, []);
|
||||||
|
});
|
||||||
|
|
||||||
test('proxy enqueues addNote bare numeric response for enrichment', async () => {
|
test('proxy enqueues addNote bare numeric response for enrichment', async () => {
|
||||||
const processed: number[] = [];
|
const processed: number[] = [];
|
||||||
const proxy = new AnkiConnectProxyServer({
|
const proxy = new AnkiConnectProxyServer({
|
||||||
|
|||||||
@@ -218,15 +218,17 @@ export class AnkiConnectProxyServer {
|
|||||||
requestJson: Record<string, unknown> | null,
|
requestJson: Record<string, unknown> | null,
|
||||||
responseBody: Buffer,
|
responseBody: Buffer,
|
||||||
): void {
|
): void {
|
||||||
if (!requestJson || !this.deps.shouldAutoUpdateNewCards()) {
|
if (!requestJson) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
const autoUpdate = this.deps.shouldAutoUpdateNewCards();
|
||||||
|
|
||||||
const action =
|
const action =
|
||||||
typeof requestJson.action === 'string'
|
typeof requestJson.action === 'string'
|
||||||
? requestJson.action
|
? requestJson.action
|
||||||
: String(requestJson.action ?? '');
|
: String(requestJson.action ?? '');
|
||||||
if (action === 'updateNoteFields') {
|
if (action === 'updateNoteFields') {
|
||||||
|
if (!autoUpdate) return;
|
||||||
const params = requestJson.params;
|
const params = requestJson.params;
|
||||||
if (
|
if (
|
||||||
!params ||
|
!params ||
|
||||||
@@ -279,7 +281,12 @@ export class AnkiConnectProxyServer {
|
|||||||
params.subminerEnrich === false
|
params.subminerEnrich === false
|
||||||
) {
|
) {
|
||||||
// Stats owns the saved sentence and media; the live mpv context is unrelated.
|
// Stats owns the saved sentence and media; the live mpv context is unrelated.
|
||||||
if (noteIds.length > 0) this.deps.recordCardsAdded?.(noteIds.length, noteIds);
|
if (autoUpdate && noteIds.length > 0) this.deps.recordCardsAdded?.(noteIds.length, noteIds);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!autoUpdate) {
|
||||||
|
// Nothing to enrich, so the added notes are already in their final state.
|
||||||
|
if (noteIds.length > 0) this.deps.onCardsReady?.(noteIds);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
this.maybeTrackDuplicateNoteIds(requestJson, action, responseResult);
|
this.maybeTrackDuplicateNoteIds(requestJson, action, responseResult);
|
||||||
|
|||||||
@@ -152,7 +152,7 @@ interface CardCreationDeps {
|
|||||||
trackLastAddedDuplicateNoteIds?: (noteId: number, duplicateNoteIds: number[]) => void;
|
trackLastAddedDuplicateNoteIds?: (noteId: number, duplicateNoteIds: number[]) => void;
|
||||||
findDuplicateNoteIds?: (expression: string, noteInfo: CardCreationNoteInfo) => Promise<number[]>;
|
findDuplicateNoteIds?: (expression: string, noteInfo: CardCreationNoteInfo) => Promise<number[]>;
|
||||||
recordCardsMinedCallback?: (count: number, noteIds?: number[]) => void;
|
recordCardsMinedCallback?: (count: number, noteIds?: number[]) => void;
|
||||||
/** Called once a created card has all its fields and media written. */
|
/** Called once card creation is done writing a card; queued YouTube media may still be pending. */
|
||||||
onCardsReady?: (noteIds: number[]) => void;
|
onCardsReady?: (noteIds: number[]) => void;
|
||||||
reviewMediaTiming?: (
|
reviewMediaTiming?: (
|
||||||
request: Omit<MediaTimingReviewRequest, 'audioPadding' | 'maxMediaDuration'>,
|
request: Omit<MediaTimingReviewRequest, 'audioPadding' | 'maxMediaDuration'>,
|
||||||
|
|||||||
@@ -56,6 +56,8 @@ export interface PendingYoutubeMediaQueueDeps {
|
|||||||
) => string;
|
) => string;
|
||||||
showStatusNotification: (message: string) => void;
|
showStatusNotification: (message: string) => void;
|
||||||
showNotification: (noteId: number, label: string | number, errorSuffix?: string) => Promise<void>;
|
showNotification: (noteId: number, label: string | number, errorSuffix?: string) => Promise<void>;
|
||||||
|
/** Called once a queued note's media was applied or dropped, so the card is in its final state. */
|
||||||
|
onUpdateSettled?: (noteId: number) => void;
|
||||||
logInfo: (message: string, ...args: unknown[]) => void;
|
logInfo: (message: string, ...args: unknown[]) => void;
|
||||||
logWarn: (message: string, ...args: unknown[]) => void;
|
logWarn: (message: string, ...args: unknown[]) => void;
|
||||||
logError: (message: string, ...args: unknown[]) => void;
|
logError: (message: string, ...args: unknown[]) => void;
|
||||||
@@ -79,9 +81,18 @@ function shouldGenerateImage(config: AnkiConnectConfig): boolean {
|
|||||||
|
|
||||||
export class PendingYoutubeMediaQueue {
|
export class PendingYoutubeMediaQueue {
|
||||||
private updates: PendingYoutubeMediaUpdate[] = [];
|
private updates: PendingYoutubeMediaUpdate[] = [];
|
||||||
|
// Jobs taken off the queue whose media is still being applied or dropped.
|
||||||
|
private settlingNoteIds = new Set<number>();
|
||||||
|
|
||||||
constructor(private readonly deps: PendingYoutubeMediaQueueDeps) {}
|
constructor(private readonly deps: PendingYoutubeMediaQueueDeps) {}
|
||||||
|
|
||||||
|
/** True while the note still waits on queued media. */
|
||||||
|
hasPendingNote(noteId: number): boolean {
|
||||||
|
return (
|
||||||
|
this.settlingNoteIds.has(noteId) || this.updates.some((update) => update.noteId === noteId)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
enqueue(job: PendingYoutubeMediaUpdate): void {
|
enqueue(job: PendingYoutubeMediaUpdate): void {
|
||||||
if (!job.generateAudio && !job.generateImage) {
|
if (!job.generateAudio && !job.generateImage) {
|
||||||
return;
|
return;
|
||||||
@@ -195,6 +206,8 @@ export class PendingYoutubeMediaQueue {
|
|||||||
'Failed to apply queued YouTube media update:',
|
'Failed to apply queued YouTube media update:',
|
||||||
error instanceof Error ? error.message : String(error),
|
error instanceof Error ? error.message : String(error),
|
||||||
);
|
);
|
||||||
|
} finally {
|
||||||
|
this.settle(job);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -234,16 +247,24 @@ export class PendingYoutubeMediaQueue {
|
|||||||
'Failed to show queued YouTube media failure notification:',
|
'Failed to show queued YouTube media failure notification:',
|
||||||
error instanceof Error ? error.message : String(error),
|
error instanceof Error ? error.message : String(error),
|
||||||
);
|
);
|
||||||
|
} finally {
|
||||||
|
this.settle(job);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private settle(job: PendingYoutubeMediaUpdate): void {
|
||||||
|
this.settlingNoteIds.delete(job.noteId);
|
||||||
|
this.deps.onUpdateSettled?.(job.noteId);
|
||||||
|
}
|
||||||
|
|
||||||
private takeMatchingUpdates(sourceUrl: string): PendingYoutubeMediaUpdate[] {
|
private takeMatchingUpdates(sourceUrl: string): PendingYoutubeMediaUpdate[] {
|
||||||
const matched: PendingYoutubeMediaUpdate[] = [];
|
const matched: PendingYoutubeMediaUpdate[] = [];
|
||||||
const remaining: PendingYoutubeMediaUpdate[] = [];
|
const remaining: PendingYoutubeMediaUpdate[] = [];
|
||||||
for (const job of this.updates) {
|
for (const job of this.updates) {
|
||||||
if (youtubeMediaUrlsMatch(job.sourceUrl, sourceUrl)) {
|
if (youtubeMediaUrlsMatch(job.sourceUrl, sourceUrl)) {
|
||||||
matched.push(job);
|
matched.push(job);
|
||||||
|
this.settlingNoteIds.add(job.noteId);
|
||||||
} else {
|
} else {
|
||||||
remaining.push(job);
|
remaining.push(job);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -49,3 +49,47 @@ test('polling runner records new cards and reports each ready after enrichment',
|
|||||||
Date.now = originalDateNow;
|
Date.now = originalDateNow;
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('polling runner keeps going after one note fails to enrich', async () => {
|
||||||
|
const originalDateNow = Date.now;
|
||||||
|
const events: string[] = [];
|
||||||
|
const warnings: unknown[][] = [];
|
||||||
|
let tracked = new Set<number>();
|
||||||
|
const responses = [[10], [10, 12, 13]];
|
||||||
|
try {
|
||||||
|
Date.now = () => 120_000;
|
||||||
|
const runner = new PollingRunner({
|
||||||
|
getDeck: () => undefined,
|
||||||
|
getPollingRate: () => 250,
|
||||||
|
findNotes: async () => responses.shift() ?? [],
|
||||||
|
shouldAutoUpdateNewCards: () => true,
|
||||||
|
processNewCard: async (noteId) => {
|
||||||
|
events.push(`process ${noteId}`);
|
||||||
|
if (noteId === 12) throw new Error('overlay callback failed');
|
||||||
|
},
|
||||||
|
onCardsReady: (noteIds) => {
|
||||||
|
events.push(`ready ${noteIds.join(',')}`);
|
||||||
|
},
|
||||||
|
isUpdateInProgress: () => false,
|
||||||
|
setUpdateInProgress: () => undefined,
|
||||||
|
getTrackedNoteIds: () => tracked,
|
||||||
|
setTrackedNoteIds: (noteIds) => {
|
||||||
|
tracked = noteIds;
|
||||||
|
},
|
||||||
|
showStatusNotification: () => undefined,
|
||||||
|
logDebug: () => undefined,
|
||||||
|
logInfo: () => undefined,
|
||||||
|
logWarn: (...args) => {
|
||||||
|
warnings.push(args);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
await runner.pollOnce();
|
||||||
|
await runner.pollOnce();
|
||||||
|
|
||||||
|
assert.deepEqual(events, ['process 12', 'ready 12', 'process 13', 'ready 13']);
|
||||||
|
assert.deepEqual(warnings, [['Failed to auto-enrich note 12:', 'overlay callback failed']]);
|
||||||
|
} finally {
|
||||||
|
Date.now = originalDateNow;
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -89,6 +89,9 @@ export class PollingRunner {
|
|||||||
for (const noteId of newNoteIds) {
|
for (const noteId of newNoteIds) {
|
||||||
try {
|
try {
|
||||||
await this.deps.processNewCard(noteId);
|
await this.deps.processNewCard(noteId);
|
||||||
|
} catch (error) {
|
||||||
|
// One failed note must not leave the rest of the batch unprocessed.
|
||||||
|
this.deps.logWarn(`Failed to auto-enrich note ${noteId}:`, (error as Error).message);
|
||||||
} finally {
|
} finally {
|
||||||
this.deps.onCardsReady?.([noteId]);
|
this.deps.onCardsReady?.([noteId]);
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in new issue
Block a user