sudacode/SubMiner
1
0
Fork 0
mirror of https://github.com/ksyasuda/SubMiner.git synced 2026-07-27 04:49:49 -07:00
Code Issues Packages Projects Releases Wiki Activity

feat(sync): emit structured remote merge-summary event

Browse Source 
- Remote merge step now passes --json to get NDJSON output
- Parses merge-summary from remote stdout; emits as merge-summary/remote event
- Falls back to remote-output for non-JSON runs
- Updates docs and changelog to reflect per-machine merge summaries
This commit is contained in:
Kyle Yasuda
2026-07-13 02:40:59 -07:00
parent 4c63f7e3b0
commit 4a10257bc9
4 changed files with 35 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
changes/sync-ui-window.md
+1 -1
View File
@@ -1,5 +1,5 @@
type: added
area: sync
- Added a sync window (`subminer sync --ui`, or **Sync Stats & History** in the tray menu) for cross-machine immersion sync: saved devices with per-host direction (two-way/push/pull) and remove, one-click sync with live stage-by-stage progress and merge summaries, connection testing for first-time setup, cancellable runs with a one-click `--force` retry when the running-app guard trips, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Hosts with auto-sync enabled are synced in the background on a configurable interval while no mpv session or stats server is writing the database, with results reported as overlay notifications. Hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and show up in the window automatically. `subminer sync --ui` launches silently in the background and returns the shell immediately; closing that standalone Sync window shuts down the app.
- Added a sync window (`subminer sync --ui`, or **Sync Stats & History** in the tray menu) for cross-machine immersion sync: saved devices with per-host direction (two-way/push/pull) and remove, one-click sync with live stage-by-stage progress and separate merge summaries for this machine and the remote device, connection testing for first-time setup, cancellable runs with a one-click `--force` retry when the running-app guard trips, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Hosts with auto-sync enabled are synced in the background on a configurable interval while no mpv session or stats server is writing the database, with results reported as overlay notifications. Hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and show up in the window automatically. `subminer sync --ui` launches silently in the background and returns the shell immediately; closing that standalone Sync window shuts down the app.
- Added `subminer sync <host> --check` to test the SSH connection and remote launcher availability without syncing, using bounded noninteractive probes and settling when the check process exits even if an inherited output pipe remains open, without dropping terminal progress that races process exit. Linux AppImage sync commands clear inherited GUI startup argument transport before launching the bundled engine, then exit directly, so sync window checks and remote snapshots never initialize the GUI or require a display server. Added `subminer sync --json` for machine-readable NDJSON progress output (the protocol the sync window consumes).
docs-site/launcher-script.md
+1 -1
View File
@@ -122,7 +122,7 @@ Unfinished sessions (a crash mid-playback) are skipped until the app finalizes t
- **Devices:** saved hosts with a per-host direction (two-way / push / pull), an auto-sync toggle, last-sync status, and one-click **Sync now** / **Test** / **Remove**. Hosts synced from the command line appear here automatically.
- **Add a device:** test SSH + remote SubMiner availability before saving, with a setup checklist for first-time SSH configuration.
- **Activity:** live stage-by-stage progress, remote output, and a merge summary (sessions, words, kanji, rollups) when a run finishes. Runs can be cancelled, and guard failures offer a one-click `--force` retry.
- **Activity:** live stage-by-stage progress, remote output, and separate merge summaries (sessions, words, kanji, rollups) for each machine updated by the run. Runs can be cancelled, and guard failures offer a one-click `--force` retry.
- **Snapshots:** create manual database snapshots (stored in `/tmp/subminer-db-snapshots/` by default), merge a snapshot file into the local database, or reveal/delete existing snapshots.
Hosts with **Auto-sync** enabled are synced in the background on a configurable interval (default every 60 minutes) whenever no mpv session or stats server is using the database; results surface as overlay notifications. Host bookkeeping lives in `<config dir>/sync-hosts.json`.
src/core/services/stats-sync/sync-flow.test.ts
+15 -5
View File
@@ -250,13 +250,24 @@ test('runHostSync pull only snapshots remotely and merges locally', async () =>
test('runSyncFlow --json emits NDJSON progress events and a final result', async () => {
const lines: string[] = [];
const remoteSummary = {
...createEmptyMergeSummary(),
sessionsMerged: 2,
videosAdded: 1,
};
const deps = makeHostDeps([], {
consoleLog: (line) => {
lines.push(line);
},
runSsh: (_host, command) => {
if (command.includes(' sync --make-temp')) return ok('/tmp/subminer-sync-remote\n');
if (command.includes(' sync --merge ')) return ok('remote summary text\n');
if (command.includes(' sync --merge ')) {
assert.match(command, / --json(?: |$)/);
return ok(
`${JSON.stringify({ type: 'merge-summary', target: 'local', summary: remoteSummary })}\n` +
`${JSON.stringify({ type: 'result', ok: true, error: null })}\n`,
);
}
return ok();
},
});
@@ -269,10 +280,9 @@ test('runSyncFlow --json emits NDJSON progress events and a final result', async
const events = lines.map((line) => JSON.parse(line));
assert.ok(events.some((event) => event.type === 'stage' && event.stage === 'snapshot-local'));
assert.ok(events.some((event) => event.type === 'merge-summary' && event.target === 'local'));
assert.ok(
events.some(
(event) => event.type === 'remote-output' && event.text.includes('remote summary text'),
),
assert.deepEqual(
events.find((event) => event.type === 'merge-summary' && event.target === 'remote'),
{ type: 'merge-summary', target: 'remote', summary: remoteSummary },
);
assert.deepEqual(events[events.length - 1], { type: 'result', ok: true, error: null });
});
src/core/services/stats-sync/sync-flow.ts
+18 -3
View File
@@ -3,7 +3,11 @@ import path from 'node:path';
import { formatMergeSummary } from './merge';
import { quoteForRemoteShell } from './ssh';
import type { RemoteRunResult, RemoteShellFlavor, RunSshOptions } from './ssh';
import type { SyncMergeSummary, SyncProgressEvent } from '../../../shared/sync/sync-events';
import {
parseSyncProgressLine,
type SyncMergeSummary,
type SyncProgressEvent,
} from '../../../shared/sync/sync-events';
import type { SyncResultStatus } from '../../../shared/sync/sync-hosts-store';
export interface SyncFlowArgs {
@@ -276,6 +280,14 @@ function parseRemoteTempDir(stdout: string): string {
return path.posix.basename(candidate).startsWith(SYNC_TEMP_PREFIX) ? candidate : '';
}
function parseRemoteMergeSummary(stdout: string): SyncMergeSummary | null {
for (const line of stdout.split('\n')) {
const event = parseSyncProgressLine(line);
if (event?.type === 'merge-summary' && event.target === 'local') return event.summary;
}
return null;
}
function formatRemoteRunError(message: string, run: RemoteRunResult): string {
const stderr = run.stderr.trim();
return stderr ? `${message}\n${stderr}` : message;
@@ -382,10 +394,13 @@ export async function runHostSync(
await deps.ensureTrackerQuiescent(context, dbPath);
const mergeRun = deps.runSsh(
host,
`${remoteCmd} sync --merge ${quote(incomingSnapshot)}${forceFlag}`,
`${remoteCmd} sync --merge ${quote(incomingSnapshot)}${forceFlag}${args.syncJson ? ' --json' : ''}`,
);
deps.writeStdout(mergeRun.stdout);
if (mergeRun.stdout.trim()) {
const remoteSummary = args.syncJson ? parseRemoteMergeSummary(mergeRun.stdout) : null;
if (remoteSummary) {
deps.emitEvent({ type: 'merge-summary', target: 'remote', summary: remoteSummary });
} else if (mergeRun.stdout.trim()) {
deps.emitEvent({ type: 'remote-output', text: mergeRun.stdout });
}
if (mergeRun.status !== 0) {