mirror of
https://github.com/ksyasuda/SubMiner.git
synced 2026-07-27 04:49:49 -07:00
feat(sync): drop quiescence guard in favor of live WAL-based sync
- Remove `canAutoSync` dep from scheduler; auto-sync now runs during playback/stats - Sync window always passes `--force`; WAL snapshots and transactional merges handle concurrent writers - Update docs to clarify live vs cold-start sync modes
This commit is contained in:
@@ -1,5 +1,5 @@
|
||||
type: added
|
||||
area: sync
|
||||
|
||||
- Added a sync window (`subminer sync --ui`, or **Sync Stats & History** in the tray menu) for cross-machine immersion sync: saved devices with per-host direction (two-way/push/pull) and remove, one-click sync with live stage-by-stage progress and separate merge summaries for this machine and the remote device, connection testing for first-time setup, cancellable runs with a one-click `--force` retry when the running-app guard trips, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Hosts with auto-sync enabled are synced in the background on a configurable interval while no mpv session or stats server is writing the database, with results reported as overlay notifications. Hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and show up in the window automatically. `subminer sync --ui` launches silently in the background and returns the shell immediately; closing that standalone Sync window shuts down the app.
|
||||
- Added a sync window (`subminer sync --ui`, or **Sync Stats & History** in the tray menu) for cross-machine immersion sync: saved devices with per-host direction (two-way/push/pull) and remove, one-click sync with live stage-by-stage progress and separate merge summaries for this machine and the remote device, connection testing for first-time setup, cancellable live syncs while the app, stats server, or playback is active, and manual database snapshots (create/merge/reveal/delete, stored in `/tmp/subminer-db-snapshots/` by default). Live sync uses a consistent WAL snapshot and transactional merge, and excludes unfinished sessions until a later sync sees them finalized. Hosts with auto-sync enabled are synced in the background on a configurable interval, including during playback, with results reported as overlay notifications. Hosts synced from the CLI are remembered in `<config dir>/sync-hosts.json` and show up in the window automatically. `subminer sync --ui` launches silently in the background and returns the shell immediately; closing that standalone Sync window shuts down the app.
|
||||
- Added `subminer sync <host> --check` to test the SSH connection and remote launcher availability without syncing, using bounded noninteractive probes and settling when the check process exits even if an inherited output pipe remains open, without dropping terminal progress that races process exit. Linux AppImage sync commands clear inherited GUI startup argument transport before launching the bundled engine, then exit directly, so sync window checks and remote snapshots never initialize the GUI or require a display server. Added `subminer sync --json` for machine-readable NDJSON progress output (the protocol the sync window consumes).
|
||||
|
||||
@@ -97,7 +97,7 @@ How it works: each side takes a consistent snapshot of its database (`VACUUM INT
|
||||
|
||||
For a one-way transfer, `--push` snapshots the local database and merges it into the host without changing the local database. `--pull` snapshots the host and merges it into the local database without changing the host. These modes add missing data; they do not delete destination-only data or make the destination an exact mirror.
|
||||
|
||||
Close SubMiner (and stop the background stats daemon, `subminer stats -s`) on both machines before syncing; the command refuses to run while a SubMiner process may be writing the database (`--force` overrides). The mpv safety check requires a live socket connection, so a stale socket file left after mpv exits does not block sync. Both machines must be on the same SubMiner version; otherwise, the sync aborts on a stats schema mismatch.
|
||||
Command-line sync defaults to a cold-start safety check: close SubMiner (and stop the background stats daemon with `subminer stats -s`) on both machines before running it, or pass `--force`. Syncs started from the Sync window use live mode automatically, including scheduled auto-syncs while SubMiner or playback is active. SQLite WAL provides a consistent snapshot, the transactional merge serializes with live writes, and each machine's unfinished session is excluded from the transfer; that session syncs normally after it finishes. The mpv safety check requires a live socket connection, so a stale socket file left after mpv exits does not block command-line sync. Both machines must be on the same SubMiner version; otherwise, the sync aborts on a stats schema mismatch.
|
||||
|
||||
On the remote, sync looks for the `subminer` launcher first (PATH and `~/.local/bin`), then the app binary in `--sync-cli` mode (`SubMiner` on PATH, then the standard macOS `/Applications` and `~/Applications` installs), checking standard SubMiner and Bun locations (`~/.local/bin`, `~/.bun/bin`, Homebrew, `/usr/local/bin`, `/usr/bin`, and `/bin`) even when the non-interactive SSH shell omits them from `PATH`. An AppImage in a custom location can be addressed with `--remote-cmd /path/to/SubMiner.AppImage` (or symlink it as `SubMiner` somewhere on the remote PATH).
|
||||
|
||||
@@ -122,10 +122,10 @@ Unfinished sessions (a crash mid-playback) are skipped until the app finalizes t
|
||||
|
||||
- **Devices:** saved hosts with a per-host direction (two-way / push / pull), an auto-sync toggle, last-sync status, and one-click **Sync now** / **Test** / **Remove**. Hosts synced from the command line appear here automatically.
|
||||
- **Add a device:** test SSH + remote SubMiner availability before saving, with a setup checklist for first-time SSH configuration.
|
||||
- **Activity:** live stage-by-stage progress, remote output, and separate merge summaries (sessions, words, kanji, rollups) for each machine updated by the run. Runs can be cancelled, and guard failures offer a one-click `--force` retry.
|
||||
- **Activity:** live stage-by-stage progress, remote output, and separate merge summaries (sessions, words, kanji, rollups) for each machine updated by the run. Runs can be cancelled and can proceed while the app, stats server, or playback is active.
|
||||
- **Snapshots:** create manual database snapshots (stored in `/tmp/subminer-db-snapshots/` by default), merge a snapshot file into the local database, or reveal/delete existing snapshots.
|
||||
|
||||
Hosts with **Auto-sync** enabled are synced in the background on a configurable interval (default every 60 minutes) whenever no mpv session or stats server is using the database; results surface as overlay notifications. Host bookkeeping lives in `<config dir>/sync-hosts.json`.
|
||||
Hosts with **Auto-sync** enabled are synced in the background on a configurable interval (default every 60 minutes), including during active playback; results surface as overlay notifications. The unfinished playback session is skipped until a later sync sees it finalized. Host bookkeeping lives in `<config dir>/sync-hosts.json`.
|
||||
|
||||
## Common Commands
|
||||
|
||||
|
||||
+2
-4
@@ -2290,13 +2290,11 @@ const syncUiRuntime = createSyncUiRuntime({
|
||||
});
|
||||
syncUiRuntime.registerHandlers();
|
||||
|
||||
// Auto-sync only runs while no mpv session or app-owned stats server could be
|
||||
// writing the immersion database; the launcher's quiescence guard covers
|
||||
// writers this process does not know about.
|
||||
// Live sync uses a consistent WAL snapshot plus a transactional merge, and it
|
||||
// skips unfinished sessions, so resident stats writers do not block scheduling.
|
||||
const syncAutoScheduler = createSyncAutoScheduler({
|
||||
readState: () => syncUiRuntime.readState(),
|
||||
isRunning: () => syncUiRuntime.isRunning(),
|
||||
canAutoSync: () => appState.mpvClient === null && appState.statsServer === null,
|
||||
triggerHostSync: (host, direction) => {
|
||||
void syncUiRuntime.runHostSync({ host, direction }, { notify: true });
|
||||
},
|
||||
|
||||
@@ -22,7 +22,6 @@ test('tick triggers a due auto-sync host with its saved direction', () => {
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: (host, direction) => triggered.push({ host, direction }),
|
||||
nowMs: () => 100 * 60_000,
|
||||
});
|
||||
@@ -31,6 +30,20 @@ test('tick triggers a due auto-sync host with its saved direction', () => {
|
||||
assert.deepEqual(triggered, [{ host: 'auto-box', direction: 'pull' }]);
|
||||
});
|
||||
|
||||
test('tick does not require the resident app and stats writer to stop', () => {
|
||||
const state = makeState();
|
||||
const triggered: string[] = [];
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
triggerHostSync: (host) => triggered.push(host),
|
||||
nowMs: () => 100 * 60_000,
|
||||
});
|
||||
|
||||
assert.doesNotThrow(() => scheduler.tick());
|
||||
assert.deepEqual(triggered, ['auto-box']);
|
||||
});
|
||||
|
||||
test('tick skips hosts synced more recently than the interval', () => {
|
||||
let state = makeState();
|
||||
state = recordSyncResult(state, 'auto-box', {
|
||||
@@ -42,7 +55,6 @@ test('tick skips hosts synced more recently than the interval', () => {
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: (host) => triggered.push(host),
|
||||
nowMs: () => 100 * 60_000, // 10 minutes later, default interval 30
|
||||
});
|
||||
@@ -51,27 +63,17 @@ test('tick skips hosts synced more recently than the interval', () => {
|
||||
assert.deepEqual(triggered, []);
|
||||
});
|
||||
|
||||
test('tick does nothing while a run is active or auto sync is blocked', () => {
|
||||
test('tick does nothing while a run is active', () => {
|
||||
const state = makeState();
|
||||
const triggered: string[] = [];
|
||||
const running = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => true,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: (host) => triggered.push(host),
|
||||
nowMs: () => 100 * 60_000,
|
||||
});
|
||||
running.tick();
|
||||
|
||||
const blocked = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => false,
|
||||
triggerHostSync: (host) => triggered.push(host),
|
||||
nowMs: () => 100 * 60_000,
|
||||
});
|
||||
blocked.tick();
|
||||
|
||||
assert.deepEqual(triggered, []);
|
||||
});
|
||||
|
||||
@@ -82,7 +84,6 @@ test('tick triggers at most one host per tick', () => {
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: (host) => triggered.push(host),
|
||||
nowMs: () => 100 * 60_000,
|
||||
});
|
||||
@@ -98,7 +99,6 @@ test('tick logs a synchronous trigger failure and remains usable', () => {
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: () => {
|
||||
attempts += 1;
|
||||
throw new Error('spawn exploded');
|
||||
@@ -120,7 +120,6 @@ test('start/stop manage the interval timer', () => {
|
||||
const scheduler = createSyncAutoScheduler({
|
||||
readState: () => state,
|
||||
isRunning: () => false,
|
||||
canAutoSync: () => true,
|
||||
triggerHostSync: () => {},
|
||||
nowMs: () => 0,
|
||||
setIntervalFn: ((): NodeJS.Timeout => {
|
||||
|
||||
@@ -3,7 +3,6 @@ import type { SyncDirection, SyncHostsState } from '../../shared/sync/sync-hosts
|
||||
export interface SyncAutoSchedulerDeps {
|
||||
readState: () => SyncHostsState;
|
||||
isRunning: () => boolean;
|
||||
canAutoSync: () => boolean;
|
||||
triggerHostSync: (host: string, direction: SyncDirection) => void;
|
||||
nowMs: () => number;
|
||||
log?: (message: string) => void;
|
||||
@@ -18,7 +17,7 @@ export function createSyncAutoScheduler(deps: SyncAutoSchedulerDeps) {
|
||||
let timer: NodeJS.Timeout | null = null;
|
||||
|
||||
function tick(): void {
|
||||
if (deps.isRunning() || !deps.canAutoSync()) return;
|
||||
if (deps.isRunning()) return;
|
||||
const state = deps.readState();
|
||||
const intervalMs = state.autoSyncIntervalMinutes * 60_000;
|
||||
const now = deps.nowMs();
|
||||
|
||||
@@ -135,13 +135,12 @@ test('save-host rejects invalid hosts', () =>
|
||||
);
|
||||
}));
|
||||
|
||||
test('run-sync spawns the launcher, forwards progress, and rejects concurrent runs', () =>
|
||||
test('run-sync uses live mode, forwards progress, and rejects concurrent runs', () =>
|
||||
withTempDir(async (root) => {
|
||||
const { invoke, launcherCalls, sent } = makeTestRig(root);
|
||||
const start = (await invoke('sync-ui:run-sync', {
|
||||
host: 'media-box',
|
||||
direction: 'push',
|
||||
force: true,
|
||||
})) as { started: boolean; runId: number };
|
||||
assert.equal(start.started, true);
|
||||
assert.deepEqual(launcherCalls[0]!.args, ['sync', 'media-box', '--push', '--force', '--json']);
|
||||
|
||||
@@ -82,8 +82,11 @@ export function createSyncUiRuntime(deps: SyncUiRuntimeDeps) {
|
||||
const args = ['sync', host];
|
||||
if (direction === 'push') args.push('--push');
|
||||
if (direction === 'pull') args.push('--pull');
|
||||
if (request.force) args.push('--force');
|
||||
args.push('--json');
|
||||
// The sync window runs inside the resident app, so waiting for all app
|
||||
// writers to disappear would make manual and automatic sync impossible.
|
||||
// WAL keeps snapshots consistent, transactional merges serialize with live
|
||||
// writes, and unfinished sessions wait for a later sync after finalization.
|
||||
args.push('--force', '--json');
|
||||
const result = runLifecycle.startRun('host-sync', host, args, options);
|
||||
if (result.started) {
|
||||
// Persist before launcher bookkeeping lands so the host appears immediately.
|
||||
|
||||
Reference in New Issue
Block a user