Repository navigation
Description
Describe the issue
If saving a VM's state takes over 10 seconds, UTM reports "Timed out waiting for RPC" while QEMU carries on and finishes the save. Nothing records the state, so it stays in the qcow2 with nothing to delete it, using the space of a saved state each time. "Auto save on background" is on by default, so on iOS this can happen unnoticed.
Cause
QEMU runs a snapshot job's work ahead of the answer to the command that started the job, and that work holds up the main loop, so snapshot-save is answered only when the save has ended. QEMUKit waits 10 seconds for any answer (timeoutSeconds in QEMUManager.m). snapshot-load has the same order.
Over QMP, stock QEMU 11.1.2 on macOS 26.5.2, 2 GB of random guest memory:
| answered | job ended | |
|---|---|---|
snapshot-save |
2.667 s | 2.667 s |
snapshot-load |
2.207 s | 2.207 s |
Fix
Put the work behind the answer, in migration/savevm.c: emkey1/qemu@3f572363b8. The commands are then answered in about 1 ms, and QEMUKit's job handling works as its comment describes. utmapp/qemu's utm-edition has the same code. Feel free to take it as it is.
The change
diff --git a/migration/savevm.c b/migration/savevm.c
index 3f8120fa7f..0e400cb79a 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -3828,6 +3828,7 @@ typedef struct SnapshotJob {
Coroutine *co;
Error **errp;
bool ret;
+ QEMUBHFunc *work;
} SnapshotJob;
static void qmp_snapshot_job_free(SnapshotJob *s)
@@ -3886,13 +3887,30 @@ static void snapshot_delete_job_bh(void *opaque)
aio_co_wake(s->co);
}
+/*
+ * A snapshot job's work holds up the main loop, so QEMU answers nothing until
+ * it is done, and that includes the command that started the job, whose answer
+ * is still on its way out. A client that gives up on that answer takes a long
+ * save for a failed one, while the save goes on and stays in the image. So the
+ * work goes behind the answer: the monitor's wake-up is waiting in the monitor's
+ * context when this runs there, and the main context's turn comes after it.
+ */
+static void snapshot_job_after_answer_bh(void *opaque)
+{
+ Job *job = opaque;
+ SnapshotJob *s = container_of(job, SnapshotJob, common);
+
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), s->work, job);
+}
+
static int coroutine_fn snapshot_save_job_run(Job *job, Error **errp)
{
SnapshotJob *s = container_of(job, SnapshotJob, common);
s->errp = errp;
s->co = qemu_coroutine_self();
- aio_bh_schedule_oneshot(qemu_get_aio_context(),
- snapshot_save_job_bh, job);
+ s->work = snapshot_save_job_bh;
+ aio_bh_schedule_oneshot(iohandler_get_aio_context(),
+ snapshot_job_after_answer_bh, job);
qemu_coroutine_yield();
return s->ret ? 0 : -1;
}
@@ -3902,8 +3920,9 @@ static int coroutine_fn snapshot_load_job_run(Job *job, Error **errp)
SnapshotJob *s = container_of(job, SnapshotJob, common);
s->errp = errp;
s->co = qemu_coroutine_self();
- aio_bh_schedule_oneshot(qemu_get_aio_context(),
- snapshot_load_job_bh, job);
+ s->work = snapshot_load_job_bh;
+ aio_bh_schedule_oneshot(iohandler_get_aio_context(),
+ snapshot_job_after_answer_bh, job);
qemu_coroutine_yield();
return s->ret ? 0 : -1;
}
@@ -3913,8 +3932,9 @@ static int coroutine_fn snapshot_delete_job_run(Job *job, Error **errp)
SnapshotJob *s = container_of(job, SnapshotJob, common);
s->errp = errp;
s->co = qemu_coroutine_self();
- aio_bh_schedule_oneshot(qemu_get_aio_context(),
- snapshot_delete_job_bh, job);
+ s->work = snapshot_delete_job_bh;
+ aio_bh_schedule_oneshot(iohandler_get_aio_context(),
+ snapshot_job_after_answer_bh, job);
qemu_coroutine_yield();
return s->ret ? 0 : -1;
}How it was tested
An automated test in AOK VM, a fork of UTM with the same QEMUKit (e8e9e87), not in stock UTM, and with its own QEMU.
- Over QMP on macOS, with the change, x86-64 and ARM64: answered in 1 ms or less, and job times level with the unchanged build when interleaved.
- iPad Pro 13-inch (M5) simulator, iOS 27.0, x86-64 Alpine with 256 MB of random memory on a qcow2 with 512-byte clusters, to force a save of 34-45 s. Without the change: the error, nothing listed, and a 375 MiB state left in the image, 2 of 2. With it: 5 of 5 saves recorded.
- Seen without forcing it on an iPad Pro 12.9-inch (2nd generation), iOS 17.7.11: one save in 7, ReactOS at its desktop.
Configuration
- UTM Version: found in AOK VM; UTM main (a6e0c06) pins the same QEMUKit
- OS Version: iOS 27.0 (simulator), iOS 17.7.11, macOS 26.5.2
- Device Model: iPad Pro 13-inch (M5) simulator, iPad Pro 12.9-inch (2nd generation), MacBook Pro (Mac17,6)
Activity