We ware running libvirt on our tiny In-House servers. (4 cores / 8GB Ram, 8 cores / 32 GB Ram) and we are facing the same strange behavior on both when we try to run virsh backup-begin in »push mode«.
Both servers run Debian Bookworm and have a SSD Hard-drive attached for the main system to run and the backup to write to.
In order to create Backups of the (very few) running VMs I have created a script which generates all the needed xml files such that each Backup of each VM is written to /var/backup/<date>/<vm-name>/ - which is a directory within the root partition on the SSD.
I can observe the following using sudo htop.
Right after virsh backup-begin … is launched the swap memory get flooded right away and the number of IO Threads grows, until a »Load Average« of about is reached. Many of the threads end up in D State which indicates some sort of waiting for IO. In case a full Backup is created, which is about 50GB, it takes about 25 Minutes to complete and the sum of IO throughput is around 15-20MB/s.
Please note that those servers are not under heavy load while doing so.
Is there anything we can do in order to improve the situation, except buying real big servers?