I built a macOS virtual machine and an ubuntu virtual machine on the same host. The configuration of the two virtual machines is almost the same. But when I using sysbench to test the performance of my two VMs, I found that the memory Read/Write speed of macOS VM is slower than the Ubuntu VM when the Read/Write block size is less than L3 cache size. So, I think that the Read/Write speed of L1/L2/L3 cache in macOS VM is slower than that in Ubuntu VM. But, I don't know why it happens.
The xml file of my macOS VM is as follows:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>mac_os_catalina</name>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<vcpu placement='static'>12</vcpu>
<memoryBacking>
<hugepages/>
<nosharepages/>
</memoryBacking>
<cputune>
<vcpupin vcpu="0" cpuset="6"/>
<vcpupin vcpu="1" cpuset="30"/>
<vcpupin vcpu="2" cpuset="8"/>
<vcpupin vcpu="3" cpuset="32"/>
<vcpupin vcpu="4" cpuset="10"/>
<vcpupin vcpu="5" cpuset="34"/>
<vcpupin vcpu="6" cpuset="12"/>
<vcpupin vcpu="7" cpuset="36"/>
<vcpupin vcpu="8" cpuset="14"/>
<vcpupin vcpu="9" cpuset="38"/>
<vcpupin vcpu="10" cpuset="16"/>
<vcpupin vcpu="11" cpuset="40"/>
<emulatorpin cpuset="2,26,4,28"/>
</cputune>
<numatune>
<memory mode='preferred' nodeset='0'/>
</numatune>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-q35-2.11'>hvm</type>
<loader readonly='yes' type='pflash'>OVMF_CODE.fd</loader>
<nvram>OVMF_VARS-1024x768.fd</nvram>
</os>
<features>
<acpi/>
<apic/>
<kvm>
<hidden state='on'/>
</kvm>
<vmport state='off'/>
</features>
<cpu>
<topology sockets='12' cores='1' threads='1'/>
<cpu mode='host-model'/>
<cache level='3' mode='emulate'/>
<numa>
<cell id='0' cpus='0-11' memory='16777216' unit='KiB'/>
</numa>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='writeback'/>
<source file='ESP.qcow2'/>
<target dev='sda' bus='sata'/>
<alias name='sata-disk0'/>
<boot order='1'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<source file='BaseSystem.img'/>
<target dev='sdb' bus='sata'/>
<alias name='sata-disk1'/>
<boot order='2'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='writeback'/>
<source file='mac_hdd.qcow2'/>
<target dev='sdc' bus='sata'/>
<alias name='sata-disk2'/>
<boot order='3'/>
<address type='drive' controller='0' bus='0' target='0' unit='2'/>
</disk>
</devices>
<qemu:commandline>
<qemu:arg value='-cpu'/>
<qemu:arg value='Penryn,vendor=GenuineIntel,kvm=on,+sse3,+sse4.2,+aes,+xsave,+avx,+xsaveopt,+xsavec,+xgetbv1,+avx2,+bmi2,+smep,+bmi1,+fma,+movbe,+invtsc'/>
<qemu:arg value='-device'/>
<qemu:arg value='isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc'/>
<qemu:arg value='-smbios'/>
<qemu:arg value='type=2'/>
<qemu:arg value='-nographic'/>
</qemu:commandline>
</domain>
The xml file of Ubuntu VM is as follows:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>ubuntu</name>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<vcpu placement='static'>12</vcpu>
<cputune>
<vcpupin vcpu="0" cpuset="6"/>
<vcpupin vcpu="1" cpuset="30"/>
<vcpupin vcpu="2" cpuset="8"/>
<vcpupin vcpu="3" cpuset="32"/>
<vcpupin vcpu="4" cpuset="10"/>
<vcpupin vcpu="5" cpuset="34"/>
<vcpupin vcpu="6" cpuset="12"/>
<vcpupin vcpu="7" cpuset="36"/>
<vcpupin vcpu="8" cpuset="14"/>
<vcpupin vcpu="9" cpuset="38"/>
<vcpupin vcpu="10" cpuset="16"/>
<vcpupin vcpu="11" cpuset="40"/>
<emulatorpin cpuset="2,26,4,28"/>
</cputune>
<numatune>
<memory mode='preferred' nodeset='0'/>
</numatune>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-q35-2.11'>hvm</type>
</os>
<features>
<acpi/>
<apic/>
<!-- <kvm>-->
<!-- <hidden state='on'/>-->
<!-- </kvm>-->
<vmport state='off'/>
</features>
<cpu>
<topology sockets='1' cores='12' threads='1'/>
<cache level='3' mode='emulate'/>
<numa>
<cell id='0' cpus='0-11' memory='16777216' unit='KiB'/>
</numa>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<source file='hdd.img'/>
<target dev='sda' bus='sata'/>
<alias name='sata-disk0'/>
<boot order='1'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
</devices>
<qemu:commandline>
<qemu:arg value='-cpu'/>
<qemu:arg value='Penryn,kvm=on,l3-cache=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check'/>
<qemu:arg value='-device'/>
<qemu:arg value='isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc'/>
<qemu:arg value='-smbios'/>
<qemu:arg value='type=2'/>
<qemu:arg value='-nographic'/>
</qemu:commandline>
</domain>
As shown above, I only change the disk config in xml for different VMs.
The data of RAM read speed is as follows, 'seq' stands for sequential read, 'rnd' stands for random read.
I am searching for a long time on net. But no use. Please help or try to give some ideas how to solve this problem. Thanks in advance.
Since you're asking about the difference in perf between the two VMs, it would be useful to actually provide the configuration of both VMs. Normally a properly configured Ubuntu guest would be using virtio-blk for disk, and this will trivially beat the emulated sata controller for I/O performance. I can't see your Ubuntu guest config here though, so can't confirm if this is really the cause.