Usage of nested virtualization inside instances

If you want to use nested virtualization inside your compute instances you should try the parameter libvirt_cpu_mode=host-passthrough in the configuration of your compute nodes (/etc/nova/nova.conf).

While using libvirt_cpu_mode=host-model I saw the required CPU flag svm inside the instances. But loading the processor specific KVM module failed with the following messages in the kernel ring buffer.

[  664.051540] has_svm: can't execute cpuid_8000000a
[  664.051553] kvm: no hardware support

After changing the parameter libvirt_cpu_mode to host-passthrough it was possible to load the processor specific KVM module and to use KVM inside the instances.

[    1.585709] systemd[1]: Detected virtualization 'kvm'.
[    5.856891] kvm: Nested Virtualization enabled
[    5.856891] kvm: Nested Paging enabled

Of course you have to enable nested virtualization on the compute node first.

# cat /etc/modprobe.d/kvm.conf 
options kvm-intel nested=1

After reloading the CPU specific kernel module (kvm-intel or kvm-amd) with nested=1 you should get the following result.

# cat /sys/module/kvm_intel/parameters/nested
  • Daniel Ellison

    I added “libvirt_cpu_mode=host-passthrough” to nova.conf, restarted all nova-* services and rebooted the VM that I want to use as an image builder, but I see nothing in the output of the guest’s dmesg that looks like what you’ve provided above. There’s nothing like it in the host’s dmesg either.

    How can I determine if nested virtualization is enabled on the VM? A “modprobe kvm” on the guest doesn’t produce errors, nor does “modprobe kvm_intel”. so the kernel modules are there. I’ve installed qemu, kvm, qemu-kvm. “kvm-ok” on the guest tells me:

    INFO: /dev/kvm exists
    KVM acceleration can be used

    When I run ‘virsh capabilities | virsh cpu-baseline /dev/stdin I get:


    VMX is there, but there’s no SVM, if it’s supposed to be there.

    Any ideas?

    • Christian Berendt

      You have the flag SVM when you have a AMD CPU using AMD-V and you have the flag VMX when you have a Intel CPU using Intel VT.

      • Daniel Ellison

        That’s fine, but it wasn’t really my question. I’m not seeing anything in dmesg that even mentions KVM (aside from kvm-clock output). In particular, I don’t see “Nested Virtualization enabled”.

        I was just wondering if there was a way to confirm that nested virtualization was enabled on an instance in the absence of log entries.

  • Daniel Ellison

    Ah, my XML pasting was eaten by a grue, apparently. The key line was:

    [feature policy='require' name='vmx']

    with the square brackets replaced by angle brackets, of course.