Installing ESXi in a QEMU-KVM virtual machine, under libvirt / virt-manager

For a test setup, it may be useful to install VMware ESXi in a QEMU-KVM guest. If, like me, you're using libvirt (using virt-manager) to manage your VMs, here's some information to get this set up. I'm using Fedora 22, and ESXi 5.5.0.

There are other posts explaining how to set this up, but I wanted to share my experience, which is specific to virt-manager, and the newer QEMU.

Here's a step-by-step procedure for getting this working.

Add required KVM kernel module parameters:

1. Edit (or create) /etc/modprobe.d/kvm-intel.conf to look like this:

   options kvm ignore_msrs=1
   options kvm-intel nested=y ept=y
   

2. Remove the KVM module and re-load it with the new parameters:

   # modprobe -r kvm-intel kvm; modprobe kvm kvm-intel

Setup ESXi VM guest configuration:

1. Create your ESXi VM using virt-manager.
2. Change the NIC to vmxnet3. You'll have to manually type this in; it won't be in the drop-down.
3. You'll need at least 2 GiB of RAM. (During install it actually came back with:

   <MEMORY_SIZE ERROR: This host has 2.00 GiB of RAM. 3.97 GiB are needed>

4. Edit the config for this VM (named "esxi-test" here):

   # virsh -c 'qemu:///system' edit esxi-test

5. Edit the first line of the XML file to be:

   <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

6. Change the CPU type:

   <cpu mode='host-passthrough'/>

7. Add this block anywhere inside of <domain>...</domain>:

     <qemu:commandline>
       <qemu:arg value='-machine'/>
       <qemu:arg value='vmport=off'/>
     </qemu:commandline>
   

8. Save and quit

Boot into the ESXi installer, and enjoy!

In dmesg, I see kvm spewing these messages, which probably have to do with ignroe_msrs:

kvm [3864]: vcpu0 ignored rdmsr: 0x34
kvm [3864]: vcpu0 ignored rdmsr: 0x34

ESXi 6.0.0 Notes:

I tried to use ESXi 6.0.0, but it didn't seem to find a network card, even though I specified vmnet3. These notes apply to 6.0.0:

• Note that the installer appears to hang at "user loaded successfully." for about 110 seconds. "Running nfcd start" also takes a while. I have no idea why.

Resources

• Notes on getting VMware ESXi to run under KVM (rwmj.wordpress.com) - The patch to QEMU is no longer necessary; as of at least 2.3.1, QEMU accepts the option: -machine vmport=off to accomplish the same. (Here's the patch adding the option.)
• ESXi inside KVM (communities.vmware.com) - This is the original thread where people managed to figure this out.
• Install and run a full functional VMware ESX Hypervisor within a KVM virtual machine (mattinaction.blogspot.com) - Another blog post explaining the same
• How to pass QEMU command-line options through libvirt (blog.vmsplice.net)
• Enabling Virtual Machine Control Structure Shadowing On A Nested Virtual Machine (software.intel.com) - Even though my CPU doesn't support nested VMCS, this was still useful, and provided the clue to use cpu mode='host-passthrough'