Nutanix AHV: Where have all the memory gone?

In an AHV host with just the CVM and two other VMs running, there’s just about 1G of free memory. There’s 28 GB for the CVM and 8 GB each for the two other VMs. That is just 44 our of 125 GB of RAM on the host.

[root@ahv ~]# head /proc/meminfo
MemTotal:       131859556 kB
MemFree:          286384 kB
MemAvailable:    1151500 kB
Buffers:          113908 kB
Cached:           632172 kB
SwapCached:            0 kB
Active:           839356 kB
Inactive:           5600 kB
Active(anon):     106800 kB
Inactive(anon):      148 kB

This is because, from AOS 5.0, almost all memory is assigned to hugepages.

[root@ahv]# grep Huge /proc/meminfo
AnonHugePages:         0 kB
HugePages_Total:   63360
HugePages_Free:    40832
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

This host has 123 GB (HugePages_Total * Hugepagesize) assigned to huge pages!

What are huge pages? How are they different from normal page?
See here


Identify passthrough device in AHV

  1. Open an ssh session to the host
  2. Run “virsh list” and note the CVM name or ID
  3. Run “virsh dumpxml CVM_ID or CVM_name”
  4. Look for lines within hostdev tags
<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      <alias name='hostdev0'/>
      <rom bar='off'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

Mapping interfaces in a guest domain to the OVS interface

Read Correlating OVS and Guest Domain Interfaces.

I have been using a combination of virsh domiflist domain_name and ovs-vsctl to map VM interfaces to OVS interface in an AHV environment.

From Scott’s post, I learned these new commands which are new to me.

ovs-vsctl list interface vnetX
ovs-vsctl get interface vnetX external_ids

Importing a Workstation VM to AHV fails with NFS3ERR_NOENT error

To import a Workstation VM to AHV, KVM based Nutanix Hypervisor from Nutanix, I copied it to AHV container by WinSCPing to the the Prism IP:2222. While converting the vmdk to AHV format from Image Configuration, I got an error:

NFS: Lookup of /path/filename-flat.vmdk failed with NFS3ERR_NOENT(-2)

The error code NFS3ERR_NOENT means NFS is unable to find the file.

From RFC 1813:

       No such file or directory. The file or directory name
       specified does not exist.

The solution is to use .vmdk and not -flat.vmdk during conversion. The default disk format in VMware Workstation is monolithic sparse which is a single growable file.

From here:

VIXDISKLIB_DISK_MONOLITHIC_SPARSE – Growable virtual disk contained in a single virtual disk file. This is the default type for hosted disk, and the only setting in the Virtual Disk API Sample Code sample program.