In other words, is ESXi eating my VM’s memory?
Balloon driver is a method by which VMkernel reclaims memory from virtual machines. To steal memory from a VM, VMkernel will inflate the balloon driver which is inside the GuestOS. To the GuestOS, it would look like a genuine memory usage. Behind the scene, the VMkernel will un-map the parts of the physical memory formerly allocated to the VM and give it to a different VM. This would not happen unless the host is memory over committed. Memory over commitment is when memory configured to VMs on the host is more than physically available memory.
It is not necessarily bad. What you should avoid is a situation in which memory is constantly reclaimed and moved between VMs.
Check balloon driver size using vmware-toolbox-cmd
# vmware-toolbox-cmd stat balloon 0 MB
If you use Red Hat shipped balloon driver
# mount -t debugfs nodev /sys/kernel/debug # cat /sys/kernel/debug/vmmemctl target: 0 pages current: 0 pages rateNoSleepAlloc: 16384 pages/sec rateSleepAlloc: 2048 pages/sec rateFree: 16384 pages/sec timer: 5287346 start: 2 ( 0 failed) guestType: 2 ( 0 failed) lock: 0 ( 0 failed) unlock: 0 ( 0 failed) target: 5287346 ( 1 failed) primNoSleepAlloc: 0 ( 0 failed) primCanSleepAlloc: 0 ( 0 failed) primFree: 0 errAlloc: 0 errFree: 0
In this example, balloon driver does not consume anything.
“current” : the total amount of physical memory reclaimed by balloon driver. This is the total ballooned memory by the VMs.
“target”: total amount of ballooned memory expected to be. This is the total ballooned targets of the VMs.
Which balloon driver is used?
# lsmod | grep -E 'vmmemctl|vmware_balloon' vmware_balloon 7199 0
# lsmod | grep -E 'vmmemctl|vmware_balloon' vmmemctl 46424 0
vmware_balloon is Red Hat’s balloon driver
vmmemctl is VMware’s balloon driver
If the host is reclaiming memory, what can I do?
- Avoid over “over commitment”.
- Investigate if the VM really needs that amount of memory?
- Is the GuestOS swapping? If not, it is probably still happy with memory available. VMKernel can reclaim upto 65% of configured memory. See MEMSZ and MCTLMAX for a VM.
- vMotion it to a host with more free memory.
- Investigate if the VM really needs the configured amount of memory. If not, reduce it.
- To guarantee that amount memory to a VM, set memory reservation.
- Are you using resource pools? Check the mem shares. All VMs have the same share by default. If you change it, VMs with lower mem shares will be the first target for reclamation.