NAS Datastores are greyed out in ESXi hosts. VMs becomes inaccessible, lots of them. One host in the cluster was rebooted and everything looks fine except we can see what’s in the datastores. NFS MaxQueueDepth is set to 64 pretty much in all hosts. This cluster has it set to the Max value
~# esxcfg-advcfg -g /NFS/MaxQueueDepth Value of MaxQueueDepth is 4294967295
We change it to the recommended value of 64 and rebooted the host. We are still stuck. Then came our savior. After hours of troubleshooting, it takes NetApp senior support engineer to ask us “Is Jumbo frame working”?
No. It isn’t we find. We have mtu 9000 set on the vmkernel interface for NAS but we cannot send jumbo frames traffic to he NAS filer
vmkping -I vmkX -d -s 8000 NAS_FILER_IP
The reason being the vSwitch has mtu set to 1500. So we change vSwitch to 9000 mtu and the fire is gone.
The hosts in the process of migrating to vCenter 6 from vCenter 5.x, were migrated to Standard Switch from Nexus 1KV vDS and in this process jumbo frame setting was missed on the vSwitch.
For any NAS issue, the questions have always been:
1) Is this volume exported?
2) Can you ping the NAS filer IP?
3) Is jumbo frame working? Can you ping the NAS filer with mtu > 1500? That is if you think you have jumbo frame enabled. If the vmk interface and vSwitch/vDS Port Profile mtu settings match, it should still work, be it 1500 or 9000.
The savior from NetApp explained-
There will be no problem mounting the datastore because small frames are used. But to access data, it will use big frames. Because of the vmkernel interface and vSwitch mtu setting mismatch, the frames could not go through.