As a Linux System Admin, we have come across such requirement multiple times for Vertical scaling of Linux VMs in VMware infrastructure. Vertical scaling means adding of additional resources like CPU, memory, Storage to server or VM for increasing its capacity to address more load of Application or DB hosted on Linux VM but what happen when this vertical scaling is require for those Mission Critical Applications or Database VMs where taking the downtime of such VMs are difficult, for addressing such requirement VMware has provided hot add or sometime it is also known as hot plug feature for its VMs where System Admin can add more vCPU or memory on the fly but unlike adding more vdisks (virtual disks) and virtual network adapter which feature is available by default for VM, enabling the vCPU and memory hot add feature require downtime for VM first time but once this hot add feature available you won’t require VM downtime further for adding more vCPUs or memory in VM.
In this Article I will explain the procedure on how to enable the VMware hot add feature for Linux VM also how to troubleshoot the issue when after hot adding the memory it’s updated memory information is not visible in Linux VM.
Prerequisite
These are below prerequisite for enabling vCPU and memory hot add feature in VM:
- The guest operating system (OS) must support device hot add. – In the case of Red Hat Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SLES) both guest OSs support this feature.
- Virtual machine hardware compatibility (virtual hardware version) is 7 or higher.
- VMware Tools should be installed on the guest OS – If you are using RHEL 7 or above and SLES 12 or above both guest OSs does not require VMware Tools to be install separately as both OSs vendor provide VMware Tools as part of OS bundled packages.
- The virtual machine must be powered off to enable the Hot Add feature.
- A VMware vSphere Advanced, Enterprise, or Enterprise Plus license must be used. VMware vSphere Enterprise Plus supports both CPU hot add and memory hot add.
Procedure
Below are steps to enable the VMware Hot Add option for adding vCPU and memory by using vCenter log in to VMware vSphere Client and do the following:
- Go to Hosts and Clusters.
- Select the needed virtual machine in the navigation pane. The VM must be powered off.
- Click the Edit Settings icon. As an alternative, you can right-click the VM name and in the menu that opens, hit Edit Settings.
- For Enabling vCPU hot add feature:
- In the Edit Settings window, expand the CPU settings.
- In the CPU Hot Plug line, select the Enable CPU Hot Add checkbox.
5. For Enabling memory hot add feature:
-
- In the Edit Settings window, expand the memory settings.
- In the Memory Hot Plug line, select the Enable checkbox.
6. Click OK to save VM settings and close the window.
7. Now you can Power On the VM again.
Troubleshooting
In case after hot adding the memory in Linux (RHEL, CentOS, and SLES OS) VM if it does not show updated memory under free or top command then follow the below steps for fixing this issue:
- To set the memory online in Linux VM:
- Look for memory that appears offline.Run this command to check the state of the memory:
grep line /sys/devices/system/memory/*/state
- When memory appears offline, run this command to set it to online:
echo online >/sys/devices/system/memory/memory[number]/state
Things need to be considered while planning to enable hot add feature in Linux VM.
- vNUMA (Virtual Non-Unified Memory Access) feature is meant for optimizing the performance of the VMs which host memory intensive application but when you enable Hot Add in Linux VM, vNUMA is automatically disabled for that VM. By default, vNUMA is enabled for all VMs that have 8 vCPUs or more to improve performance. Below is the command to check how many vNUMA nodes are assigned to Linux VM by default:
numactl –hardware
- Hot remove feature is not available for VM as removing the vCPU or memory would require Linux VM downtime hence plan for resource allocation accordingly.
He is a lazy System Administrator who likes to Automate his task through Shell scripts and Ansible playbooks. He has contributed his many Automations work in Kyndryl and also maintain his own internal company Github repo.
His hobby is researching on new technologies and sharing this through his blog but if he is not researching or writing blog then you will find him listening music while sipping cup of coffee and enjoying the nature view.
You can find him on Twitter, Facebook, Linkedin or send him an email at webmaster@cjcheema.com