Table of Contents
- Introduction
- What is VMware EVC Mode?
- How Does VMware EVC Mode Work?
- Benefits and Use Cases
- Enable VMware EVC Mode on a vSphere Cluster
- Difference between cluster level EVC and VMware per-VM EVC
- Enabling Per-VM EVC Mode
- How to Check EVC Mode
- How to Change and Turn Off EVC Mode
- VMware vCenter Server Version Needs to be Considered
- How Does VMware EVC Mode Affect Performance?
- Disadvantages and Final Thoughts
Introduction
For most who remember their first experience with virtualization, it was the VMware vMotion process that was the moment when they knew virtualization was here to stay. The VMware vMotion process has allowed countless virtual machines to be “unlocked” from the underlying hardware and moved to a different server in the VMware vSphere cluster so that maintenance, upgrades, and other tasks performed on the actual server hardware can be done with no down time to business-critical virtual machines.
The vMotion process represents the power of virtualization at its best with the ability to seemingly defy traditional infrastructure constraints and make VMs transient without being limited to the specific host hardware runs from currently. There is another technology that coincides with VMware vMotion technology that makes this process even more powerful, flexible, and allows your cluster to scale without the limit of hardware generations. This technology is VMware EVC Mode. What is VMware EVC mode? How does it work? What is its purpose? Are there disadvantages to using it? Let’s dive into VMware EVC Mode and see how it fits into your VMware vSphere infrastructure.
What is VMware EVC Mode?
The VMware EVC Mode is directly related to vMotion. In fact, the term EVC stands for Enhanced vMotion Compatibility. The whole purpose of VMware EVC Mode is to allow the ability to be able to migrate virtual machines between ESXi hosts in the same cluster that are running dissimilar processors architectures.
VMware EVC Mode allows this functionality by forcing a specific CPUID instruction set between the various ESXi hosts so that only a single “family” of processor or instruction set is presented to the virtual machine, regardless if the ESXi host differs in the CPU instruction set it can present to the virtual machine compared to the instruction set of the other hosts in the cluster.
VMware EVC Mode is able to mask the CPU instruction set so these appear to be exactly the same across the entire vSphere cluster which may contain a number of different underlying CPU architectures in each ESXi host. As you can imagine this allows you to have the opportunity to do things that you would not otherwise be able to do when scaling your vSphere clusters.
VMware EVC Mode can be enabled at two different levels as of vSphere 6.7. VMware EVC can be enabled at the vSphere cluster-level and at the virtual machine level. We will discuss both of these configurations in the following sections.
How is this functionality of controlling the CPUID that is presented to the virtual machines in a vSphere cluster possible using VMware EVC Mode? To get a better understanding of this, let’s discuss how VMware EVC Mode works.
How Does VMware EVC Mode Work?
We have already touched a bit on this, however, let’s explain this in a bit more detail. We already know that the ESXi hypervisor is able to virtualize the CPU, memory, storage, etc. Since the hypervisor is abstracting the actual physical CPU from the virtual machine and controlling this through a layer of abstraction, this means that vSphere is able to control which CPU instructions are presented to the guest virtual machines.
VMware EVC Mode uses this as an advantage since it is able to set a “lock” of sorts on the features, functionality, CPU instructions, etc, that are presented to the guest virtual machines. This is known as an EVC Baseline. It allows all hosts in the vSphere cluster to advertise the same CPU instructions that are defined by means of VMware EVC Mode, regardless if some of the CPUs in the vSphere cluster are newer than others. An important point to note is that all your CPUs must be from the same vendor. VMware EVC Mode does not allow you to change an Intel CPU into an AMD CPU or vice versa.
When you enable EVC at the cluster-level, all hosts in the cluster are configured to present the CPU features of a user-selected processor type to all virtual machines running in the cluster. This allows the benefit of being able to ensure compatibility for the purposes of vMotion between ESXi hosts. Why is that important?
When you vMotion a live virtual machine from one host to another, you would run into issues if the target host has different CPUs configured that would be presented to the virtual machine. For compatibility, interoperability, and stability, these features need to be the same.
The advantage of this is that it ensures CPU compatibility for vMotion operations even though the underlying hardware might be different from one ESXi host to another ESXi host. Identical CPU features are exposed to virtual machines regardless of which host they are running on, so that the virtual machines can migrate between any hosts in cluster.
Benefits and Use Cases
Some might say, “why do I need this when I would certainly make sure the CPUs are identical when I configure my vSphere cluster ESXi hosts?”. In most greenfield deployments, this is most generally the case when the bill of materials is put together for the vSphere cluster hosts. Identical CPUs are chosen for the cluster hosts.
However, what happens two years later if you run out of compute capacity in your vSphere cluster and you need to scale out? You may not be able to build an exactly identical server build as you were when the cluster was originally configured. This means that most likely, any new servers will have slightly different hardware configurations, including the CPU configured.
The VMware EVC mode for the cluster would allow you in that case to set the specific CPU model type and instruction set that will be presented at the cluster level. New hosts that are added to the vSphere cluster would inherit those EVC settings and present only the specified CPU instruction set and features. This allows you to effectively scale out some time in the future without the worry of having different CPU configurations.
Another scenario comes to mind when VMware EVC Mode is extremely valuable and that is when you are refreshing your hardware in a vSphere cluster. You can effectively bring in your new hosts to the cluster. With EVC enabled at the cluster-level, you can migrate your VMs over to the new hosts without issues with newer CPUs. Then, after the VMs are migrated, you can deprovision the old ESXi hosts and then bring up the level configured for the VMware EVC Mode.
Enable VMware EVC Mode on a vSphere Cluster
Let’s now take a look at how you can enable the VMware EVC Mode on a vSphere cluster. The process is rather straightforward and involves configuring the setting in the vSphere Client for the vSphere cluster.
Navigate to the specific vSphere cluster in your inventory you want to change the VMware EVC mode on > Configure > Configuration > VMware EVC > Edit.
A helpful bit of PowerCLI code that can help with determining the maximum VMware EVC Mode setting is the following:
- Get-VMHost | Select-Object Name,ProcessorType,MaxEVCMode
On the Change EVC Mode screen, you will see the mode is set by default to Disable EVC. To enabled EVC, simply choose the radio button for your CPU vendor to enable it.
After you click the radio button, you will see a dropdown list for the different CPU families that are available for your specific CPU vendor. As you pick the specific EVC Mode you want to activate, the description will change accordingly. After you have settled on the EVC Mode you want, click OK to enable EVC for your cluster.
After you click OK, you will be taken back to the main VMware EVC screen which will show if EVC is enabled for the cluster or not and at what mode. You can also see Current CPUID Details.
If you try to enable VMware EVC Mode on a cluster that is lower than the CPU instructions set currently presented to any suspended or powered-on VMs, you will run into issues.
There are a few things to keep in mind when enabling VMware EVC Mode at the cluster-level.
- A VM does not recognize the features instituted by VMware EVC mode until the VM is powered off and then powered on. Simply rebooting the VM is not enough to instantiate the VMware EVC Mode on the VM
- When a host is removed from a VMware EVC Mode-enabled cluster, the host will revert back to its normal behavior and will present all of the CPU instructions that exist on the physical CPU installed to VMs that are moved to the host after its evacuation from the EVC-enabled cluster
- Keep in mind that you will have issues if you try to enable an EVC mode that is lower than the mode currently used by any suspended or powered-on VMs
Difference between cluster level EVC and VMware per-VM EVC
As mentioned, there are now two different scopes for which VMware EVC Mode can be enabled – at the vSphere cluster and at the VM-level. The VM-level implementation is new as of vSphere 6.7. It provides new functionality that allows your EVC implementation to be more granular at the vSphere virtual machine level as opposed to a cluster-level implementation.
When you have already implemented an EVC mode at the cluster-level, it can be difficult to change a cluster EVC baseline when you have live, running virtual machines. With the Per-VM EVC mode, the EVC implementation is an attribute of the virtual machine rather than the processor generation that it is booted on in the vSphere cluster.
There are a couple of requirements needed for VMware EVC Mode at the per-VM level to work. This requires:
- vSphere 6.7
- Virtual Machine Hardware Version 14 or higher
- VM must be powered-off to change the per-VM EVC
The per-VM EVC attributes are saved in the virtual machine VMX file. The following lines are added like the following:
featMask.vm.cpuid.Intel = “Val:1”
featMask.vm.cpuid.FAMILY = “Val:6”
featMask.vm.cpuid.MODEL = “Val:0x4f”
featMask.vm.cpuid.STEPPING = “Val:0”
featMask.vm.cpuid.NUMLEVELS = “Val:0xd”
Enabling Per-VM EVC Mode
Once you have met the requirement of vSphere 6.7 and a hardware version level of 14 or higher for your VM, you can enable the Per-VM EVC mode.
Right click your VM > Configure > VMware EVC > Edit.
Enable the mode for your CPU vendor, and change your Per-VM EVC Mode setting and click OK.
If you have many different settings that include both cluster-level EVC and Per-VM EVC mode, what is a good way to check your EVC Mode?
How to Check EVC Mode
Outside of checking EVC in the vSphere Client, PowerCLI offers the best solution to quickly gain visibility to the current EVC mode. There are a couple of very useful PowerCLI one-liners that make this possible.
There is a script provided by Kyle Ruddy with VMware that easily pulls the Cluster-EVC setting, the VM-level EVC setting and hardware version of your VMs.
Get-VM | Select Name,HardwareVersion,
@{Name=’VM_EVC_Mode’;Expression={$_.ExtensionData.Runtime.MinRequiredEVCModeKey}},@{Name=’Cluster_Name’;Expression={$_.VMHost.Parent}},@{Name=’Cluster_EVC_Mode’;Expression={$_.VMHost.Parent.EVCMode}} | ft
Using the script, you can easily see the VM names, the current hardware versions, the VM EVC Mode, cluster name, and Cluster EVC Mode. This is extremely helpful to be able to easily view the information in one location.
How to Change and Turn Off EVC Mode
There are some considerations to be made when you change your EVC mode of an existing EVC-enabled cluster. Prerequisites need to be met for the operation to be successful and there are different requirements depending on if you are raising or lowering the level of EVC in your EVC-enabled cluster.
These include:
- Verify the hosts in your vSphere cluster have supported CPUs for the EVC mode you want to enable
- Verify all ESXi hosts are connected and registered in vCenter Server. Disconnected hosts are not allowed to enable EVC
| EVC Mode | Virtual Machine Power Action |
|---|---|
| Raise the EVC mode to a CPU baseline with more features. | Running virtual machines can remain powered on. New EVC mode features are not available to the virtual machines until they are powered off and powered back on again. A full power cycling is required. Rebooting the guest operating system or suspending and resuming the virtual machine is not sufficient. |
| Lower the EVC mode to a CPU baseline with fewer features. | Power off virtual machines if they are powered on and running at a higher EVC Mode than the one you intend to enable. |
Turning off EVC will mean that you simply toggle off the VMware EVC Mode in the properties of the cluster.
To assume the new CPU instruction set that may be presented by turning off the EVC Mode, your VMs, if running, will need to be powered off to pick up the new CPU changes. A reboot is not sufficient enough to refresh CPU settings.
VMware vCenter Server Version Needs to be Considered
There is some consideration that needs to be given to your vCenter Server version. While the technology relates to the CPU baseline that is assigned to the ESXi hosts in the vSphere cluster, the version of vCenter Server has to be new enough to “understand” these baselines that are available for assignment.
Below is a graph from VMware outlining some of these considerations for EVC Mode according to the version of vCenter Server you are running. Currently, vCenter Server 6.7 and higher has full capabilities to work with the latest and greatest EVC modes that are available in the current CPU architecture.
How Does VMware EVC Mode Affect Performance?
VMware has posted a study as to their findings of how performance is affected with various VMware EVC Mode setting changes with different types of applications.
Overall, most applications were not affected much at all. VMware found that a large class of mainstream workloads including databases and Java applications showed little to no impact in performance. There were other applications and use cases that may not fare quite as well. This included AES-encryption and video encoding.
As many are aware, Intel released AESNI enhancements in Westmere and onwards which allow major improvements in encryption related application use cases. Customers would need to be aware if their workloads ran applications that rely on encryption technologies, selecting an EVC baseline prior to Westmere would result in a drop in performance.
Multimedia workloads running on EVC Mode baselines of “Merom” showed a minor performance loss due to losing the SSE4.1 instruction set. For the most part however, there is no significant drop in performance by stepping down to older processor generations.
You can check out the official report from VMware on EVC Mode Performance impact on Applications here.
Disadvantages and Final Thoughts
VMware EVC Mode allows you to scale your vSphere environments in ways that would not be possible otherwise due to the differences in CPU technologies that naturally come about after short periods of time. Even when new ESXi hosts with newer CPUs are added to a vSphere cluster, VMware EVC Mode allows masking newer CPU technologies so VMs can be vMotioned between the older hosts and newer hosts.
There are mixed opinions on whether or not you configure VMware EVC Mode on a cluster right from the start. We have talked about the advantages it brings for scalability. With every upside, there is generally a downside. However, in my opinion, these are minimal compared to the advantage of scale and interoperability you get.
As a disadvantage, you are losing CPU capabilities that you would otherwise have access to via newer hardware. Most do not want to pay for new CPUs and not be able to use the new features and instruction set improvements they are paying for. However, when you enable EVC Mode, you are essentially doing this. As VMware’s report shows however, the loss in performance for most workloads are negligible if no impact.
As another disadvantage, you will see more than negligible impacts on workloads that utilize encryption or multimedia encoding. Newer Intel CPUs included instruction sets that provided a significant performance boost in these areas. This is something that you want to make sure to be aware of when setting the EVC baseline to a level below where these improvements were introduced.
All in all, the EVC Mode technology is a great solution from VMware that allows you to scale vSphere clusters as needed and still have the ability to vMotion between hosts that have different generations of processors. What you lose in top-end performance from a new CPU by setting an older baseline is offset by the scalability and interoperability improvements you gain.
With new enhancements with vSphere 6.7 with the Per-VM EVC Mode, you can further tweak the ability to implement the EVC baselines at the VM level. This allows you to have more flexibility in particular VM levels when compared to the cluster level.
There are some things to be considered with VMware EVC mode that you want to be aware of as discussed such as the need to power off your VMs after the change is made on a cluster with running VMs. Also, be aware that you cannot lower the EVC Mode below the level currently being used in the cluster if VMs are running. You will have to power off all VMs first, and then lower the level of the EVC Mode baseline.
Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.
