Create Guest Clusters with Hyper-V Shared VHDX and VHD Sets

One of the key benefits of virtualized environments is the ability to build very highly available solutions for running business-critical workloads. Today’s hypervisors like VMware vSphere and Microsoft Hyper-V have high-availability features that can be enabled at the physical host cluster level that allows virtual machines running on top of the hypervisor to have a great degree of high-availability.

Table of Contents

1. What is Guest Clustering?
2. Requirements of WSFC Guest Clustering
3. Configuring Shared VHDX
4. Adding the Shared VHDX on the Second Guest Cluster Host
5. VHD Set
6. VHD Set Requirements
7. Configuring VHD Set Virtual Disks
8. Adding VHD Set Virtual Disk to the Second Guest Cluster Host
9. Converting VHDX to VHD Set File

Keying in on Microsoft’s Hyper-V hypervisor, there are other technologies that can be utilized to increase the high-availability of your most critical workloads even further. At the heart of Microsoft Hyper-V’s high-availability features are Windows Server Failover Cluster and the HA mechanisms it provides. Using this same feature in a secondary, “guest cluster” can allow increasing your high-availability. Microsoft has introduced a couple of key virtual disk technologies that help to make this particular high-availability mechanism even more powerful and easier to implement from a storage perspective.

In this post, we will take a look at these two storage-level features called Shared VHDX and VHD Sets.

What is Guest Clustering?

Before getting into the nuts and bolts of some of the technologies that can be used to make shared storage between guest virtual machines easier, why would you care about creating this configuration in the first place? Well, it all has to do with having the highest availability possible for your most critical applications.

A physical Windows Server Failover Cluster running Hyper-V allows the VMs that are running on the Hyper-V hosts to be restarted on healthy hosts in the event that a physical Hyper-V host fails. The VMs running on the failed host will be restarted on the remaining healthy hosts that are running in the Hyper-V cluster.

However, as you would imagine, the VMs that are restarted on healthy Hyper-V hosts will have the downtime that is required to be restarted on a healthy Hyper-V host. While this may only require a relatively short period of time, for workloads running very critical applications that may need to always be available, even a couple of minutes of unexpected downtime might be unacceptable.

It is important to understand the high-availability that is enabled by a physical Hyper-V cluster is at the VM level and not at the application level. This is a key difference to understand as some may have the misconception that your application will not fail when it is running on a physical Hyper-V cluster. This is not an accurate assumption when running a physical Windows Server Failover Cluster with Hyper-V. In fact, there are several scenarios that can result in downtime, including:

• One of the most common scenarios is if you have a physical Hyper-V host failure in the Hyper-V cluster. The failover behavior would kick in, however, it could take several minutes or longer before your VM comes back online, even on a healthy hosts. This can be affected by the load on the remaining healthy Hyper-V hosts, the priority of the VM relative to other VMs
• Applying critical application or security patches would require downtime to the underlying VM housing application
• Failed guest operating system that is housing the business-critical application. This could be the equivalent to a “blue screen” or other OS-specific error
• A failure in the actual application, causing the application to crash and perhaps cascading out to the VM

Since there are several situations that can result in downtime for your application, you need to have a process to protect you at the application level, which is beyond the mechanism provided by the physical Hyper-V cluster.

Guest Clustering is the process of creating a Windows Server Failover Cluster running as guest virtual machines inside your physical Hyper-V Cluster environment. While the physical Hyper-V cluster provides the high-availability at the VM level, the guest cluster that is created inside the physical Hyper-V cluster environment provides high-availability at the application level.

How does simply running a guest cluster on top of a physical cluster provide high-availability at the application level?

Aside from running a guest cluster inside of a physical Hyper-V cluster environment, your application must be compatible with the high-availability provided by Windows Server Failover Cluster services. Familiar applications that are compatible with the failover cluster are applications like Microsoft SQL Server.

The guest cluster hosts provide high-availability of the application. This is because the guest cluster hosts are hosting the application inside the physical Hyper-V cluster. If a physical Hyper-V cluster host goes down, only one of the guest cluster hosts will go down with the failed Hyper-V host. The other guest cluster host will assume running the “application” so the downtime related to the other guest cluster host going down will not affect the application.

Instead of minutes of downtime, running the guest cluster for your application may only result in a slight hiccup in the network for a moment and the application will continue running as expected. Of course, there are several considerations and configurations to implement to make sure this behavior is executed as expected such as anti-affinity rules for the guest cluster VMs running in the Hyper-V environment.

You may think that creating a cluster inside a cluster would add a lot of complexity to the underlying infrastructure of your application. You would be right. Guest clustering is certainly additional configuration to provide the same end-user experience during regular activity. However, where the guest cluster shines in during times of failures. The guest cluster allows providing that extra layer of resiliency to your application so that downtime is greatly minimized.

As you would imagine, there are requirements for implementing a guest cluster in Hyper-V. What are these requirements? How do the Shared VHDX and VHD Sets make satisfying these requirements easier?

Requirements of WSFC Guest Clustering

Running a guest cluster inside a physical Hyper-V cluster as guest VMs instead of physical Windows Server Failover Cluster hosts have the same requirements that you would expect for physical clusters. This includes at a bare minimum:

• Running at least two Windows Servers
• Running the same Windows Server version between the guest cluster hosts
• Having a cluster network that functions as a heartbeat network for the cluster
• Shared storage between the cluster nodes

Let’s see how advancements in Windows Server Failover Cluster Hyper-V technology is making the shared storage requirement much easier to satisfy.

Shared VHDX

When looking at the above requirements, shared storage often stands out as one of the absolutely required components that allows Windows Server Failover Clusters to work. With shared storage, multiple WSFC hosts have access to the same data. If there is a failure, there is no movement required to keep accessing data.

In traditional Windows Server Failover Clustering shared storage, physical servers generally use shared storage technologies such as iSCSI, Fibre Channel, and Fibre Channel over Ethernet (FCoE). While guest cluster hosts can technically use iSCSI targets for shared storage in the guest cluster, this can add yet additional complexity to the underlying configuration due to the need to configure such technologies as NPIV, virtual HBA and virtual SAN.

Starting with Windows Server 2012 R2, Microsoft added a new feature to assist customers wanting to make use of guest clusters, the Shared VHDX. The new Shared VHDX capability allows an easier way to get up and running with shared storage between guest cluster hosts.

There are a couple of supported configurations using the Shared VHDX feature for shared storage between guest cluster hosts.

• Configure a guest cluster on a physical Hyper-V failover cluster configured with a Cluster Shared Volume (CSV) on block-level storage (LUN). The shared VHDX used with the guest cluster is stored on the CSV stored alongside the other virtual machine files
• Configure a guest cluster on a physical Hyper-V cluster with Scale-Out File Server (SOFS) configured. The shared VHDX is stored on the SMB 3.0 share hosted on the SOFS. The VMs are hosted as roles on a separate Hyper-V cluster

Shared VHDX Limitations

The following limitations exist when making use of Shared VHDX:

• Backups of the VM at the Hyper-V host level
• Creating Checkpoints
• Storage migration is not possible for the shared VHDX
• Hyper-V replicas of the virtual machines connected to shared VHDX is not possible

Shared VHDX Requirements

The following are the requirements for configuring the Shared VHDX storage technology:

• The virtual machines need to be powered off to add the Shared VHDX
• Requires at least Windows Server 2012 R2 Hyper-V or higher
• The guest operating system must be Windows Server 2012 or higher and be running the Integration services from Windows Server 2012 R2
• The disk type must be the VHDX type. VHD disks are not supported
• The functionality is not supported for operating system boot drives. Shared VHDX is only supported for data disks
• Cluster Shared Volume (CSV) or Scale-Out File Server (SoFS) backend storage is required for the shared VHDX to be functional
• NTFS does not contain the feature set to implement the Shared VHDX. If a non-CSV volume is used to force the attaching of a shared VHD filter driver it is only loaded until the disk is offline

Let’s take a look at how the Shared VHDX is configured and see how this can easily be set up between two guest cluster hosts running on top of a physical Hyper-V cluster.

Configuring Shared VHDX

The first step in configuring the Shared VHDX component on a Hyper-V virtual machine is navigate to Settings on the Virtual Machine, then click your SCSI Controller and in the Add section on the right-hand side of the Settings dialog box, you will see the Add button. Click the “Hard Drive” option and then click Add to start the New Virtual Hard Disk wizard.

This opens the New Virtual Hard Disk Wizard.

Next, choose the Disk Type. The options are:

• Fixed-size
• Dynamically expanding
• Differencing

Name the new virtual hard disk and choose the storage location. Note for a supported location, you need to target a Cluster Shared Volume. Below, we are not, to see the error that will result.

Configure the size of the new virtual hard disk.

Finalize the creation of the new virtual hard disk for use as a Shared VHDX.

Now that the new hard disk is created, to enable Virtual hard disk sharing, click the Advanced Features option under the new hard disk that is added. Place a check in the checkbox next to the Virtual hard disk sharing option. Note, you have to have the VM shutdown to add the shared VHDX.

If you attempt to enable sharing on a virtual hard disk that is housed on a location that is not supported, you will see the following error message:

The error helps to indicate the virtual disk with the intention of being used as a Shared VHDX solution is not housed on a supported platform, like Cluster Shared Volumes storage.

Adding the Shared VHDX on the Second Guest Cluster Host

After properly housing your virtual hard disk for sharing purposes you can move forward. On your second guest cluster host, browse to the shared VHDX.

You will need to Enable virtual hard disk sharing on the second guest cluster host as well.

VHD Set

New with Windows Server 2019, Hyper-V now supports a new kind of file that enhances the capabilities of guest clusters even further from a storage perspective. The VHD Set file is a virtual disk format that is intended for use with guest clusters. It is created with a new virtual hard disk file type .VHDS file extension. It also supports several features that are not supported by the older Shared VHDX technology.

Basically, the new VHD Set resolves the issues that were present with the Shared VHDX, namely:

• Hyper-V Replica support
• Online resizing
• Application-consistent checkpoints for backup purposes

Hyper-V handles all aspects of managing the checkpoint chains and merging the shared VHD set. Management software can run disk operations like online resizing on VHD Set files in the same way it does for .VHDX files. This means that management software doesn’t need to know about the VHD Set file format.

Out of the three features, for many, running the Shared VHDX without the ability to perform backups of the shared virtual disks was a deal-breaker. The new VHD Set shared virtual disk is able to divert file locks as

The new VHDS file type that has been introduced called the VDHS file allows diverting file locks as is typical with backup solutions that create snapshots and divert the data being written so the backup solution can copy the changed data. The VHDS file is only 260KB in size and has an automatically generated AVHDX file that holds the data of the VHD set. The VHD Set can be fixed or dynamic in size.

• VHDS file – This is the VHD Set file which is the file your virtual machines mount
• AVHDX file – This is the advanced VHDX file which is a checkpoint file where data is actually stored

VHD Set Requirements

The following are the requirements of the VHD Set guest clustering virtual disk technology:

• The virtual machines need to be powered off to add a VHD Set virtual disk
• The VHD Set format is only supported on the guest operating system running Windows Server 2016
• Compatible underlying storage for the VHD Set storage, i.e. Cluster Shared Volumes (CSV)

Configuring VHD Set Virtual Disks

Let’s walk through the steps of configuring a VHD Set file for guest clustering. On the settings page of your first guest cluster host, click the SCSI Controller and then select Shared Drive > Add.

Click New to begin the New Virtual Hard Disk Wizard.

The New Virtual Hard Disk Wizard begins.

On the Choose Disk Format page, select VHD Set.

Choose the Disk Type for the new VHD Set virtual disk.

Specify the Name and Location of the new VHDS virtual disk.

Configure the size of the VHD Set file.

Review the options of the new VHD Set file before creating and then click Finish.

The new VHD Set file is added to the first guest cluster host.

Adding VHD Set Virtual Disk to the Second Guest Cluster Host

Now that the VHD Set virtual disk has been created and added to the first guest cluster host, we need to add it to the second guest cluster host.

Open the Settings of the second guest cluster host, click the SCSI Controller > Shared Drive > Add.

Browse to the location of the VHD Set virtual disk that was created.

Choose the new .vhds virtual disk and click Open.

The new VHDS VHD Set virtual disk is added to the second guest cluster host. Click Apply.

Converting VHDX to VHD Set File

What if you have a Shared VHDX virtual disk that you would like to convert to a VHD Set? This can be done using the Convert-VHD cmdlet in PowerShell.

Convert-VHD -Path < source disk you want to convert >.vhdx -DestinationPath < destination path for VHDS file >.vhds

Once you run the Convert-VHD cmdlet, you can browse to your configured Disks directory on your Cluster Shared Volume and see the newly created (converted) VHD Set file pair (VHDS and AVHDX).

Once the conversion process is complete, you can add the newly created VHDS file to your guest cluster hosts by adding the Shared Disk under the SCSI controller of your Hyper-V guest cluster VMs.

Concluding Thoughts

High-availability in today’s extremely business-critical workload environments is becoming more important than ever. Technologies like Hyper-V high-availability allows virtual machines to run in a resilient and highly-available manner. However, when high-availability is needed at the application level in virtual environments, more is needed.

In this case, you can create a guest cluster of Windows Server Failover Cluster hosts running as virtual machines. This allows you to effectively provide resiliency for the application that is compatible with WSFC. After creating your guest cluster hosts, you run each guest VM on a different Hyper-V physical host. In case one host goes down, the other guest cluster host will assume ownership of the application.

One of the key requirements of WSFC is having shared storage. In Windows Server 2012 R2, Microsoft introduced the Shared VHDX virtual disk. This allowed a much easier approach for meeting the shared disk requirement. However, it had limitations like no effective way to backup the data, no resize capability and no possibility for Hyper-V Replicas.

In Windows Server 2016, an even better solution that solves those limitations was introduced. The VHD Set solves the limitations of the Shared VHDX, including the ability to have effective backups of the VHD virtual disk. By using the VHD Set with Windows Server 2016 and higher, businesses have an easy and effective way to create guest clusters for application high-availability.

Related Posts:

Windows Server 2019 Failover Clustering Types and Uses
Hyper-V Guest Clustering – Part 1 – Overview and Requirements
Hyper-V Guest Clustering – Part 2 – Steps to Create a Hyper-V Guest Cluster

Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.