Quick Bites:
- VMware vSAN is a software-defined storage solution that allows you to pool together multiple storage devices and create a single shared datastore that can be used by all virtual machines running on a vSphere cluster
- To configure vSAN, you need to first enable it on your vSphere cluster, create a vSAN storage policy, and assign it to your virtual machines
- You can configure vSAN using either the vSphere Web Client or the vSphere Client, depending on the version of vSphere you are using
- When configuring vSAN, you need to pay attention to factors like the number of hosts in your cluster, the amount of storage available, and the level of redundancy you want to achieve
- Proper configuration and management of vSAN can help you improve storage performance, reduce storage costs, and simplify storage management in your vSphere environment
One of the software-defined technologies that have gained tremendous momentum in the past couple of years is software-defined storage. Most hardware and software companies are jumping on the bandwagon to offer software-defined storage solutions. Software-defined storage offers great benefits for housing your production virtualized workloads.
The dominant leader in the enterprise data center when it comes to software-defined storage is VMware vSAN.
VMware vSAN is continuing to pick up many new customers with each new release. The features and functionality with each new release of vSAN continue to raise the bar for competitors to the product.
In this post, we will take a deeper look into VMware vSAN.
- What is vSAN cluster in VMware?
- How does it work?
- What are the licensing, host, storage, and networking requirements?
- Configuration of vSAN 6.7 Update 2
- Pros & Cons of running a VMware vSAN storage solution
Table of Contents
- What is vSAN Cluster in VMware?
- Architecture of VMware vSAN
- How Does VMware vSAN Work?
- VMware vSAN Deployment Options
- VMware vSAN 6.7 Licensing
- VMware vSAN Licensing Considerations
- VMware vSAN Requirements
- vSAN Hardware Requirements
- vSAN Cluster Requirements
- vSAN Software Requirements
- vSAN Networking Requirements
- Other vSAN Network Considerations
- vSAN License Requirements
- How to Configure vSAN 6.7
- Backup VMware vSAN Virtual Machines
What is vSAN Cluster in VMware?
VMware was a pioneer of the Software-Defined Data Center (SDDC) concept which it coined back in 2012 to virtualize all aspects of the data center including compute, network, and storage. VMware vSAN was a product of this new way of thinking with an official release of vSAN in 2014.
The software-defined data center is the new mindset moving forward that allows your business to be as agile, automated, and abstracted from underlying hardware infrastructure as possible. As one of the foundational pillars of your SDDC, software-defined storage is at the heart of your data.
VMware vSAN is VMware’s software-defined storage solution that allows automation and pooling of storage through a software control plan, and the ability to provide storage from industry-standard servers. By using automation and storage pooling, this allows abstracting the hardware from the storage solution and easily pooling resources between servers.
This allows your environment to gain many benefits using VMware vSAN, including easily scaling up and scaling out as a mechanism that is built right into the vSAN solution and in essence, is now built into the vSphere solution in general as part of the hypervisor itself and vCenter Server.
VMware vSAN allows you to move beyond the traditional concept of storage LUNs that were extremely labor-intensive to change, grow, or reconfigure. In times past, this generally meant destroying the original LUN and re-provisioning a new LUN with new characteristics.
With VMware vSAN, this can all be achieved in the software-defined architecture of the vSAN storage model. In addition to easily changing your storage configuration, VMware vSAN allows a powerful new way to control storage down to the VMDK level of a virtual machine. This is referred to as Storage Policy-based Management (SPBM).
When a virtual machine is provisioned you can choose a VM storage policy to support the application that is running on a particular virtual machine. With the appropriate storage policy selected, SPBM makes sure the virtual machine is allocated the resources needed and is provisioned on the correct tier of storage configured based on the performance and other characteristics.
Architecture of VMware vSAN
VMware vSAN is an object-based storage solution. Each component of the virtual machine is created as an underlying virtual machine object. Each object is an individual block storage device. Objects that make up the VM stored in vSAN include VMDKs, VM swap object, VM namespace, VM snapshot/delta disk object, etc.
Components are pieces of the VM objects that are stored on a particular cache or capacity device. The maximum number of components per host limit is 9000.
How are the objects and components laid out physically across the vSAN environment?
VMware vSAN takes care of the layout of vSAN objects automatically and bases the placement decisions based on a number of factors including:
- vSAN verifies that the virtual disk requirements are applied according to the specified virtual machine storage policy settings.
- vSAN verifies that the correct cluster resources are used at the time of provisioning.
For example, based on the protection policy, vSAN determines how many replicas to create. The performance policy determines the amount of flash read cache allocated for each replica and how many stripes to create for each replica and where to place them in the cluster. - vSAN continually monitors and reports the policy compliance status of the virtual disk
With the new vSphere Client, VMware provides great visibility into the layout and location of the objects in the vSAN environment.
You can find this information in the vSphere Client by navigating to Monitor > vSAN > Virtual Objects and selecting an object, then clicking on the View Placement Details.
After clicking the View Placement Details, you can see the location of the various components making up a VMware virtual machine housed in VMware vSAN.
This provides many tremendous benefits when thinking about configuring and provisioning the vSAN solution. It is a native part of VMware vSphere. As mentioned previously, vSAN is integrated into the hypervisor as a kernel-based solution. This means there are no additional installations that are required or needed. It is part of vSphere.
How does VMware vSAN provide the availability and performance of the software-defined storage solution in a vSphere environment?
It uses a distributed RAID methodology to protect your data, providing RAID protection over the network between the hosts in the vSAN cluster.
This allows VMware vSAN to withstand failures of the underlying infrastructure and still maintain the resiliency of your data. This could be anything from a failed disk drive (cache or capacity) or a failure in the network (network card, etc). The amount of resiliency of VMware vSAN comes down to configuring the Failures to Tolerate (FTT). This setting defines the amount of availability desired by the customer.
VMware vSAN achieves the availability desired as specified in the Failures to Tolerate by creating replica copies of your data. This mitigates the risk of a host failure or other failure in a portion of the underlying infrastructure so that data is continuously available. With the way vSAN creates replicas for availability, this will mean more storage will be used for the resiliency setting chosen.
The most basic form of Failures to Tolerate is the FTT=1 configuration setting. This setting provides the mirrored (RAID1) resiliency functionality to your data. Below is an example of increasing the FTT value in the configuration and the resiliency this provides.
FTT Configuration
FTT=0 – No resiliency
FTT=1 – (n+1)
FTT=2 – (n+2)
FTT=3 – (n+3)
With all-flash vSAN, customers can achieve much greater space savings due to a number of technologies that can be utilized with the all-flash vSAN configuration. What are those technologies?
- Erasure coding – Two other RAID types included by vSAN, RAID-5 and RAID-6, that save on capacity usage by using a distributed parity mechanism instead of using mirrors.
- Deduplication and Compression – can be enabled on flash vSAN only but results in a tremendous space savings
VMware vSAN since vSAN 6.0 uses an on-disk format called vSANFS. The vSANFS file system is a result of an acquisition of VMware by VirstoFS. The new vSANFS on-disk format provided significant performance improvements to virtual machine snapshots. This is accomplished through the use of a sparse filesystem known as vsanSparse.
How Does VMware vSAN Work?
We have taken a look at a bit of the architecture of VMware vSAN and this is helpful when thinking about how VMware vSAN works. It uses a software-defined approach that creates shared storage for virtual machines. Local physical storage found on the ESXi hosts is turned into pools of storage that are used for virtual machine storage.
These virtual machine assignments are made according to the storage policy that was defined via the SPBM. Again, these can be assigned very granularly, down to the individual VMDKs of the virtual machines.
Within a VMware vSAN-enabled cluster, the vSAN cluster supports all of the VMware features of traditional external shared storage clusters such as HA, vMotion, and DRS without having the need for external shared storage.
VMware vSAN is created at the vSphere cluster level. They are created as either hybrid or all-flash clusters. The hybrid vSAN cluster is comprised of cache devices that are made up of flash storage and capacity devices that are comprised of traditional spindles. For the all-flash clusters, flash storage is used for both the cache and capacity tiers.
VMware vSAN Deployment Options
VMware vSAN provides a number of different deployment options when it comes to configuring a VMware vSAN-enabled cluster. What are those options?
- Standard vSAN Cluster – The standard vSAN Cluster contains a minimum of three hosts. These are usually in the same fault domain or in the same location and are connected to the same L2 network. VMware recommends 10 Gb networking for hybrid configurations and requires it for all-flash configurations.
- Two Host vSAN Cluster – The two host vSAN cluster is a unique configuration that is well-suited for ROBO type installations where a small number of workloads may reside. With the two-host configuration, you need to have a witness component that is recommended to live in a different environment or different site. The two-host witness component is created via a special, purpose-built “nested” ESXi host that runs as a VM inside a production vSphere environment. The witness component contains only metadata and is not used for any of the disk objects. The witness host will typically reside in the same location as the vCenter Server.
- vSAN Stretched Cluster – With a vSAN stretched cluster, VMware vSAN provides resiliency against the total loss of an entire site. The hosts are distributed evenly between the two sites, with a third witness component in a third site. With the stretched cluster, there need to be no more than 5 milliseconds of latency between the sites.
VMware vSAN 6.7 Licensing
While VMware vSAN is built into the ESXi hypervisor, you must license VMware vSAN to enable the solution at the cluster level in vSphere. The VMware vSAN license is a special license that you assign to vSAN clusters. There are several different licensing levels for VMware vSAN. These include the following:
- Standard
- Advanced
- Enterprise
- Standard for ROBO
- Advanced for ROBO
- Enterprise for ROBO
Comparing the features of the different VMware vSAN licensed versions (VMware vSAN 6.7 Licensing Guide):
VMware vSAN Licensing Considerations
There are various considerations and items to note when looking at VMware vSAN licensing. These include
- Upgrades are allowed between existing vSAN license levels by purchasing an upgrade license
- The ROBO license levels provide great value for smaller environments that only run a few VMs between locations. The VMware vSAN ROBO license level is limited to 25 VMs.
- Two-node VMware vSAN configuration witness host has an embedded license from VMware which negates the need for you as the customer to purchase a special license just for vSAN witness capabilities.
VMware vSAN Requirements
There are many considerations to be made from a requirements perspective when provisioning a VMware vSAN software-defined storage solution.
Components required to get up and running with a vSAN enabled cluster
- vCenter Server – Centralized management tool
- ESXi hypervisor – Hypervisor with vSAN kernel-enabled modules
- Local physical storage – If there is a storage controller “in front of” the locally attached disks, it should be able to pass through the underlying physical disks so there is not a layer of RAID being provided by a storage controller on top of the disks. This is also known as JBOD functionality. If this is not possible, configurations of RAID0 in the storage controller can be supported by vSAN.
There are various other requirements for enabling VMware vSAN including the areas of:
- Hardware requirements
- Cluster requirements
- Software requirements
- Networking requirements
- License requirements
vSAN Hardware Requirements
These have been mentioned in earlier sections, but hardware requirements for the ESXi hosts include:
- Must have a cache and a capacity device
- Storage controllers must be able to do storage pass through or RAID 0 mode with a single disk
- Don’t use the same storage controller for vSAN devices and non-vSAN devices such as a boot disk
- You can mix and match SAS and SATA devices in a disk group – however, it is always recommended to stay uniform in storage hardware as this can lead to unexpected performance anomalies
- Memory requirements vary depending on the number of disk groups and devices the ESXi host has to manage
vSAN Cluster Requirements
The following are cluster-level considerations that must be made for VMware vSAN:
- All devices (firmware, storage, and otherwise) need to be listed in the VMware Compatibility Guide as compatible for use with VMware vSAN
- A standard vSAN cluster needs to have at least 3 hosts that are participating in vSAN and contributing storage capacity
- A host cannot participate in another cluster and a vSAN cluster at the same time
vSAN Software Requirements
With greenfield installations of VMware vSAN, using the latest versions of ESXi and vCenter Server allows you to take advantage of all the latest and greatest VMware vSAN features. If you have previous versions of ESXi and VMware vSAN, keep in mind the “on-disk” format version may be a previous version. You can keep a legacy on-disk format version, but will not be able to benefit from all the new features when keeping a legacy on-disk format. Upgrading to the latest version allows making use of the newest features and capabilities. VMware vSphere 6.7 Update 1 vSAN on-disk format version is “7.0”.
vSAN Networking Requirements
VMware vSAN is extremely network dependent. When you have a distributed RAID solution that provides storage that is aggregately pooled from multiple hosts, the way the data is synchronized and communicated is over the network. The key to any vSAN installation that is stable and performs well often comes down to the health of the network. This can’t be underscored enough – the network is critical to vSAN functionality.
Since the network is extremely important, paying attention to the best practices and recommendations from VMware for configuring the network with vSAN is key to avoiding performance and stability issues. As you can imagine, using the network to sling around storage packets means it needs to have enough bandwidth, low latency, and throughput to sustain the needed performance.
- Recommended host bandwidth – 1 Gb dedicated network interface for hybrid configurations and 10 Gb dedicated network for all-flash configurations
- vSAN VMkernel interface is created on all hosts that are part of a vSAN cluster, regardless – if the host is contributing storage capacity. This is used for vSAN communication between hosts.
- vSAN supports connections to either Layer 2 or Layer 3 networks
- IPv4 and IPv6 are both supported
- Supported network latency:
- Maximum of 1 ms RTT for standard (non-stretched) vSAN clusters between all hosts in the cluster
- Maximum of 5 ms RTT between the two main sites for stretched clusters
- Maximum of 200 ms RTT from the main site to the vSAN witness host
- Be sure to validate the NIC, firmware, and driver versions as compatible with VMware vSAN
Other vSAN Network Considerations
There are a few additional network considerations and items of interest with VMware vSAN. These include vSwitch types, NIC teaming, Jumbo Frames, and Network I/O Control.
- VMware vSAN supports the use of both the vSphere Standard Switch and the vSphere Distributed Switch, however, you get the VDS as part of the vSAN license. The VDS switch introduces many benefits when used in conjunction with vSAN. This includes the ability to use the Network I/O Control (NIOC) feature when using converged networking with more than one traffic type, including vSAN
- NIC teaming can be used for ensuring high availability and is certainly a best practice. Using teaming in conjunction with LACP can improve performance. Jumbo frames can provide performance and CPU efficiency benefits when used in vSAN. However, care needs to be taken to ensure the implementation of the Jumbo frame is configured uniformly across the environment.
- Network I/O Control can be used with the vSphere Distributed Switch. When you have more than one traffic type including vSAN traversing a single physical NIC, NIOC helps to ensure that vSAN can have the bandwidth needed when it is shared with vMotion and Virtual Machine traffic.
- VMware vSAN and NSX are compatible as they can coexist in the same vSphere infrastructure without any issues. However, it is not supported to place vSAN network traffic on an NSX managed VXLAN/Geneve overlay
- As of VMware vSphere 6.5, VMware supports direct network connections without a switch for vSAN connectivity
vSAN License Requirements
VMware vSAN requires a license to be installed for the vSAN enabled cluster. (See the section above covering vSAN licensing in detail).
How to Configure vSAN 6.7
Now that we have taken a look at many aspects of what VMware vSAN is, its requirements, and other aspects of the solution, let’s look at how to configure vSAN 6.7.
One of the really great features of VMware vSphere vSAN 6.7 is the ability to use the Quickstart workflow to configure vSAN.
What does the QuickStart Wizard do?
As part of the Quickstart workflow, when you create a new vSphere cluster in the vSphere Client, you will see a pop up that lets you set the cluster services, including vSAN, that you want to make use of in the vSphere cluster.
As you can see below, you can enable DRS, HA, and vSAN during the creation of the new vSphere cluster.
Once the cluster is created, you can continue the QuickStart workflow. Navigate to the properties of the cluster Configure > Configuration > Quickstart to find the Cluster Quickstart.
The Quickstart workflow consists of (3) steps:
- Cluster basics – Select services to enable on the cluster
- Add hosts – Connect to hosts that will be added to the new vSphere cluster and validate them according to VMware health checks
- Configure Cluster – Configure network settings for vMotion and vSAN traffic, review and customize cluster services, and set up a vSAN datastore
Under the second 2. Add hosts step, click Add and you will be presented with the Add hosts wizard.
Finish out the Add hosts wizard to add hosts to the new vSphere cluster in the Quickstart workflow.
As you can see, the hosts are added to the cluster. Additionally, you will see various health checks ran against the host. Any issues found are displayed in each box of the Quickstart workflow. The hosts added here are “nested” ESXi hosts so are showing hardware compatibility warnings.
Click Configure on the third step 3. Configure Cluster
The great thing about the Quickstart workflow is that it takes care of the complexity of setting up the networking for vSAN automatically for you, including migrating networking from vSphere Standard Switches to vSphere Distributed Switches, all from within the wizard!
The Configure cluster wizard launches. The first part of the wizard is configuring the Distributed Switches. You can optionally configure the networking later, however, the wizard makes this really easy. This consists of entering:
- Number of distributed switches
- Names of distributed switches
- Port group names and associated VDS
- Physical adapters and which VDS they are backing
Below, we are configuring (2) VDS’s with associated port groups attached to each VDS. One is used for Mgmt and one for Services.
The Quickstart Configure Cluster wizard takes care of addressing your vMotion and vSAN VMkernel ports for your hosts. You can fill in the IP address of one host and hit the Autofill button which sequentially populates IPs for you.
The same is done for vSAN.
The Advanced options screen allows configuring cluster services such as HA, DRS and your vSAN options such as encryption, deduplication, and compression, etc. Additionally, you can configure NTP and EVC settings as well.
On the Claim Disks screen of the Quickstart workflow, you claim your disks for vSAN.
After claiming the disks for vSAN, the Quickstart wizard is ready to complete. When you click Finish, it will start provisioning the options specified for the cluster, including vSAN.
As mentioned, one of the benefits of the Quickstart wizard is that it takes care of the networking configuration, even migrating your networking from VSS to VDS virtual switches as part of the process.
You will also start to see the disk groups created as part of the process.
The process to configure and create your vSphere cluster with services such as HA and DRS, and to have your vSAN configuration automatically created is surprisingly easy and quick! After only a few minutes you will have a vSAN datastore configured and ready to host workloads.
Wrapping Up – VMware vSAN Pros and Cons
VMware vSAN provides a robust, fully-featured, easily provisioned and configured software-defined storage solution for enterprise data centers. It is gaining huge momentum among enterprise customers and allows you to solve many challenging technological hurdles in the area of storage.
With all the powerful features provided by VMware vSAN, is it your business? Is there any reason at this point to be looking at anything besides software-defined storage solutions?
Let’s consider VMware vSAN pros and cons to consider how VMware vSAN compares against traditional SANs that most are familiar with.
Pros
- Storage solution and provisioning is inside of vSphere
- No involvement needed from a storage administrator
- Much easier configuration of storage than traditional SAN (Quickstart wizard is a few clicks to provision storage)
- Easily scales out and up – add storage capacity or nodes as needed
- Easy migration of data
- Distributed RAID provides many capabilities including stretched clusters across geographic sites
- Resiliency/space consumption can be adjusted based on Failures to Tolerate setting (mirror, RAID 5, RAID 6)
- All-flash vSAN clusters can use deduplication and compression which can reclaim much of the consumed space from replica data, etc
- Storage Policy-Based Management (SPBM) can provide granular control over tiering and performance of data down to the individual VMDKs of virtual machines
- Storage at the software layer allows great analytics features
- Support for Windows Failover Clusters
Cons
- If you have traditional SAN storage and aligned with a particular vendor, vSAN will represent a shift in processes and procedures operationally
- Many traditional SANs can provide SAN level replication which is an easy way to duplicate data between Prod and DR which is not a feature included in vSAN
- Resiliency of data and RAID level is determined by the number of hosts in the cluster.
- vSAN does not support vSphere DPM and Storage I/O Control
- vSAN does not support hosts participating in multiple vSAN clusters as of yet
Traditional SAN storage RAID levels are not limited by the number of hosts in the vSphere cluster
Backup VMware vSAN Virtual Machines
Even with the tremendous resiliency and availability features of VMware vSAN, you still must protect your production workloads with effective backups. Data loss can happen even with the best high-availability and software-defined storage solutions. BDRSuite provides a great way to protect your virtual machines running in VMware vSAN.
Download a fully-featured trial version of BDRSuite and try it out with your VMware virtual machines today.
Related Posts:
VMware vVOLs and vSAN – Overview, Similarities, and Differences
Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.
