When we think about VMware backups and other mechanisms that allow third party backup solutions to have hooks into the vSphere environment, none of those capabilities would be possible without the vSphere Storage APIs. In today’s world of application development and hooks into vendor software, restful APIs are all the rage and allow for a powerful way to expose the underlying functionality in software. VMware has been using restful APIs for interacting with storage for quite some time. Let’s take a look at the specific VMware vSphere Storage APIs that allow for third party integration especially with backup solutions, hardware offloading, and other future features of VMware storage integration – VMware vSphere Storage API – Data Protection (VSA-DP), VMware vSphere Storage APIs Array Integration (VAAI), and VMware APIs for I/O filtering.
VMware vSphere Storage API (VSA-DP)
The heart of VMware backup integration is based on the VMware vSphere Storage APIs which is part of VMware’s Data Protection framework. These APIs are responsible for performing centralized off-host LAN free backup of VMware virtual machines. The availability of this API allows backup vendors to utilize centralized backup capabilities without having the need for backup agents loaded inside the virtual machine. Also, this offsets the processing of the backup operation onto another server besides the ESXi host which makes the VMware backup operation less costly from an ESXi host performance impact perspective.
Additionally, these APIs allow the use of leveraging VMware snapshots to be able to backup virtual machines without any disruption or backup window. All of these benefits allow for no business impact to end users or applications running inside virtual machines.
The Data Protection framework APIs from VMware were produced to interact with the vSphere Data Protection which is a backup and recovery solution designed for VMware. Back in April 2017, VMware announced that it was ending the availability of Data Protection. VMware has made clear that vSphere 6.5 will be the final release to include vSphere Data Protection and future vSphere releases will no longer include Data Protection.
However, VMware says their focus will instead be on “Storage APIs – Data Protection” to further strengthen the vSphere backup partner relationships. Instead of stifling the integrations from backup vendors, this should help to open up the integration from very innovative backup companies who are pushing the envelope working with the storage APIs from VMware.
How do we interact with vSphere Storage APIs? VMware is able to preset the vSphere API interface as a web service that is running on VMware vCenter server. This API interface is what provides the access to the vSphere specific management components that allow the management and interaction with virtual machine functions such as snapshots, etc.
VMware vSphere Storage APIs Array Integration (VAAI)
The VAAI or array integration specific APIs are powerful APIs that allow for the offloading of storage processing and duties from vSphere Storage to the hardware array itself. As you can imagine, this offloading to hardware greatly improves performance for those storage processing functions that can be offloaded. This includes both block level storage technologies such as Fiber Channel and iSCSI but also includes NFS. Depending on the storage hardware vendor, such VAAI directives as the following may be supported by storage array integration:
- Full or XCOPY storage operation
- Block Zero
- Hardware assisted locking
- ATS heartbeat
- Unmap
- Compression
Note how the datastores shown below display the Hardware Acceleration status:
To check the status of VAAI and whether or not it is enabled or not you can use the ESXi command line using the command:
- esxcli storage core device list
- Look for the VAAI status entry
VMware APIs for I/O Filtering (VAIO)
The newer VAIO framework allows for third party vendors to use software defined data services allowing them to integrate directly with a filter plugin to the I/O path to and from virtual disks. This allows the vendor to implement new features based on policy. This I/O filter plugin allows partners to intercept and filter I/O data in transit between virtual disks and guest operating systems. Some of the use cases today would be things such as VM replication especially where we no longer have to rely on external components or other kernel modules to allow this functionality as often it is implemented today. A few other use cases that exist and may not exist as of yet using VAIO:
- Caching
- Deduplication
- Encryption
A conceptual Look at VAIO and how it is implemented into the vSphere Storage stack – Image courtesy of VMware
The VAIO filter framework combines a user world and VMkernel infrastructure provided by ESXi. The filters execute in a certain order such as the replication filter executes before the caching filter. An added benefit is since the filter is running in the user space, filter failures are virtual machine specific and do not affect ESXi host.
Final Thoughts
VMware’s commitment to providing powerful Storage APIs that allow third party vendors to build on and innovate the vSphere platform. This allows for exciting new products and extended feature sets on existing products and technologies. The VMware vSphere Storage APIs allow modern backup solutions to integrate and provide API driven backup solutions. The VAAI APIs allow storage hardware vendors to integrate with vSphere and offload vSphere storage functions to very powerful and capable hardware devices. The newer VAIO API framework is providing a very interesting new approach for third party vendors to create filters that intercept I/O streams at the kernel level and perform certain functions and very interesting use cases. VMware’s restful Storage APIs are certainly a powerful means of interacting with vSphere storage and allow the possibility of new and exciting features and functionality on the horizon.
Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.
