As we have seen, Libvirt is a tool that provides API management for multiple hypervisors, including Kernel-based Virtual Machine). Managing storage with a KVM virtual environment is important to the overall infrastructure. We will consider different storage types with KVM, such as iSCSI, NFS, GlusterFS, and Ceph. We will also look at virtual disk images and formats to use with KVM.

KVM Storage Overview

KVM can use many different storage backends to meet the storage needs of virtualized environments. The different storage types provide options for different use cases. Admins can choose the best storage solution for their environment depending on their needs and what kinds of workloads they are hosting.

Protect Your Data with BDRSuite

Cost-Effective Backup Solution for VMs, Servers, Endpoints, Cloud VMs & SaaS applications. Supports On-Premise, Remote, Hybrid and Cloud Backup, including Disaster Recovery, Ransomware Defense & more!

Let’s look at:

  • iSCSI (Internet Small Computer Systems Interface)
  • NFS (Network File System)
  • GlusterFS (now deprecated)
  • Ceph

iSCSI

When you think of an enterprise storage technology for block storage, most will no doubt think about iSCSI. It has been around for quite some time. It is a storage protocol that sends SCSI commands over IP networks. It quickly began to replace Fiber channel storage networks since it can use the same type of network hardware and technologies as IP-based networks. It is often seen used in conjunction with virtualized environments and storage for VMs.

Advantages of iSCSI

Note the following advantages of iSCSI storage solutions:

Download Banner
  • Flexibility: iSCSI can be used over existing IP networks, meaning you don’t need a dedicated storage network
  • Scalability: It supports large storage pools, which makes it suitable for scaling your environment
  • Performance: iSCSI can offer high performance using modern high-speed and high-bandwidth networks

Disadvantages of iSCSI

iSCSI storage may see the following disadvantages:

  • Network Dependency: iSCSI performance can be significantly affected by network latency and congestion
  • Complex Configuration: Setting up and managing iSCSI targets and initiators can be complex, especially in large environments
  • Security Concerns: ISCSI traffic must be properly secured. It is vulnerable to interception and attacks without proper security measures

Configuring iSCSI with Libvirt

Install iSCSI Initiator:

sudo apt-get install open-iscsi

Discover and Login to iSCSI Target:

sudo iscsiadm -m discovery -t sendtargets -p
sudo iscsiadm -m node -T -p –login

Verify Connection:

sudo iscsiadm -m session -o show

Add a storage pool for iSCSI:

virsh pool-define-as iscsi_pool iscsi –source-host –source-path
virsh pool-start iscsi_pool
virsh pool-autostart iscsi_pool

NFS

Like iSCSI, NFS is also very common. It is a distributed file system protocol allowing remote directories to be shared over a network. NFS provides file-based access to virtual machines with a hypervisor.

Advantages of NFS

Note the following advantages of NFS storage solutions:

  • Ease of Use: NFS is easy to configure and manage
  • Cost-Effective: It uses existing network infrastructure, so like iSCSI, you don’t need a dedicated environment
    • >Compatibility: NFS is supported by many guest operating systems and hypervisors

Disadvantages of NFS

NFS storage may see the following disadvantages:

  • Performance Overhead: NFS can introduce additional overhead, impacting performance, especially in high I/O operations
  • Single Point of Failure: NFS servers can become a single point of failure if not configured with high availability
  • Scalability Issues: NFS may struggle with scalability in environments requiring large-scale, high-performance storage

Configuring NFS

Install NFS Client:

sudo apt-get install nfs-common

Mount NFS Share:

sudo mkdir -p /mnt/nfs
sudo mount -t nfs :/path/to/share /mnt/nfs

Add a storage pool for NFS:

virsh pool-define-as nfs_pool netfs –source-host –source-path /path/to/share –target /mnt/nfs
virsh pool-start nfs_pool
virsh pool-autostart nfs_pool

GlusterFS

This type of storage is no longer recommended for greenfield installations. RedHat Gluster Storage will reach end-of-life in 2024. Using GlusterFS as a storage type for Storage Domains is now deprecated for use. However, it is worth mentioning as there are likely still deployments out there built on top of the solution.

Advantages of GlusterFS

Note the following advantages of GlusterFS:

  • Scalability: GlusterFS can scale out by adding more storage nodes
  • High Availability: It supports data replication and self-healing
  • Performance: It is optimized for large-scale data storage and access

Disadvantages of GlusterFS

  • Deprecated: GlusterFS is now deprecated and not recommended for use

Configuring GlusterFS

Install GlusterFS Client:

sudo apt-get install glusterfs-client

Mount GlusterFS Volume:

sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs :/volname /mnt/glusterfs

Configure GlusterFS Storage in Libvirt:

virsh pool-define-as gluster_pool gluster –source-host –source-path volname –target /mnt/glusterfs
virsh pool-start gluster_pool
virsh pool-autostart gluster_pool

Ceph

What is Ceph? It is a distributed file system. It provides an object store that allows organizations to run hyperconverged infrastructure (HCI). Ceph admins have a single solution containing block, object, and file storage. These features make it a versatile choice for businesses looking to satisfy their storage needs in virtualization environments.

Advantages of Ceph

Note the following advantages of Ceph

  • Scalability: Ceph can scale out to thousands of nodes.
  • Reliability: It ensures data redundancy and high availability.
  • Performance: Ceph is optimized for high performance in large-scale deployments.

Disadvantages of Ceph

You may see the following disadvantages working with Ceph storage:

  • Complex Deployment: Deploying and configuring a Ceph cluster can be complex and time-consuming.
  • Resource Requirements: Ceph requires substantial hardware resources, including CPU, memory, and storage
  • Maintenance Overhead: Regular maintenance and monitoring are required to ensure optimal performance and reliability.

Configuring Ceph with Libvirt

Install Ceph Client:

sudo apt-get install ceph

Configure Ceph Authentication:

sudo ceph auth get-key client.admin > /etc/ceph/ceph.client.admin.keyring
sudo chmod 600 /etc/ceph/ceph.client.admin.keyring

Configure Ceph Storage:

virsh secret-define –file secret.xml
virsh secret-set-value –secret –base64
virsh pool-define-as ceph_pool rbd –source-host –source-name pool –auth-username libvirt

virsh pool-start ceph_pool
virsh pool-autostart ceph_pool

Virtual Disk Images and Formats for KVM

KVM supports various virtual disk image formats, each with its advantages and use cases. The most common formats include:

  • QCOW2 (QEMU Copy On Write v2): It has features like snapshots, compression, and encryption. This format is the one seem most often with the KVM hypervisor
  • RAW: The RAW format is simple and uncompressed. It has great performance but lacks advanced features of other disk formats
  • VMDK (Virtual Machine Disk): This format is widely known as used by VMware but is supported by KVM for compatibility
  • VHD (Virtual Hard Disk): This format is used with Microsoft Hyper-V but also supported by KVM

Choosing the Right Disk Format

Note the following when choosing which disk format to use in your virtualization environment. Each has its strengths:

  • QCOW2 – Use this disk format in environments where snapshots and efficient storage is needed
  • RAW – If you have the need for performance-sensitive applications with direct disk access, RAW is best
  • VMDK/VHD – These formats provide the best format for mobility and interoperability of virtual machines between different hypervisors

Wrapping up

Libvirt provides many different storage solutions for your virtualization environment. Many viable KVM storage solutions exist, including iSCSI, NFS, GlusterFS, and Ceph. These allow admins to create storage solutions for their KVM-based virtual environments.

These have advantages and disadvantages, allowing admins to choose which solution fits their needs. The different virtual disk images and formats allow you to optimize your storage for guest operating systems that serve different purposes. With these different storage solutions, admins can create scalable and reliable storage for VMs and other workloads.

Related Posts:

KVM for Beginners – Understanding Libvirt Networking – Part 4

Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.

Rate this post