What do you think this is?

Just thoughts of a restless mind...

Moving KVM/QEMU VMs

I often have multiple virtual machines for different purposes. Some I keep idle, some I destroy and recreate when needed, and some I use for actual work and they stay there forever.

Recently I ran out of space. It was not really bad planning on my side, I just didn't have more space to allocate at the moment. I got a new disk to dedicate just to VMs. The new disk is faster but it's not very big; if I'm not careful I will run out of space again.

I decided to move some virtual machines; just the ones that have a GUI. The server based ones that are not interactive can stay in the old and slightly slower disk.

Moving a VM in QEMU is not very difficult, but if you're a GUI kind of person and work with virt-manager (which is the suggested GUI) you will be disappointed. Unlike other virtualization products such as VirtualBox that have the ability to just "move" a VM to a different location, virt-manager doesn't offer this option.

I had my old virtual machines under a partition, mounted under /VMS (nothing to do with VMS or openVMS obviously, but I expect most of my readers won't even know what that means). I want to move them in the new disk that I will mount under /newvms.

If I wanted to move them all, the proper way would be to just move the files and change the fstab entry. Since this is not the case, let's see how you move one of them.

First thing you want to do is to shutdown your virtual machines. And as always, verify :

virsh list --all

You need to make sure the machine is shut off. The output should look like :

image0 You also need to check which storage pools you have defined - just to make sure.

virsh pool-list --all

that will give you the defined pools

image1 I will start by moving Kali, so I need to know what disks (volumes) it uses. The method I explain here works for disk devices of type "file". Disks and CDROMs can be moved this way. If you want to "move" -actually redirect in that case- raw partitions or network devices, then the process is slightly different, and in many ways simpler.

Let's see which volumes we will need to move. You can do so by looking at the virtual machine definition.

virsh dumpxml --domain Kali | grep "source file"

You will get as many lines as storage devices you use.

image2 It is easy to assume that the storage under /VMS/mk/Kali is in the Kali storage pool, but always verify

virsh pool-dumpxml --pool Kali | grep path

and as expected, they match.

image3 You can also check the volume path

virsh vol-list --pool Kali

and again, we can confirm that they match.

image4 So far we're only checking that we know what we need to move. Next, we need to create new storage pools on the new disk which will be mounted under /newvms

Although one might think that the command to create a new storage pool would be virsh pool-create, that is wrong. pool-create only generates transient storage and our use case calls for persistent storage. So the right command is pool-define. pool-define takes an XML file as an input; the command that takes parameters instead of an xml definition is pool-define-as.

Let's create our new storage pool - obviously you need a new name. You also need to define the type and the path that needs to exist. I prefer to keep my virtual machines separated, for proper housekeeping. This is not necessary, it's just how I like it; the result is that I create one storage pool per VM.

mkdir /newvms/Kalinew virsh pool-define-as Kalinew dir --target /newvms/Kalinew

The output would be a simple success message

image5 Next step is to make sure it auto starts, otherwise the VMs will fail to start.

virsh pool-start Kalinew virsh pool-autostart Kalinew

The output would be again simple success messages

image6 Of course verify

virsh pool-list | grep Kali

image7 Three more things to do:

First move the file in the storage pool. I always start with a copy until I confirm that it's working co a simple "cp" command will do.

Now you need to restart libvirt so that the new pool is read and scanned.

service libvirtd restart virsh vol-list Kalinew

If everything looks ok, you need to change the VM definition, and this again is done with virsh

virsh edit Kalinew

Find the line that refers to the old file / disk. It should start with: <source file=

Change the path to point to the new location of the file. Do that for all the files that you have moved. Make sure that the new locations are visible after the virsh vol-list command, otherwise go back and check again your process.

After you confirm that it works, by starting the VM itself and making sure that the volume / storage file that is modified and accessed is the new one, you probably want to release the space.

You do that by these two commands:

virsh pool-destroy Kali virsh pool-undefine Kali

Then just "rm" the old files and you're done.

You just successfully moved the storage volumes, such as disks and cdroms that were used by your virtual machine, to a new location.