Xen and loop devices continued

I created one more guest and that made them nine in total, but when I tried to power it on, I got the following error message:

# xm create node9
Using config file "/etc/xen/node9".
Error: Device 51712 (vbd) could not be connected. Failed to find an unused loop device

Once again problem with the disk image in combination with loop device but different than yesterday’s problem.
I checked which loop devices are used:

# losetup -a
/dev/loop0: [0811]:161693699 (/data/guests/node1.img)
/dev/loop1: [0811]:161693702 (/data/guests/node2.img)
/dev/loop2: [0811]:161693703 (/data/guests/node3.img)
/dev/loop3: [0811]:161693704 (/data/guests/node4.img)
/dev/loop4: [0811]:161693705 (/data/guests/node5.img)
/dev/loop5: [0811]:161693706 (/data/guests/node6.img)
/dev/loop6: [0811]:161693707 (/data/guests/node7.img)
/dev/loop7: [0811]:161693708 (/data/guests/node8.img)

And then how many are available in total:

# ls -l /dev/loop*
brw-r----- 1 root disk 7, 0 Mar  9 22:08 /dev/loop0
brw-r----- 1 root disk 7, 1 Mar  9 22:08 /dev/loop1
brw-r----- 1 root disk 7, 2 Mar  9 22:08 /dev/loop2
brw-r----- 1 root disk 7, 3 Mar  9 22:08 /dev/loop3
brw-r----- 1 root disk 7, 4 Mar  9 22:08 /dev/loop4
brw-r----- 1 root disk 7, 5 Mar  9 22:08 /dev/loop5
brw-r----- 1 root disk 7, 6 Mar  9 22:08 /dev/loop6
brw-r----- 1 root disk 7, 7 Mar  9 22:08 /dev/loop7

All eight loop devices are used. If each guest had a second disk image as well, I’d face the problem after node4. The problem resides to the fact that the default number of loop devices provided by the kernel are eight, and every Xen guest that doesn’t use the Xen blktp driver will use a loop device for every disk image it’s assigned with. The solution to this is to increase the number of loop devices. That can be done by editing /etc/modprobe.conf and adding a new definition for the maximum loop devices:

options loop max_loop=24

In order for that to take effect, you’ll need either to reboot the system or reload the loop kernel module. In order to reload the module, the current loop devices must be free, so every guest should be powered off. Then simply:

# rmmod loop
# modprobe loop
# ls -l /dev/loop*
brw-r----- 1 root disk 7,  0 Mar 24 12:25 /dev/loop0
brw-r----- 1 root disk 7,  1 Mar 24 12:25 /dev/loop1
brw-r----- 1 root disk 7, 10 Mar 24 12:27 /dev/loop10
brw-r----- 1 root disk 7, 11 Mar 24 12:27 /dev/loop11
brw-r----- 1 root disk 7, 12 Mar 24 12:27 /dev/loop12
brw-r----- 1 root disk 7, 13 Mar 24 12:27 /dev/loop13
brw-r----- 1 root disk 7, 14 Mar 24 12:27 /dev/loop14
brw-r----- 1 root disk 7, 15 Mar 24 12:27 /dev/loop15
brw-r----- 1 root disk 7, 16 Mar 24 12:27 /dev/loop16
brw-r----- 1 root disk 7, 17 Mar 24 12:27 /dev/loop17
brw-r----- 1 root disk 7, 18 Mar 24 12:27 /dev/loop18
brw-r----- 1 root disk 7, 19 Mar 24 12:27 /dev/loop19
brw-r----- 1 root disk 7,  2 Mar 24 12:25 /dev/loop2
brw-r----- 1 root disk 7, 20 Mar 24 12:27 /dev/loop20
brw-r----- 1 root disk 7, 21 Mar 24 12:27 /dev/loop21
brw-r----- 1 root disk 7, 22 Mar 24 12:27 /dev/loop22
brw-r----- 1 root disk 7, 23 Mar 24 12:27 /dev/loop23

Powering the guests and running losetup -a:

# losetup -a
/dev/loop0: [0811]:161693699 (/data/guests/node1.img)
/dev/loop1: [0811]:161693702 (/data/guests/node2.img)
/dev/loop10: [0811]:161693703 (/data/guests/node3.img)
/dev/loop11: [0811]:161693704 (/data/guests/node4.img)
/dev/loop12: [0811]:161693705 (/data/guests/node5.img)
/dev/loop13: [0811]:161693706 (/data/guests/node6.img)
/dev/loop14: [0811]:161693707 (/data/guests/node7.img)
/dev/loop15: [0811]:161693708 (/data/guests/node8.img)
/dev/loop16: [0811]:161693709 (/data/guests/node9.img)

Paravirtualised guests can make use of the blktp driver to access the virtual block device directly, without using a loop device. To do so, the guest’s configuration file must specify ‘tap:aio:’ instead of of ‘file’ at the disk entry. This is not true for full virtualised guests. However, both paravirtualsied and full virtualised guests can make use of physical partitions (defined as ‘phy’) and that eliminates the use of loop devices among other advantages.

Advertisement