Xen-test-debian

Installing Xen basic packages in Debian
First of all, update the APT repository

apt-get update

Xen Hypervisor
Install the Xen hypervisor. Note that the version may have changed: search for the current one using 'apt-cache search xen-hypervisor'.

apt-get install xen-hypervisor-3.0.3-1-i386

In the case you get "CPU0 panic" errors when booting this hypervisor, try with the PAE version instead:

apt-get install xen-hypervisor-3.0.3-1-i386-pae

Install the Xen kernel
apt-get install linux-image-xen-686

Install Xen tools
apt-get install xen-tools

Install the libc6-xen
It is not mandatory, but very recommendable (otherwise, you will get "4gb seg fixup" error messages in dom0):

apt-get install libc6-xen

Grub boot configuration
Kernel boot configuration should be upgraded when installing linux-image-xen-686 package. However, this automatic upgrade doesn't seem to work properly, so manual editing of /boot/grub/menu.lst is required (Debian has shown this behaviour; however, other distributions, like Ubuntu, seems not being suffering it).

Considering the linux-image-xen-686 installed /boot/vmlinuz-2.6.18-4-xen-686 (exact version may change, so check it), you should configure the following booting option in your /boot/grub/menu.lst file:

title          Debian GNU/Linux, kernel 2.6.18-4-xen-686 root           (hd0,0) kernel         /xen-3.0.3-1-i386-pae.gz   module          /vmlinuz-2.6.18-4-xen-686 root=/dev/sda2 ro   module          /initrd.img-2.6.18-4-xen-686 savedefault boot

Configure networking
I needed to edit /etc/xen/xend-config.sxp and configure the network-script option, in order virtual machines get directly bridged to the external interface (eth1 in my case, change it appropriately):

(network-script 'network-bridge netdev=eth1')

Configure loop
If you want to run many virtual machines simultaneously (more than 8) you have to use max_loop option for loop module (http://lists.xensource.com/archives/html/xen-users/2006-06/msg01150.html). For example:

rmmod loop modprobe loop max_loop=64

You can also configure as default option for modprobe, editing /etc/modprobe.d/options and adding the following line. If the file doesn't exist, create it. (Thanks to Ramon Casellas for the tip :)

options loop max_loop=64

Creating a virtual machine
Use the following configuration file (let's say test.cfg):

kernel = '/boot/vmlinuz-2.6.18-4-xen-686' ramdisk = '/boot/initrd.img-2.6.18-4-xen-686' memory = '50' root   = '/dev/sda1 ro' disk   = name   = 'occ0-v' vif    = [ '' ] on_poweroff = 'destroy' on_reboot  = 'restart' on_crash   = 'restart'

Comments:

 You can use the VNUML filesystems as xen_fs. It is highly recommendable to install kernel modules before. The commands are (exact version may change, so check it):

mount /usr/share/vnuml/filesystems/xen_fs /mnt/loop -o loop cd /mnt/loop/lib/modules cp -va /lib/modules/2.6.18-4-xen-686. cd / umount /mnt/loop

Warning: : differently from the usual configuration with VNUML, this filesystem doesn't work in COW mode, all modification performed during virtual machine operation will be stored in the xen_fs. In order to avoid changes in the root_fs file, I recommend you to use a copy ('cp root_fs xen_fs') and not to use directly the one you are using for your COW-ed based VNUML scenarios.

The vif line configures a network interface in the virtual machine (named 'eth0'). I haven't already checked, but it is supposed that additional vif lines will add additional interfaces (eth1, eth2, etc. This configuration uses 50MB of memory This configuration does not use swap memory



In order to boot the machine (note that you don't have to specify the complete pathname of test.cfg if it is located in a default directory; it seems that xm looks in /etc/xen by default).

xm create test.cfg -c

It also can take a while to boot the virtual machine. The '-c' is to get the booting output and loging in the same console where xm is invoked (this is very useful to know when the virtual machine has been booted). To come back to the host enviroment just press Ctrl+].

Uselful management operations
To list active domains:

xm list

To connect to the virtual machine console:

xm console 

To finish with the virtual machine:

xm destroy 

To get performance info:

xm top

Troubleshooting


If you have problems related with memory when booting virtual machines, you may find useful the dom0_mem kernel switch. Edit your GRUB configuration so your kernel line looks like this:

kernel /boot/xen-3.0.3-1-i386.gz dom0_mem=512M

changing 512 for the desired memory in your case, of course. Note that if you use a dom0_mem value too close to your actual physical RAM memory you will get a panic error when booting the host: as a hint dom0_mem<= phy_mem-32M (assuming the Xen hypervisor consumes less than 32MB).

If you run Xen in a VMware virtual machine using NAT, you should use an special MAC prefix for your Xen virtual machine: 00:0C:29. Edit the /etc/xen/test.cfg file and edit the vif line so looks (for example):</li>

vif = [ 'mac=00:0C:29:11:22:33' ]

the problem is quite similar to the one described in the following post in the vnuml-users list: https://lists.dit.upm.es/pipermail/vnuml-users/2005-September/000204.html

If you get the following error when creating a new virtual machine:</li>

Error: I need 65536 KiB, but dom0_min_mem is 200704 and shrinking to 200704 KiB would leave only 43236 KiB free.

This happens because of the sum of memory assigned to virtual machines is exceeding the limit reserve to dom0. To reduce that limit, edit the dom0-min-mem parameter in /etc/xen/xend-config.sxp file. For example, in order to use a 96MB limit:

(dom0-min-mem 96)

You need to restart xend in order changes in /etc/xen/xend-config.sxp take effect:

/etc/init.d/xend restart

Virtual machines seem to work properly (i.e., you can use 'xm console' to access the domU and issue commands without problems), but after a while (even at the very beginning of create them) network start to fail and they loss connectivity. In addition, the following error message start to appear in /var/log/messages in dom0:</li>

xen_net: Memory squeeze in netback driver.

After some googling and dead ends (see the references gathered in the bug report at http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=762), the final solution was found in the following thread: http://lists.xensource.com/archives/html/xen-users/2007-01/msg00428.html.

In conclusion, you have to use dom0_mem hypervisor boot switch (see the first bullet in this section). If you are using dom0-min-mem (see previous bullet), then make dom0-min-mem = dom0_mem.

Strange messages appear in the virtual machine console:</li>

4gb seg fixup, process syslogd (pid 1107), cs:ip 73:b7efc5f0 4gb seg fixup, process syslogd (pid 1107), cs:ip 73:b7efc570 4gb seg fixup, process klogd (pid 1113), cs:ip 73:b7e2e5f0 4gb seg fixup, process klogd (pid 1113), cs:ip 73:b7eafd10 printk: 38 messages suppressed. 4gb seg fixup, process named (pid 1128), cs:ip 73:b7bc5c1c

The problem is described in the Xen FAQ. It is not only a "aesthetic" issue: it impacts performance.

This seems to be due processes running in domU are using a wrong version of libc:

domU:~# ldd /sbin/init | grep libc.so.6 libc.so.6 => /lib/tls/libc.so.6 (0xb7e0f000)

The solution is to install a right version of libc in the domU filesystem (not only in dom0 filesystem, as described above), in particular the libc6-xen package.

After that, you can check that now processes are using the right version of libc in domU:

domU:~# ldd /sbin/init | grep libc.so.6 libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e36000)

If you get the following error when trying to create a domU:</li>

Error: Device 0 (vif) could not be connected. Backend device not found.

and your /var/log/xen/xend-debug.log shows something complaining about brctl, like that:

/etc/xen/scripts/network-bridge: line 114: brctl: command not found

it seems that bridge-utils package is not installed in your host, so:

apt-get install bridge-utils /etc/init.d/xend restart

should solve the problem.

If you get the following error when trying to create a domU (note that it's similar to the former bullet, but different):</li>

Error: Device 2049 (vbd) could not be connected. Backend device not found.

you maybe have not load the loop module. Use 'lsmod | grep loop' to check it. In that case, load loop ('modprobe loop') and try again.

</ul>

Uselful references
 http://www.debian-administration.org/articles/396</li> http://wiki.xensource.com/xenwiki/XenNetworking</li> </ul>