Thursday, June 10, 2010

Setting up the Xen-kernel for Virtual Machines

Since many days, I've been trying hard to install xen on various architectures using various methods.. but with a very low success rate. Here are the steps that worked for me (These steps are taken from here, with some modifications):

1. Take a machine with 64 bit architecture. To check the config of a machine:
$uname -a
If the line contains i386 or i686, the machine is 32-bit. If it contains x86_64, the machine is 64-bit.
(Alternatively, you can do:
$egrep -c ' lm ' /proc/cpuinfo
If the result is 1 or higher, it is a 64-bit machine; if it is a 0, it is not a 64-bit machine)


2. Install Ubuntu 8.10.

3.
$sudo su
Configure network, sources, etc.

4.
$apt-get install linux-image-server linux-server
$apt-get install ubuntu-xen-server build-essential libncurses5-dev gawk mercurial

$mkdir -p ~/build/linux-2.6.27-xen
$cd /usr/src/
$hg clone http://xenbits.xensource.com/ext/linux-2.6.27-xen.hg

$cd linux-2.6.27-xen.hg
$make O=~/build/linux-2.6.27-xen/ menuconfig

5.
In the kernel configuration menu, make sure that you select the following options:
General setup ---> Choose SLAB allocator (SLUB (Unqueued Allocator)) ---> (X) SLAB
Processor type and features ---> Subarchitecture Type (PC-compatible) ---> (X) Enable Xen compatible kernel
Bus options (PCI etc.)  ---> [*] PCI support
                             [*]   Xen PCI Frontend
                             [ ]     Xen PCI Frontend Debugging (NEW)
Networking support ---> Networking options ---> <*> 802.1d Ethernet Bridging
Device Drivers ---> [*] Network device support ---> [ ] Ethernet (10000 Mbit) --->
Make the Xen section look as follows (make sure you select Xen version compatibility (3.0.4 and later) instead of Xen version compatibility (3.0.2 and later)):
Device Drivers  ---> XEN  ---> [*] Privileged Guest (domain 0)
                               <*> Backend driver support (NEW)
                               <*>   Block-device backend driver (NEW)
                               <*>   Block-device tap backend driver (NEW)
                               <*>   Network-device backend driver (NEW)
                               (8)     Maximum simultaneous transmit requests (as a power of 2) (NEW)
                               [ ]     Pipelined transmitter (DANGEROUS) (NEW)
                               < >     Network-device loopback driver (NEW)
                               <*>   PCI-device backend driver (NEW)
                                       PCI Backend Mode (Virtual PCI)  --->
                               [ ]     PCI Backend Debugging (NEW)
                               < >   TPM-device backend driver (NEW)
                                  SCSI backend driver (NEW)
                                Block-device frontend driver
                                Network-device frontend driver
                                  Network-device frontend driver acceleration for Solarflare NICs (NEW)
                                SCSI frontend driver (NEW)
                               <*> User-space granted page access driver (NEW)
                               <*> Framebuffer-device frontend driver (NEW)
                               <*>   Keyboard-device frontend driver (NEW)
                               [*] Disable serial port drivers (NEW)
                               <*> Export Xen attributes in sysfs (NEW)
                               (256) Number of guest devices (NEW)
                                   Xen version compatibility (3.0.4 and later)  --->


6. After this, you need to do the following:
Open the file /usr/src/linux-2.6.29.2-xen/arch/x86/kernel/time_32-xen.c

$vim /usr/src/linux-2.6.29.2-xen/arch/x86/kernel/time_32-xen.c

Add the following line (highlighted) to the code at around line 500:
/* System-wide jiffy work. */
 if (delta >= NS_PER_TICK) {
         do_div(delta, NS_PER_TICK);
         processed_system_time += delta * NS_PER_TICK;
         while (delta > HZ) {

         asm("":"+r"(delta));

         do_timer(HZ);
         delta -= HZ;
    }
    do_timer(delta);
 }

(Note: if you don't add this line in the code, then while building the kernel in the next step, you may see too many warnings flashing on the screen and the build process may terminate after throwing an error.
Note2 (imp): The above step applies to the linux-2.6.29.2-xen sources. I have tried it on linux-2.6.27-xen and it worked. It may not apply to later releases. If you have a later release, try skipping this step)


7.
$make O=~/build/linux-2.6.27-xen/
$make O=~/build/linux-2.6.27-xen/ modules_install install

$depmod 2.6.27.5
$update-initramfs -c -k 2.6.27.5

$update-grub


8.
$vim /etc/modules

 and append the line:
loop max_loop=64

9.
$reboot

And after the reboot,
$uname -r
should show the xen kernel version (2.6.27.5).

10. After this you may think it is all done. Thats right, except that networking might have been screwed up!
If that is the case, continue with the following steps:
$apt-get remove network-manager

11.
$vim /etc/network/interfaces
and add the following lines:
auto eth0
iface eth0 inet static
        address         <dom0 IP>
        gateway         <a relevant gateway>
        netmask         <a relevant netmask>
        broadcast       <a relevant broadcast address>
You may get most of this info by doing 'ifconfig' on a similar machine.

12.
$vim /etc/resolv.conf
and add an entry for the DNS server you use:
nameserver <nameserver-IP>
you may add more than one entries.

Further, you may want to try out the following:
1. Create and Start Virtual Machines
     - this is a very good reference for creating image based virtual machines.
2. Migrating Virtual Machines
     - see this.


Other Ref:
http://www.infohit.net/blog/post/running-xen-on-ubuntu-intrepid-and-jaunty.html


10 comments:

Sue said...

linux-2.6.29.2-xen is probably the current xen repo. So the bug fixing that you have suggested related to time_32-xen.c will only hold valid until the next xen repo rollout.

The next roll-out may be bug-less or have different bugs. So it may make this post obsolete, unless you put a note about this somewhere...?

Dhaval said...

True! Thanks for pointing that out, Sujesha!
I have added a note there now.

You've been really helpful during all this, thanks... :-)

Yogesh Bagul said...
This comment has been removed by the author.
Yogesh Bagul said...

Yogesh:
Hi, I have install xen as per the steps given here, but after a half booting it got following error :
input: AT Translated Set 2 keyboard as
> >/devices/platform/i8042/serio0/input/input4

Any suggestion about this issue?
Thanking in Anticipation,

Dhaval said...

Hi Yogesh,
Are you using ubuntu 8.10? I too have seen this error but with ubuntu 9.10..

Yogesh Bagul said...

Actually I am using ubuntu 8.10 but it was 32 bit version.
I think this may be the problem.
but after this I again install xen on ubuntu 10.04 -amd64 everything goes fine.
No problem with Dom0 installtion.
but at the time of domU installation
it hangs at
adding 102545K swap on /dev/xvda2 ...
Can u Suggest something on this?
Or I will have to install Ubuntu8.10 64 bit version?

Vaibhao Tatte said...

Yogesh:

For the steps given by Dhaval, you MUST use a 64 bit version of Ubuntu 8.10.
Reason - The Xen source code that you get from http://xenbits.xensource.com/ext/linux-2.6.27-xen.hg is of 64 bit & not of 32 bit. So it a 64 bit Xen source so not compile with 32 bit OS.

For 10.04, you are providing 64 bit OS, so dom0 should be compiled. But you have to make sure that Kernel Image is appropriate (initrd.img & vmlinuz). I think if you do a use these images with little extra care, then you should be able to use in 10.04.

~Vaibhao

PS - Ubuntu has removed support for 8.10, so it become nightmare to use it withour FTP support. So if possible try to use th latest version of Ubuntu.

Vaibhao Tatte said...

Just little bit more clarification...

"But you have to make sure that Kernel Image is appropriate (initrd.img & vmlinuz). I think if you do a use these images with little extra care, then you should be able to use in 10.04."

This is for domU (guest) installation. You have to use "appropriate" image kernel image. Please don't ask me what is appropriate. I am also not very clear about it. :P
I have wasted my lot of stage 1 nights in compiling & installing with wrong Kernel Images.

Yogesh Bagul said...

@vaibhao: Thanks for Explanation
I tried a lot but not able to find suitable kernel for domU.
So for now I just install Ubuntu 8.10 and have some experiment on it. then possible after everything settle right I will again work on it.

Yogesh Bagul said...

Actually I am facing one more problem :
when I do ssh from my Dom0 to DomU then for domU it gives error of
forget password but I am sure that this is misleading.
have you face such problem?