Home » Featured, Headline, Kernel, Linux, Tuning

Patching the linux kernel without rebooting

12 March 2009 One Comment
Applying kernel patches without reboot

Applying kernel patches without reboot

I think, everyone from Linux industry, knows about kexec (fast rebooting … actually not rebooting just switching kernels) or Pannus (live kernel patching, but right now the project is dead) and their use. The Linux community started very earlier to think at a method to switch or patch a live kernel, but this things should work only on a very good kernel with very good memory and process isolation … but this was achieved some time ago, we have a very good and mature Linux kernel (in my opinion was perfect from 2.4.2x) what can run for years without a reboot (my record is 4 years on a system with kernel 2.6.8 and this kernel is far to be perfect). This days I found a tool what is a gold mine for system administrators : KSplice.

What is ksplice ?

Ksplice is the first practical technology for updating the Linux kernel without rebooting.

So right now we don’t need to reboot our production machines just for a simple security patch. Now we can do it with the kernel and processes running. So our uptime will increase and our clients will be satisfied and happy. Thank you Kspice ! This is a very very good tool for servers what are running critical services.

What you can do:
Patching the running kernel if the patch doesn’t come with semantic structure modifications

What you can’t do:
Replace the entire kernel and you can’t switching from a kernel version to other just applying patches.

KSplice supports just x86-32, x86-64, and ARM architectures and have been tested on Linux kernel versions ranging from 2.6.8 to 2.6.28-rc6.

This how-to is totally based on ksplice example documentation page what you can find it here : http://www.ksplice.com/example-update.

This how-to requirements:

  • laterst Ubuntu update (Ubuntu 8.10)
  • Linux kernel sources from ubuntu repository (2.6.27-7)
  • Gcc C compiler 4.2.3
  • and an amd_x64 processor

The first step is to install the linux source code and linux headers (I think here is .config) from Ubuntu repository

root@randombugs:~# apt-get install linux-source linux-headers-2.6.27-7 linux-headers-2.6.27-7 bzip2 build-essentials

Don’t forget to check for the version compiler what was used for our kernel. To do that just open /proc/version and there you will find the gcc version.

root@randombugs:~# cat /proc/version
Linux version 2.6.27-7-generic (buildd@crested) (gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu11) ) #1 SMP Tue Nov 4 19:33:06 UTC 2008

In our case is gcc 4.3.2.

After installing you will have Linux source tree packed in /usr/src. Just unpack them:

root@randombugs:~# bzcat linux-source-2.6.27.tar.bz2 | tar -xf –

ksplice is based on the System.map and Linux configuration and you need to find them in /boot

root@randombugs:~# ls -Al /boot/
-rw-r–r– 1 root root 85316 2008-11-04 22:53 config-2.6.27-7-generic
[ … lines skiped … ]
-rw-r–r– 1 root root 1352144 2008-11-04 22:53 System.map-2.6.27-7-generic

If you have them then we can start the next step to create the environment for ksplice (System.map is coming directly with the kernel but config file is in linux-headers)

root@randombugs:~# mkdir -p /usr/src/linux-source-2.6.27/ksplice
root@randombugs:~# cp /boot/System.map-2.6.27-7-generic /usr/src/linux-source-2.6.27/ksplice/System.map
root@randombugs:~# cp /boot/config-2.6.27-7-generic /usr/src/linux-source-2.6.27/ksplice/.config
root@randombugs:~# cp /boot/config-2.6.27-7-generic /usr/src/linux-source-2.6.27/.config

Now we get a Linux patch (I used the printk patch provided by ksplice team and you can get it from here http://www.ksplice.com/doc/printk.patch) and apply it.

root@randombugs:~# ksplice-create –patch=printk.patch /usr/src/linux-source-2.6.27

Right now your kernel it will start compiling and if it succeed you will get a message something like:

Ksplice update tarball written to ksplice-1d3i75an.tar.gz

Now let’s apply the patch to the running kernel. Just type following command, CLOSE YOUR EYES, and press Enter.

root@randombugs:~# ksplice-apply ./ksplice-1d3i75an.tar.gz

Now open your eyes and look at what stupid thing you done. Ok … retry without closed eyes because you don’t need to be nervous … is really working without problems.

If you used the exactly the same patch you will find in dmesg or /var/log/syslog this message:

Quoth the kernel: [ … and your kernel message … ]

Or if you don’t get this message then try to generate one.

root@randombugs:~# modprobe loop

Now look in syslog or dmesg again.

Ok … so far so good, but the story doesn’t end here. Is possible to want to reverse the patch. And this also is a nice feature. You can see the applied patches and you can reverse only the patch what you want.

To see the patches applied just run:

root@randombugs:~# ksplice-view

If you don’t remember the id of the patch and you don’t know what patch it is then you can see the patch itself.

root@randombugs:~# ksplice-view –id=1d3i75an
Ksplice id 8c4o6ucj is present in the kernel and is applied.

Here is the source code patch associated with this update:
— linux-2.6/kernel/printk.c 2008-06-28 22:26:15.000000000 -0400
+++ linux-2.6-new/kernel/printk.c 2008-06-29 00:19:16.000000000 -0400
@@ -609,6 +609,7 @@
va_list args;
int r;

+ vprintk(“Quoth the kernel:n”, NULL);
va_start(args, fmt);
r = vprintk(fmt, args);

Now we know what patch we need to reverse and we just run the following command to reverse it:

root@randombugs:~# ksplice-undo 1d3i75an
root@randombugs:~# dmesg
[ … striped lines … ]
ksplice: Update 1d3i75an reversed successfully

Good luck with your kernel patches !
P.S.: Is still anyone believe in Windows 7?

One Comment »

  • Julie G said:

    Thanks so much for posting this… I think you just saved me a lot of time and a HUGE headache!! I am excited to see if I can do this alone… we will see! 🙂

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.