Patching the linux kernel without rebooting
I think, everyone from Linux industry, knows about kexec (fast rebooting … actually not rebooting just switching kernels) or Pannus (live kernel patching, but right now the project is dead) and their use. The Linux community started very earlier to think at a method to switch or patch a live kernel, but this things should work only on a very good kernel with very good memory and process isolation … but this was achieved some time ago, we have a very good and mature Linux kernel (in my opinion was perfect from 2.4.2x) what can run for years without a reboot (my record is 4 years on a system with kernel 2.6.8 and this kernel is far to be perfect). This days I found a tool what is a gold mine for system administrators : KSplice.
What is ksplice ?
Ksplice is the first practical technology for updating the Linux kernel without rebooting.
So right now we don’t need to reboot our production machines just for a simple security patch. Now we can do it with the kernel and processes running. So our uptime will increase and our clients will be satisfied and happy. Thank you Kspice ! This is a very very good tool for servers what are running critical services.
What you can do:
Patching the running kernel if the patch doesn’t come with semantic structure modifications
What you can’t do:
Replace the entire kernel and you can’t switching from a kernel version to other just applying patches.
KSplice supports just x86-32, x86-64, and ARM architectures and have been tested on Linux kernel versions ranging from 2.6.8 to 2.6.28-rc6.
This how-to is totally based on ksplice example documentation page what you can find it here : http://www.ksplice.com/example-update.
This how-to requirements:
- laterst Ubuntu update (Ubuntu 8.10)
- Linux kernel sources from ubuntu repository (2.6.27-7)
- Gcc C compiler 4.2.3
- and an amd_x64 processor
The first step is to install the linux source code and linux headers (I think here is .config) from Ubuntu repository
root@randombugs:~# apt-get install linux-source linux-headers-2.6.27-7 linux-headers-2.6.27-7 bzip2 build-essentials
Don’t forget to check for the version compiler what was used for our kernel. To do that just open /proc/version and there you will find the gcc version.
root@randombugs:~# cat /proc/version
Linux version 2.6.27-7-generic (buildd@crested) (gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu11) ) #1 SMP Tue Nov 4 19:33:06 UTC 2008
In our case is gcc 4.3.2.
After installing you will have Linux source tree packed in /usr/src. Just unpack them:
root@randombugs:~# bzcat linux-source-2.6.27.tar.bz2 | tar -xf –
ksplice is based on the System.map and Linux configuration and you need to find them in /boot
root@randombugs:~# ls -Al /boot/
-rw-r–r– 1 root root 85316 2008-11-04 22:53 config-2.6.27-7-generic
[ … lines skiped … ]
-rw-r–r– 1 root root 1352144 2008-11-04 22:53 System.map-2.6.27-7-generic
If you have them then we can start the next step to create the environment for ksplice (System.map is coming directly with the kernel but config file is in linux-headers)
root@randombugs:~# mkdir -p /usr/src/linux-source-2.6.27/ksplice
root@randombugs:~# cp /boot/System.map-2.6.27-7-generic /usr/src/linux-source-2.6.27/ksplice/System.map
root@randombugs:~# cp /boot/config-2.6.27-7-generic /usr/src/linux-source-2.6.27/ksplice/.config
root@randombugs:~# cp /boot/config-2.6.27-7-generic /usr/src/linux-source-2.6.27/.config
Now we get a Linux patch (I used the printk patch provided by ksplice team and you can get it from here http://www.ksplice.com/doc/printk.patch) and apply it.
root@randombugs:~# ksplice-create –patch=printk.patch /usr/src/linux-source-2.6.27
Right now your kernel it will start compiling and if it succeed you will get a message something like:
Ksplice update tarball written to ksplice-1d3i75an.tar.gz
Now let’s apply the patch to the running kernel. Just type following command, CLOSE YOUR EYES, and press Enter.
root@randombugs:~# ksplice-apply ./ksplice-1d3i75an.tar.gz
Now open your eyes and look at what stupid thing you done. Ok … retry without closed eyes because you don’t need to be nervous … is really working without problems.
If you used the exactly the same patch you will find in dmesg or /var/log/syslog this message:
Quoth the kernel: [ … and your kernel message … ]
Or if you don’t get this message then try to generate one.
root@randombugs:~# modprobe loop
Now look in syslog or dmesg again.
Ok … so far so good, but the story doesn’t end here. Is possible to want to reverse the patch. And this also is a nice feature. You can see the applied patches and you can reverse only the patch what you want.
To see the patches applied just run:
If you don’t remember the id of the patch and you don’t know what patch it is then you can see the patch itself.
root@randombugs:~# ksplice-view –id=1d3i75an
Ksplice id 8c4o6ucj is present in the kernel and is applied.
Here is the source code patch associated with this update:
— linux-2.6/kernel/printk.c 2008-06-28 22:26:15.000000000 -0400
+++ linux-2.6-new/kernel/printk.c 2008-06-29 00:19:16.000000000 -0400
@@ -609,6 +609,7 @@
+ vprintk(“Quoth the kernel:n”, NULL);
r = vprintk(fmt, args);
Now we know what patch we need to reverse and we just run the following command to reverse it:
root@randombugs:~# ksplice-undo 1d3i75an
[ … striped lines … ]
ksplice: Update 1d3i75an reversed successfully
Good luck with your kernel patches !
P.S.: Is still anyone believe in Windows 7?