Tuesday, June 5, 2012

How is linux kernel tested?


How do the Linux kernel developers test their code locally and after they have it committed? Do they use some kind of unit testing, build automation? test plans?



Source: Tips4all

4 comments:

  1. The linux kernel has a heavy emphasise on community testing.

    Typically any developer will test their own code before submitting, and quite often they will be using a development version of the kernel from Linus, or one of the other unstable/development trees for a project relevant to their work. This means they are often testing both their changes and other people's changes.

    There tend not to be much in the way of formal test plans, but extra testing may be asked for before features are merged into upstream trees.

    As Dean pointed out, there's also some automated testing, the linux test project and the kernel autotest.

    Developers will often also write automated tests targetted to test their change, but I'm not sure there's a (often used) mechanism to centrally collect these adhoc tests.

    It depends a lot on which area of the kernel is being changed of course - the testing you'd do for a new network driver is quite different to the testing you'd do when replacing the core scheduling algorithm.

    ReplyDelete
  2. How do the Linux kernel developers test their code locally and after they have it committed?

    Do they use some kind of unit testing, build automation?


    In classic sense of words, no.

    E. g. Ingo Molnar is running the following workload:
    1. build new kernel with random set of config options
    2. boot into it
    3. goto 1

    Every build fail, boot fail, BUG or runtime warning is dealt with. 24/7.
    Multiply by several boxes, and one can uncover quite a lot of problems.


    test plans?


    No.

    There may be misunderstanding that there is central testing facility, there is none.
    Everyone does what he wants.

    ReplyDelete
  3. Its not very easy to automate kernel testing. Most Linux developers do the testing on their own, much like adobriyan mentioned.

    However, there are a few things that help with debugging the Linux Kernel:


    kexec: A system call that allows you to put another kernel into memory and reboot without going back to the BIOS, and if it fails, reboot back.
    dmesg: Definitely the place to look for information about what happened during the kernel boot and whether it works/doesn't work.
    Kernel Instrumentation: In addition to printk's (and an option called 'CONFIG_PRINTK_TIME' which allows you to see (to microsecond accuracy) when the kernel output what), the kernel configuration allows you to turn on a LOT of tracers that enable them to debug what is happening.


    Then, developers usually have others review their patches. Once the patches are reviewed locally and seen not to interfere with anything else, and the patches are tested to work with the latest kernel from Linus without breaking anything, the patches are pushed upstream.

    Edit: Here's a nice video detailing the process a patch goes through before it is integrated into the kernel.

    ReplyDelete
  4. I would imagine they use virtualization to do quick tests, something like QEMU, VirtualBox or Xen, and some scripts to perform configurations and automated tests.

    Automated testing is probably done by trying either many random configurations or a few specific ones (if they are working with a specific issue). Linux has a lot of low-level tools (such as dmesg) to monitor and log debug data from the kernel, so I imagine that is used as well.

    ReplyDelete