Ksplice is an amazing new technology which allows patches to be applied directly into a running kernel, without needing a reboot. Linux Magazine talks with co-founder and Chief Operating Officer and discusses the origins of the project and what it has to offer.
Ksplice is an advanced new project which enables patches to be applied directly into a running kernel, without the need for a reboot. The project has just announced a free service for users of Ubuntu Jaunty Jackalope which enables this right on the desktop. Linux Magazine had a chat with co-founder and Chief Operating Officer, Waseem Daher, about this exciting new system, what inspired the project and what it has to offer.
Christopher Smart: Could you tell us about Ksplice, what it is and how it works?
Waseem Daher: Sure. It’s technology that Jeff Arnold developed at MIT (Massachusetts Institute of Technology) as his Masters thesis, which he started working on in mid-2006, and it lets you update a running Linux kernel without requiring a reboot. The technical details are explored in detail in the paper available on the website, but at a high level, the interesting thing about Ksplice is that it’s technology that takes a traditional source patch that the developers on the LKML (Linux Kernel Mailing List) are already writing and produces a rebootless update. That’s really interesting because it makes this a very practical technology.
CS: Who started the project and what was the main motivation for it?
WD: Jeff, when he was a student and MIT, was administering servers and a new security update became available in the middle of the week. This was over the Summer and he had a Summer job and he said: “Look, I don’t want to reboot the system during the day because there are lots of users and I don’t want to disrupt them, so I’m going to wait until Sunday at 2am.” That’s basically standard practice for sysadmins of popular systems. But of course, lo and behold, a few days later, someone broke into the system and he had to re-install everything, which was a big pain.
So that really got him thinking: “Well, why is it fundamental that these kernel updates require a reboot at all? Why can’t we just do this modification on the fly?” And so he began tackling this as his thesis project, which later won the “Charles and Jennifer Johnson” thesis prize. When we all graduated in June 2008 with our Masters degrees from MIT, the four of us (Jeff Arnold, Waseem Daher, Tim Abbott and Anders Kaseorg) co-founded Ksplice the company to take this technology further, to take it from the lab and start to get it into the hands of the people who really need it.
CS: What are the main goals of the project and company?
WD: The long term vision is that, at the end of the day, all updates will be hot updates — updates that don’t require a reboot or an application restart. This is actually a big problem because if you look at technology used in data centers, no-one has a good solution for software updates, from as low level as your router or SAN, up to your virtualisation solution, the operating system, the database, and the critical applications. Right now, all these updates require you either to reboot the system or restart the service.
This is a big pain point for sysadmins because, on the one hand you have to apply the updates so that you can fix important security problems, but on the other if you don’t then you’re vulnerable. When you do apply them, though, there’s downtime and that’s lost productivity. There’s a real cost associated with the downtime. We want to take the technology that we’ve developed and use it to make life easier in the data center. That’s the broad vision for where we’re going with the company, and we’re starting with Linux.
CS: Ksplice seems to be geared more towards the server market, but end users can use it too. Do you envisage that desktop users will take it up as well?
WD: We’d definitely like them to! It’s an interesting technology and we hope that people find that it’s fun to use, because rebooting on the desktop is certainly an inconvenience.
CS: Speaking of which, you’ve released your Uptrack service which I installed on an Ubuntu Jaunty machine and it worked very well! However, I had no way of knowing whether the patches actually worked except that the GUI told me so. Do you think that an online service which can run these exploits might be useful to verify that your system is secure?
WD: In our FAQ (Frequently Asked Questions) we have a sample program that demonstrates one of the bugs fixed by our patches. We picked a relatively harmless example, because in general, the bugs that these patches fix could cause the system to lock up or lose data if theyâ€™re triggered.
CS: When I was running Ksplice on Ubuntu it provided updates to the running kernel which was version 2.6.28-11. Where were the patches for that coming from?
WD: The patches were coming from the Ubuntu Git source tree for their kernel. With Ksplice Uptrack, we will give you rebootless versions of all of the security updates that Ubuntu releases.
CS: So currently Ubuntu gathers patches but waits to roll them out in a new release such as 2.6.28-13.
WD: Yes, that’s right. In general, they also release “proposed” kernels before they actually end up releasing the real kernel, as a way of having people try them out beforehand.
CS: I noticed that after I had updated to the most recent kernel from Ubuntu, there were still several new patches that Ksplice wanted to apply. Does that mean that by default Ubuntu kernels are more vulnerable?
WD: Those patches are commits in the Ubuntu Git tree which have security and reliability impacts, that they haven’t yet released a new kernel for. Our goal is to track what Ubuntu is releasing very closely, because it’s desirable to know that what you’re getting with the rebootless version is what you’d be getting with the traditional kernel update as well.
CS: When a user runs the system update in Ubuntu sometimes they do get that new kernel, but the system tells them that they need to reboot in order to get those latest updates (even though with Ksplice they actually don’t). Should they reboot anyway, Ksplice is then re-run against that new kernel and will continue to apply patches, correct?
WD: Yes. To be clear: if you’re running Ksplice Uptrack, you never need to reboot for kernel security fixes — we will provide you with rebootless versions of all current and future updates. However, you still need to continue getting traditional updates from Ubuntu because they are providing updates to your user-space software, and we’re not. That said, if you want to reboot into a newly-released Ubuntu kernel, you can still do that too, and Ksplice will pick up from there and apply any patches for it directly.
CS: And then if you do reboot with the new kernel, then you get the latest patches from Ksplice anyway.
WD: Yes, that’s exactly right.
CS: Do you guys have any plans to work directly with Canonical to incorporate Ksplice into all kernel updates so that any patches and updates can be applied in realtime without needing to get that updated kernel all the time? This could simply do away with the old traditional update method.
WD: I’d rather not comment on the specifics of our plans, but one thing that we are doing is beginning to roll out support for other distros like RHEL (Red Hat Enterprise Linux) commercially. The model there is that companies would pay us a monthly fee, like they already pay Red Hat for support, to get these updates in rebootless form.
CS: If any companies are interested in this service, who should they contact? When do you think this will be available?
WD: They should definitely get in touch with us — the contact information is available on our website. We’re currently starting to deploy with a select set of customers, on distributions like RHEL, and you should expect to see the general availability of the service by the end of this year.
CS: What about userspace applications? Do you have any plans to begin working on those, or do you see that as a less important problem?
WD: At our hearts we’re all technologists, so we have no shortage of enhancements and new ideas that we’ve got on the back burner. However, our primary focus these days is really getting Ksplice more widely deployed, so you should expect to see that first.
CS: What are your long term plans for the technology? Are there any enhancements already in the pipeline that we can expect to see over the next year?
WD: The big goal for us is to start getting Ksplice out there on more distributions. One thing we are planning to do by the end of the year is the general availability of an enterprise-grade Ksplice subscription service for major Linux distributions.
CS: Can users run their own offline update service, or do they have to use the online service from Ksplice?
WD: If you want rebootless updates delivered to your machine, you have to use the Ksplice Uptrack subscription service. If you’re a sophisticated kernel hacker, you can play around with the Ksplice utilities, which allow you to build your own rebootless versions of source code patches. In general, the patches tend to be tricky, and you have to understand the limitations of the Ksplice technology: what updates are safe to apply without writing new code and what aren’t. That said, we have a sample rebootless update on our website that people can build using the Ksplice utilities, if people want to play around with it. The Ksplice utilities and the Uptrack client are both released under the GPLv2.
CS: How well has the project been received by the kernel development community? Have you had any specific interaction with them?
WD: Yes, we’ve proposed Ksplice for merge into mainline. One exciting feature about Ksplice is that it does not require advance kernel modification, so we can update your systems without being in mainline, but we realize that Linux is not just a piece of software — it’s a community. We want to be involved in that community, and we’d love for Ksplice to be the de facto rebootless update solution for Linux. At the moment, some of our patches have been merged, and the maintainers are looking for more wide-scale deployment before they consider merging the rest. Our Ubuntu offering is one of the ways we hope to achieve this wide-scale deployment.
CS: How can users or developers get involved in the project?
WD: Like any free software project, we love hearing your feedback and getting your contributions and patches. The big thing that we’re looking for at the moment is to get the technology deployed on many machines, to hear what people find valuable and what they don’t, so I’d really encourage people to give it a try. If you’re running Ubuntu Jaunty, you should head on over to our website and install Ksplice Uptrack with our one-click installer.
CS: Fantastic. Well thank you very much for your time, it’s a great project and quite exciting technology. We wish you all guys all the best!
WD: Thank you very much!