dcsimg

Drizzle: Rethinking the MySQL Database Kernel

Drizzle is a re-thought and re-worked version of the MySQL kernel designed specifically for high-performance, high-concurrency environments. In this exclusive article, MySQL guru Jeremy Zawodny takes an inside look at the goals and state of Drizzle development.

If you visit the Drizzle project homepage, it explains:

Drizzle is building a database optimized for cloud and [network] applications. [Drizzle] is being designed for massive concurrency on modern multi-cpu/core architectures. The code is originally derived from MySQL.

In other words, Drizzle is a re-thought and re-worked version of the MySQL kernel designed specifically for high-performance, high-concurrency environments. The goal is to make Drizzle an extensible database kernel that runs extremely fast on emerging hardware.

Much like Linux is a fast kernel that can be extended with loadable modules (notably filesystems and device drivers, among others), Drizzle explicitly says “no” to building in some features, instead pushing them out of the core and into plug-ins. This separation allows Drizzle to focus on only the most essential features of a modern database kernel. In fact, it’s almost a become a joke on the Drizzle mailing list that someone will say “Make it a plugin!” when anyone questions removing functionality from the server.

At this point you may be wondering who would be crazy enough to think that MySQL, known for its speed and agility, needs to be radically re-worked. After all, it’s been the dominant open source relational database for the better part of a decade now.

The Drizzle project is headed by Brian Aker, the former director of architecture for MySQL. Aker now works in the Office of the Chief Technology Officer at Sun (soon to be Oracle) and heads up a team of developers working on Drizzle and a few other projects (see one of Aker’s early posts on Drizzle). Some of those developers are Sun employees (Eric Day, Jay Pipes, Monty Taylor, Mark Atwood, and others), while others are either volunteers or contributing to Drizzle on behalf of their employer. Toru Maesaka of Mixi is one such developer.

A lot of the Drizzle work is driven by significant changes in hardware that have occurred since widespread adoption of MySQL began. Before discussing what Drizzle is and is not, it’s important to understand a bit about the last 5-7 years of evolution in hardware and systems and the challenges those presented to MySQL users, especially those who were constantly pushing the envelope of what’s possible.

Moore’s Law Turned Sideways

The most significant change in recent hardware history is that Moore’s Law took a surprising turn. For years, developers have been able to depend on the clock speed of CPUs doubling roughly every 18 months. In other words, the code you ran in early 2001 would be more than twice as fast if you simply upgraded the CPU in your server in late 2002. Why “more than” twice as fast? Not only did clock speeds increase, changes in the architecture of CPUs made them more efficient, too: larger caches, more registers, better instruction pipelines, and so on.

This doubling trend went on for so long that Moore’s Law was practically law and something destined to continue for the foreseeable future. But then something strange happened a couple years ago. Instead of seeing 4 GHz CPUs appear to replace 3.2 GHz CPUs, Intel and AMD started pushing multi-core CPUs that ran at slower clock speeds. Effectively, the megahertz and gigahertz game had come to an end. Instead of pure “vertical” scaling, Moore’s Law had been redirected along a different axis: horizontal scaling.

At the 2009 MySQL Conference, Sun co-founder Andy Bechtolsheim delivered a keynote presentation called The Solid State Storage Revolution, which contained a chart plotting CPU clock speeds over the last 20 years or so. It illustrated this sudden change very clearly and removed any doubt about where things are headed. (Sadly, his slides are not available online).

In a sense this isn’t new. Symmetric multiprocessing (SMP) has been around for years now in the form of motherboards with two or four physically separate CPUs. But having multiple CPU cores on the same physical chip was a new approach. Intel previewed the future with the introduction of Hyper-Threadingon some Pentium 4 CPUs. The technology allowed a single CPU to run two execution threads in parallel. It’s not the same as having two physical cores on the same chip, but it was a nice boost at the time.

With Moore’s Law overturned, the new road to efficiency and performance is to design for concurrency and parallelism. Any problem that can be split into sub-problems and executed in parallel fits this new world very well. As you’ll see momentarily, there are places where MySQL is able to exploit multi-core well, but others in which it was of no help at all.

RAID and RAM

As with all things hardware, the cost of running substantial RAID arrays (6-12 disks) with battery-backed write caches has dropped substantially. That doesn’t mean that disk I/O isn’t still the single biggest bottleneck in large databases—it is. Any time your data size grows substantially beyond what can fit in RAM, I/O bottlenecks appear. But today’s “commodity” hardware lets you push that barrier out quite a long way.

Thanks to commodity 64-bit CPUs, it’s not hard to find a server than can be loaded with 64 GB or even 128 GB of memory. That alone is sufficient to keep a very large percentage of MySQL deployments from having to worry about going to disk that often. But what about those really big deployments?

The transfer rates of modern disks and their controllers is measured hundreds of megabytes per second. But that’s assuming a best case scenario: linear (sequential) reads or writes. The real killer is random disk I/O. Any time the disk head needs to move (seek), you incur a delay of several milliseconds. Event the fastest hard disks available claim average seek times in the 4 ms range. Expecting more than about 200 seeks per second is unrealistic. In fact, the rule of thumb for years has been not to expect more than about 100 I/O operations (iops) from a single spindle.

Having larger disk arrays means being able to perform a greater number of iops per second: typically something like (N/2)*100 where N is the number of disks. You must divide by two because of mirroring. In a typical setup of 12 disks, there are 6 mirrored pairs (RAID-1) and stripe (RAID-0) data across them. This is commonly referred to as RAID 1+0.

Using this model, the only way to get another 200 iops out of your array (without sacrificing data integrity) is to add two or four more disks. Like Moore’s Law, this had been true for a very long time. But just like Moore’s Law, the storage game has recently changed in a most unexpected way.

Comments on "Drizzle: Rethinking the MySQL Database Kernel"

Hmm it looks like your site ate my first comment (it was super long) so I
guess I’ll just sum it up what I wrote and say, I’m thoroughly enjoying your
blog. I too am an aspiring blog blogger but I’m still new to everything.
Do you have any helpful hints for rookie blog writers? I’d genuinely appreciate
it.

This post will help the internet viewers for setting up new blog
or even a blog from start to end.

Hello, i think that i saw you visited my website so i came to “return the favor”.I’m attempting to find things to enhance my website!I suppose its ok
to use some of your ideas!!

Feel free to surf to my web page; cialis dosage options

I always emailed this web site post page
to all my associates, as if like to read it afterward my contacts will too.

Who knows stiletto heels could prove to be our ultimate savior in this recession and save the world
economy from depression. Most of the online
stores are fairly popular finish line, amazon, foot locker, etc.
We pride ourselves in designing fully-integrated business solutions specifically for the independent specialty retailer-whether
that means a single store or sixty stores.

My page :: giuseppe zanotti outlet

I’m impressed, I have to admit. Rarely do I come across a
blog that’s both equally educative and engaging,
and let me tell you, you have hit the nail on the head.
The problem is something which too few people are speaking intelligently about.

I’m very happy I found this during my search for something regarding this.

nulsoai8

vrpp5p3e

insurance

d3ndujaj

d6uqu44k

Individuals with the ribbons upward again could
be a delight for the males; an extra could be that a ribbons up back will match a number of dimensions when you increase along with decrease a
gown dimensions it’ll remain suit for your body.
Often they tell others about the company directly while indirectly promoting your
company. Most people consider a good smile as one that has the 2 central
front teeth dominant and have a 4-to-5 width-to-length ratio.

Also visit my web blog … michael kors canada

I’m really loving the new Zune, and hope this. I’m really loving the new Zune, and hope this.
I’m really loving the new Zune, and hope this. I be taught something tougher on completely different blogs everyday.
I be taught something tougher on completely different blogs everyday. I be taught something tougher on completely different blogs everyday.
http://www.vremya.ru/2008/13/10/196565.html
http://www.tstanki.ru/news.html?data=2009-7
http://te.ua/sn/banews-vlast/syn_glavy_medzhlisa_vozvodit_samostroi_na_territorii_bahchisaraiskogo_dvorca.html

I be taught something tougher on completely different blogs everyday. I be taught something tougher on completely different blogs everyday.

Leave a Reply