Need a key/value store that doesn't compromise functionality for performance? Have a look at redis.
It’s common to use MySQL as the backend for storing and retrieving what are essentially key/value pairs. I’ve seen this over-and-over when someone needs to maintain a bit of state, session data, counters, small lists, and so on. When MySQL isn’t able to keep up with the volume, we often turn to memcached as a write-thru cache. But there’s a bit of a mis-match at work here. Memcached doesn’t have much in the way of a query language or server-side operations it can perform on the data. In other words, it’s a pretty “dumb” store.
MySQL lies at the other end of the spectrum. It has a rich query language and support for all sorts of server-side processing on the data. So when you combine the two, your app has to know it’s talking to both and deal with coordinating data changes between the cache and the back-end server.
But what if there was a middle ground? A fast, efficient key/value store that provides a reasonable set of server-size operations aimed at making the most common operations trivial. That’s what redis is all about.
Calling redis a key/value store doesn’t quite due it justice. It’s better thought of as a “data structures” server that supports several native data types and operations on them. That’s pretty much how creator Salvatore Sanfilippo (known as antirez) describes it in the documentation. Let’s dig in and see how it works.
Get Started Quickly
One of the first themes you encounter with redis is a form of minimalism. It has virtually no external dependencies and the entire unpacked soure distribution is well under 2MB in size. It has no autoconf supported build process. It comes with no init script and no installation process. Thankfully, it’s trivial to get started.
First, get and build the code:
$ wget http://redis.googlecode.com/files/redis-0.900_2.tar.gz
$ tar -zxvf redis-0.900_2.tar.gz
$ cd redis-0.900
Then, start up a server:
$ ./redis-server >& /tmp/redis.log &
And start talking to it:
$ telnet localhost 6379
Connected to localhost.
Escape character is '^]'.
SET mykey 12
Connection closed by foreign host.
Okay, let’s have a quick look at what just happened there. The redis server started up and waited for connections on TCP port 6379 (which you can see if you look in the log file), so we can just use
telnet localhost 6379 and start talking to it. The redis protocol is a very simple interactive plain-text protocol that’s reminiscent of the POP3 email protocol.
SET command is used to create a new key and associate a string value with it. It expects a key name and a value length. We set
mykey equal to
GET command is how you fetch the value associated with a key. To check for the existence of a key, use the
EXISTS command, which will return a boolean value.
It’s worth noting that call commands in the redis protocol are case-insensitive (though the documentation typically shows them in uppercase for readability) and you really don’t need to get into the details of the wire protocol most of the time. There are readily available client APIs available for a number of languages, including:
The overriding focus for redis is performance and the benchmark numbers really speak for themselves. Seeing 80,000-100,000 (or more) operations per second on modest modern hardware is likely enough to make a dramatic difference to most applications.
Much of this performance comes from it’s minimal feature set. The author has carefully chosen features that can be supported by very fast algorithms and, in come cases, even lock-free atomic operations.
Data Types and Operations
Redis provides three native data types, any one of which may be associated with a given key:
The full Redis Command Reference is obviously the definitive source for all the operations, but I’d like to highlight the most common and useful ones here to give you an idea of what makes redis so useful.
- GETSET: returns the previous value and sets new one
- MGET: mutli-key get
- SETINX: set if not exists
- INCR: increment
- INCRBY: increment by specific value
- DECR: decrement
- DECRBY: decrement by specific value
- DEL: delete key
- TYPE: return type of key
- RPUSH: append item to tail
- LPUSH: append item to head
- LEN: get number of items in list
- LRANGE: fetch a range (subset) of items from the list
- LTRIM: trim list to the specified range
- LSET: set new value for the Nth element
- LREM: remove one or more elements
- LPOP: atomically remove and return the first element
- RPOP: atomically remove and return the last element
- SADD: add an item to a set
- SREM: remove an item from a set
- SPOP: remove and return a random element from a set
- SCARD: return the cardinality (number of elements) of a set
- SISMEMBER: test for membership in a set
- SMEMBERS: return all member of a given set
There are also some interesting commands for computing and optionally storing the intersection (SINTER, SINTERSTORE), union (SUNION, SUINIONSTORE), differences (SDIFF, SDIFFSTORE) among sets.
Other Operations and Commands
There are also commands to trigger saving data to disk, monitoring the server, and meta-operations on individual databases. Again, see the command reference for details.
- KEYS: list all keys matching a pattern
- EXPIRE: set expiration time (in seconds) of a key
- TTL: get the time-to-live of a key
- SAVE: save data to disk (synchronously)
- BGSAVE: save data to disk (asynchronously)
- LASTSAVE: get timestamp of last saved data
- SHUTDOWN: performs a SAVE and then shuts down the server
Durability and Availability
memcached, redis can save its state to disk so that you can shut down a server without losing all the data. For performance reasons, redis doesn’t log every change to disk as it happens (then it’d be a lot more like a database). Instead, it can be configured to save a copy of the database to disk after a certain amount of time has passed or a certain threshold of changes have occurred in the data.
The method of writing to disk requires no locking and has no consistency problems. The server simply forks a child which inherits a copy of the data it can write to disk. That process of writing to disk should take somewhere from a few seconds to maybe a minute or two. But thanks to copy-on-write memory management, that copy doesn’t require much extra space.
Redis also has built-in support for master/slave replication, so it’s possible to scale in read-heavy environments
Redis is nearing a 1.0 release soon and can already serve as the foundation for a remarkably fast and durable in-memory data store. The on-line documentation is a good place to start reading about it. I think you’ll be surprised at the simplicity, functionality, and performance if redis.