News about Redis: 2.8 is shaping, I'm back on Clus

This is a very busy moment for Redis because the new year started in a very interesting way: 1) I finished the Partial Resynchronization patch (aka PSYNC) and merged it into the unstable and 2.8 branch. You can read more about it here: htt

This is a very busy moment for Redis because the new year started in a very interesting way:

1) I finished the Partial Resynchronization patch (aka PSYNC) and merged it into the unstable and 2.8 branch. You can read more about it here: http://antirez.com/news/47
2) We finally have keyspace changes notifications: http://redis.io/topics/notifications

Everything is already merged into our development branches, so the deal is closed, and Redis 2.8 will include both the features.

I'm especially super excited about PSYNC, as this is a not-feature, simply you don't have to deal with it, the only change is that slaves work A LOT better. I love adding stuff that is transparent for users, just making the system better and more robust.

What I'm even more excited about is the fact that now that PSYNC and notifications are into 2.8, I'll mostly freeze it and can finally focus on Redis Cluster.

It's a lot of time that I wait to finish Redis Cluster, now it's the right time because Redis 2.6 is out and seems very stable, people are just starting to really discovering it and the ways it is possible to use Lua scripting and the advanced bit operations to do more. Redis 2.8 is already consolidated as release, but requires a long beta stage because we touched the inner working of replication. So I can pause other incremental improvements for a bit to focus on Redis Cluster. Basically my plan is to work mostly to cluster as long as it does not reach beta quality, and for beta quality I mean, something that brave users may put into production.

Today I already started to commit new stuff to Cluster code. Hash slots are now 16384 instead of 4096, this means that we are now targeting clusters of ~1000 nodes. This decision was taken because there are big Redis users with already a few hundred of nodes running.

Another change is that probably, in order to ship Cluster ASAP, in the first stage I plan to use Redis Sentinel in order to failover master nodes (but Sentinel will be able to accept as configuration a list of addresses of cluster nodes and will fetch all the other nodes using CLUSTER NODES).

So basically the first version of Redis Cluster to hit a stable release will have the following features:

1) Automatic partition of key space.
2) Hot resharding.
3) Only single key operations supported.

The above is already implemented but there is more work to do in order to move all this from alpha to beta quality. There is also a significant amount of work to do in library clients, and I'll try to provide an initial reference implementation based on redis-rb (actually I hope to just write a wrapper library).

Note that "3" is here to stay, there are currently no plans to extend Cluster to anything requiring keys to be moved back on forth magically. But MIGRATE COPY will provide a way for users to move keys into spare instances to perform computations with multiple keys.

Of course all this modulo critical bugs. If there is something odd with stable releases I'll stop everything and fix it as usually :-) Comments