To scale at Digg they followed a set of practices very similar to those used at eBay. No joins, no foreign key constraints (to scale writes), primary key look-ups only, limited range queries, and joins were done in memory. When implementing the comment feature a 4,000 percent increase in performance was created by sorting in PHP instead of MySQL. All this effort required to make a relational database scale basically meant you were using a non-relational database anyway. So why not just use a non-relational database from the start?

[via High Scalability]

Digg Firefox 3 Extension from Kevin Rose on Vimeo.

I have!

*toot toot* (that’s the sound of my own horn)

For several months now I’ve found Digg a little frustrating to actually use. With 6 main categories – each with numerous sub-categories, a video section and (a useless) podcast directory there is just way too much content to be able to follow casually. The customizable topics feature helps a little, but with such a large userbase the front page content changes really quickly. I always feel like I miss tonnes of good stories. Digg’s comment system has never been great, but the new threaded/fetch on request comment system is borderline terrible – I don’t understand how they cannot afford enough servers to push out their comments properly. I was a pretty big fan of Digg’s last redesign when they first launched it, but when you compare it to reddit’s simplicity, Digg looks like the 40lb gorrilla it really is.

Reddit’s design is far more functional/utilitarian than digg; it has good navigation, a lack of ads and it’s just plain usable. Reddit’s story placement algorithm is far superior to digg, they seem to use a formula based on votes over time, possibly taking into account click-thrus and comments over time as well. I’m finding that stories stay on top of the list much longer – up to a day or more – and new stories generally rise to the top a lot more slowly; this has the net result of giving me more time to look at a story. I know that if I don’t read the story right away it will still be there later – Reddit also has a ‘saved’ stories feature that I just noticed, again making it easier for me to read a story later. The actual content of the stories appearing on the front page seems to be quite diverse, stories related to a given subject seem to move up and down the list in an organic wave. Reddit seems to be a bit more of a Zietgiest. It’s hard to say whether or not their algorithm would scale well to meet the Digg’s traffic, I suspected it would.

If you don’t really care about getting the latest news right away and/or you don’t have the time to wade through a thousand stories an hour, give Reddit a try. You’ll still be getting your news way faster than your non nerd friends anyways.