Scaling Digg and Other Web Applications
From http://highscalability.com/scaling-digg-and-other-web-applications, about a Digg presentation at Web 2.0 Expo.
Key points:
- Scale out, not up (ie, more servers, not bigger servers) by partitioning users off
- Scale databases both horizontally and vertically (eg, split users horizontally, questions on dedicated db)
- Hide partitioning from the application
- Problems of denormalization (copies of the same data)
- Push non-realtime updates to a queue (ie, look at which DB updates to the ‘master’ can be delayed)
- Cache appropriately: strategies vary depending on lifetime and changeability
- Don’t rush out new features: scale from the start
