Interdependent Web

Printer-friendly version

session details: The Next Decade

• there are now 4.5 million tweets per day -- people are creating volumes of content on their cell phones
• why is Twitter significant? because it's an example of traffic volume that cannot be handled by a single database, and Drupal is currently limited to a single database (unless you hack core...)
  
• client-server model is obsolete, can't keep up with the scale of modern applications
  • we need to lose the assumption that data needs to be consistent across servers, because it creates bottlenecks
  • consistency is an abstraction anyway; it should be a question, not an assumption
  • immediate consistency is really only important to the originator of the data; doesn't affect others in the same way.
  • what's more important is eventual consistency (5 mins or so)
  • conflicts arise when people change things like tags on one server while others are posting content with those tags on another server
  • anticipate conflicts and build applications to resolve them -- can't expect databases to handle them
• we already have this problem at drupal.org and have mostly resolved it
• can't just assign server IDs to table keys because (theoretically) two people could post at the same instant on different servers -- which one should display first?
• wikis (and Google Docs) offer a model for resolving content conflicts because revisions are essentially patches rather than entire new versions of the content, so conflicts only involve a line or two at a time instead of the whole page ... there is not yet a Drupal module to do this!
  
• everything is possible, but some things are prohibitively difficult!