Solving The Twitter Problem(s)
Posted by rbpasker on May 6, 2008
The net is atwitter with solutions to the “Twitter Problem,” which is that Twitter keeps going down, and it is annoying the bloggers themselves, who use it as a broadcast platform.
But there are really a bunch of different problems, many of which will be solved independently:
Decentralization, a la DNS, of the network of servers/services providing Twitter-like functionality, for redundancy.
Scalability using a server-side implementation that can handle the volume of messages on the busiest nodes efficiently. I’ve proposed Tervela‘s product as the hub
Inverting the protocol from “pull” to “push,” to reduce the amount of unproductive polling that takes place.
A universal addressing scheme, based on existing standards, like RFC 822‘s mailbox addressing. (This is what Jabber uses). A URL-based system would work, too.
Proper language mappings (such as Actors) that are designed to handle large numbers of messages with minimal overhead.
A common API/wire format, hopefully something much simpler than XMPP, to make it simple to write applications that can participate in this network.
Gateways to other messaging systems, such as email and SMS
Bridges (PDF) to synchronous protocols, like HTTP
The good thing is, lots of people are finally starting to think about this problem. The bad thing is, lots of people are finally starting to think about this problem (which means it will take a LONG time to solve).