Working with Twitter API

Challenge:
Implement a Markov chain type app, which will accept live Tweets as input sentences.  Will breakdown these Tweets, construct the Markov chains and store them efficiently.  Then using these chains to create sensical Tweets made from random connections.

Break-Down:
Couple things need to be done to solve this:
1) Accessing Twitter's Realtime Tweet stream -- Correct application of Twitter APIs
2) Efficient processing/buffering of the thousands of tweets coming in per minute -- the stream will be extremely busy with tweets, and all tweets are important and cannot be dropped
3) Efficient data storage of the Markov chains -- SQL is not needed here, an Key-Value solution makes more sense (i.e. Redis)
4) (Efficiently Random) Retrieval of the chains stored -- Building new sentences/Tweets requires performing Random lookup of the stored chains, and walking the data structures.

Solution: