From weather readings to financial tickers, many interesting data come in the form of streams. An issue with streaming data analysis in IPython is that one often has to either be collecting the data from the stream, or analyzing it, but not both. In this tutorial, I demonstrate a single-notebook IPython workflow for simultaneous collection and analysis of realtime streaming data:
- Collect and process streaming data without blocking notebook input,
- Use an in-memory database to store and read stream data,
- Perform analyses using up-to-the minute data,
Twitter data is just an example, this workflow is widely applicable to a whole class of streaming data sources with geographic and/or semantic metadata.
View the notebook in Wakari