From weather readings to financial tickers, many interesting data come in the form of streams. An issue with streaming data analysis in IPython is that one often has to either be collecting the data from the stream, or analyzing it, but not both. In this tutorial, I demonstrate a single-notebook IPython workflow for simultaneous collection and analysis of realtime streaming data:

  • Collect and process streaming data without blocking notebook input,
  • Use an in-memory database to store and read stream data,
  • Perform analyses using up-to-the minute data,
  • Display analysis output in the notebook with Javascript, using Google Maps API.

 

Twitter data is just an example, this workflow is widely applicable to a whole class of streaming data sources with geographic and/or semantic metadata.

View the notebook in Wakari

 


About the Author

Clayton is an interdisciplinary student and researcher, always looking to connect people and ideas from different academic tribes. Much of his current work revolves around taking scientific data and workflows to the cloud in order to inc …

Read more

Join the Disucssion