Announcing Public Anaconda Package Download Data

 

I’m very happy to announce that starting today, we will be publishing summarized download data for all conda packages served in the Anaconda Distribution, as well as the popular conda-forge and bioconda channels.  The dataset starts January 1, 2017 (April 2017 for Anaconda Cloud channels) and will be updated roughly once a month.  We hope these data will help the community understand how quickly new package versions are being adopted, which platforms are popular for users, and track the usage of different Python versions.  For example, this dataset can be used to see how the Python 2 to 3 transition has been progressing for the past 2 years:

Python 2 to 3 transition

The fraction of packages downloaded for Python 2 (relative to all Python) has dropped from 48% to 19% in the past 26 months.

To make it easier to analyze, and to lower bandwidth requirements, the dataset consists of pre-aggregated download counts per hour for all unique combinations of channel, package name, package version, platform, and required Python version (if any).  This should allow you to filter and collapse dimensions to quickly find the information you are interested in. The download data is available in the Parquet file format on Amazon S3. To get you started, we have an example notebook that you can run on Binder. Full details on the schema and how to access the data can be found in this Github repository.

We are making this data available for free under a CC-BY-4.0 license, without any S3 charges to the end user, to minimize the barriers to analyzing this data.  The Intake data catalog we have published along with the data will cache the files locally, which both improves performance and saves bandwidth.  All data updates will be announced in the CHANGE_LOG, so please don’t redownload the entire dataset unless that file shows an update.

We have ideas for other package datasets we can share in the future that will be of interest to the Anaconda community, so keep an eye on that repo.  If you have specific requests or have questions about the data, please open an issue.  Happy analyzing!


You May Also Like

Data Science Blog
Distributed Auto-ML with TPOT with Dask
By Tom Augspurger This work is supported byAnaconda, Inc. This post describes a recent improvement made to TPOT. TPOT is an automated machine learning library for Python. I...
Read More
Data Science Blog
Open Sourcing Anaconda Accelerate
We’re very excited to announce the open sourcing and splitting of the proprietary Anaconda Accelerate library into several new projects. This change has been a long time com...
Read More
Data Science Blog
Self-Service Open Data Science: Custom Anaconda Management Packs for Hortonworks HDP and Apache Ambari
As part of our partnership with Hortonworks, we’re excited to announce a new self-service feature of the Anaconda platform that can be used to generate custom Anaconda manag...
Read More