Open Source
We are proud to distribute and contribute to a variety of open source projects.
Support OSS
You can help support ongoing innovation on projects like these in the open source community. NumFOCUS is a non-profit organization supporting numerous open-source projects in support of an inclusive scientific and research community making impactful discoveries for a better world. Your donation goes directly to them and supports their work.
How to get started
More than 250 of the most commonly used open-source data science and machine learning packages are automatically installed when you download Anaconda Individual Edition, and thousands of others can be installed by simply typing conda install [package-name].
The Fundamentals
Jupyter is an open-source project created to support interactive data science and scientific computing across programming languages. Jupyter offers a web-based environment for working with notebooks containing code, data, and text. Jupyter notebooks are the standard workspace for most Python data scientists.
Jupyter 🔗
A library for tabular data structures, data analysis, and data modeling tools, including built-in plotting using Matplotlib. pandas aims to be the fundamental high-level building block for doing practical, real world data analysis in Python
pandas 🔗
The SciPy library consists of a specific set of fundamental scientific and numerical tools for Python that data scientists use to build their own tools and programs. It provides many user-friendly and efficient numerical routines, such as routines for numerical integration, interpolation, optimization, linear algebra, and statistics.
SciPy 🔗
A core package for scientific computing with Python. Numpy enables array formation and basic operations with arrays. Numpy is used for indexing and sorting but can also be used for linear algebra and other operations. Many other data-science libraries for Python are built on NumPy internally, including Pandas and SciPy.
NumPy 🔗
Machine Learning
Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
Keras 🔗
TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
TensorFlow 🔗
An open-source deep learning framework using GPUs and CPUs that consists of fundamental tools and libraries for Python AI and machine learning development.
PyTorch 🔗
A powerful and versatile machine learning library for machine learning basics like classification, regression, and clustering. It includes both supervised and unsupervised ML algorithms with important functions like cross-validation and feature extraction. Scikit-learn is the most frequently downloaded machine learning library.
scikit-learn 🔗
Data Visualization
Matplotlib is the most well-established Python data visualization tool, focusing primarily on two-dimensional plots (line charts, bar charts, scatter plots, histograms, and many others). It works with many GUI interfaces and file formats, but has relatively limited interactive support in web browsers.
Matplotlib 🔗
Bokeh is an interactive visualization library for modern web browsers. It provides elegant, concise construction of versatile graphics, and affords high-performance interactivity over large or streaming datasets. Bokeh can help anyone who would like to quickly and easily make interactive plots, dashboards, and data applications.
Bokeh 🔗
Plotly's Python graphing library makes interactive, publication-quality graphs. It is a popular and powerful browser-based visualization library that lets you create interactive, JavaScript-based plotsfrom Python.
Plotly 🔗
HoloViz is an Anaconda project to simplify and improve Python-based visualization by adding high-performance server-side rendering (Datashader), simple plug-in replacement for static visualizations with interactive Bokeh-based plots (hvPlot), and declarative high-level interfaces for building large and complex systems (HoloViews and Param).
HoloViz 🔗
Dashboarding
Panel is an open-source Python library that lets you create custom interactive web apps and dashboards by connecting user-defined widgets to plots, images, tables, or text.
Panel 🔗
Dash is a productive Python framework for building web applications. Through a couple of simple patterns, Dash abstracts away all of the technologies and protocols that are required to build an interactive web-based application.
Dash 🔗
Voilà turns Jupyter notebooks into standalone web applications. Unlike the usual HTML-converted notebooks, each user connecting to the Voilà tornado application gets a dedicated Jupyter kernel which can execute the callbacks to changes in Jupyter interactive widgets.
Voila 🔗
Streamlit is an open-source Python library that makes it easy to build beautiful custom web-apps for machine learning and data science. It runs on a simple and powerful app model that lets you build rich UIs incredibly quickly.
Streamlit 🔗
Image Processing
Pillow (a “friendly fork” of the older PIL library) is a Python imaging library and a general image processing tool with support for opening, manipulating, and saving images in many different file formats.
Pillow 🔗
Scikit-Image is an open-source Python package containing a collection of image-processing algorithms, including segmentation, geometric transformations, color space manipulation, and feature detection. It uses NumPy arrays as image objects.
Scikit Image 🔗
OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library with C++, Java, Python and MATLAB interfaces. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
OpenCV 🔗
Scalable Computing
Numba is a high-performance Python compiler. It makes Python faster and optimizes the performance of Numpy arrays, reaching the speed of FORTRAN and C without a an additional compilation step.
Numba 🔗
![Dask Logo [1]](https://cdn2.assets-servd.host/voracious-blesbok/production/resources/open-source/Dask-logo.svg){kind=logo}
Dask is a Python package used to scale NumPy workflows with parallel processing to enable multi-dimensional data analysis, enabling users to store and process data larger than their computer’s RAM. Dask can scale out to clusters, or scale down to a single computer. Dask mimics the pandas and NumPy API, making it more intuitive for Python data scientists.
Dask 🔗
The RAPIDS data science framework is a collection of libraries for running end-to-end data science pipelines completely on the GPU. The interaction is designed to have a familiar look and feel to working in Python, but utilizes optimized NVIDIA® CUDA® primitives and high-bandwidth GPU memory under the hood.
Rapids 🔗
A fault-tolerant cluster computing framework and interface for programming clusters launched by UC Berkeley. Developed for Java/Hadoop ecosystem but with support for Python. PySpark is the Python API for Spark.
Apache Spark 🔗
Data Pipelines / ETL
An open-source workflow automation tool by Apache for creating data workflows, scheduling tasks and monitoring results. It integrates with multiple cloud providers, including AWS, Azure, and Google Cloud.
Apache Airflow 🔗
A data ingest/loading library for a wide variety of file formats and data services, with hierarchical cataloguing, searching, and interactivity with remote storage platforms under a single interface.
Intake 🔗
Natural Language Processing (NLP)
An open-source Python natural language toolkit for symbolic and statistical NLP. It includes a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning in multiple languages.
NLTK 🔗
A Python library for topic modeling, document indexing, and similarity retrieval for large bodies of text with efficient multicore implementations of NLP algorithms.
Gensim 🔗
spaCy is an open-source Python library for NLP and one of the fastest, if not the fastest, syntactic parser. spaCy excels at large-scale information extraction tasks. It's written from the ground up in carefully memory-managed Cython.
spaCy 🔗
Looking Ahead: AI Frontiers We're Watching
An open neural network exchange making machine learning models portable between frameworks and platforms. Microsoft and Facebook started this community in 2017 to create an open ecosystem for interchangeable models.
ONNX 🔗
A burgeoning project by open-source developers at Microsoft. FairLearn is a Python package for assessing fairness and mitigating unfairness in ML models and AI systems.
Fairlearn 🔗
A comprehensive open-source Python toolkit of metrics that checks for and measures bias in datasets and ML models. It also included algorithms to mitigate bias. This toolkit was developed by IBM’s open-source team.
AI Fairness 360 (AIF360) 🔗
An open-source Python package that makes it easy to compare algorithms for interpretability.
It provides a “scikit-learn style uniform API” and includes an interactive visualization platform and dashboard so data scientists can compare algorithms with ease.
It provides a “scikit-learn style uniform API” and includes an interactive visualization platform and dashboard so data scientists can compare algorithms with ease.
InterpretML 🔗
LIME is a PyPI package and a model-agnostic interpretability tool. LIME explains individual predictions for text classifiers that act on tables or images. Support for scikit-learn classifiers is built into the tool.
LIME 🔗