Our team has been busy this past fall open sourcing many of the compiler features of the numbapro module into the Numba open source project, as well as consolidating the features of the numbapro module into an accelerate module that is now available to paid Anaconda subscribers and academic users. This new package provides a one stop shop for access to optimized libraries from Nvidia, Intel, and AMD in the near future.

Anaconda, our modern open source platform, is the best way for a business or lab to get started with Python, the fastest growing language for data science. Anaconda paid subscriptions add additional capabilities including fast algorithms, compilers, and optimized libraries. In the past, some of these tools were only available in a Python module called numbapro that was part of a separate proprietary Anaconda Add-On called Accelerate. Since September 2015, the Accelerate Add-On, including the numbapromodule, has only been available as part of Anaconda Workgroup or Anaconda Enterprise subscriptions. 

Our team has been busy this past fall open sourcing many of the compiler features of the numbapro module into the Numba open source project, as well as consolidating the features of the numbapro module into an accelerate module that is now available to paid Anaconda subscribers and academic users. This new package provides a one stop shop for access to optimized libraries from Nvidia, Intel, and AMD in the near future.

Historically, numbapro was a premium add-on module that extended Numba and provided: 

  • Support for parallel and GPU computing, including the CUDA-Python compiler
  • Access to Nvidia’s CUDA libraries
  • And a compiler for parallel and CUDA ufuncs

The CUDA-Python compiler was open-sourced as part of Numba in March 2014, and the remaining parts of numbapro provided access to a small number of compiler features, and access to Nvidia’s CUDA libraries such as cuBLAS and cuSPARSE. In order to provide a more consistent packaging, numbapro has been deprecated and split in two (further depicted in the diagram below):

  1. Compilers for parallel and CUDA ufuncs, support for CUDA printing from CUDA kernels, and the CUDA @reduce decorator have been open sourced and are accessible in Numba 0.22.
  2. Access to Nvidia’s CUDA accelerated libraries is moved into the accelerate module, along with additional functions from Intel’s MKL that go beyond those provided in the free MKL optimizations package.

Future versions of Accelerate will include access to additional vendor libraries, including more from Nvidia, Intel, and AMD.

Namespace Changes and Compatibility with Existing Code

The numbapro module is now deprecated, and its functions have moved to become accessible under new names as follows:

  • CUDA library functions have moved from the numbapro.cudalib package to the accelerate.cuda package
  • The gpu and parallel targets, which were previously only available in numbapro.vectorize, are now available in numba.vectorize
  • The CUDA reduction compiler, exposed as the decorator numbapro.cuda.reduce is now available as numba.cuda.reduce

Compatibility with existing code will be maintained by a new numbapro compatibility module which proxies function names in the old namespace to the functions in the new namespace. As a result, no changes to existing code are needed to upgrade to new capability in accelerate. However, we strongly recommend that new code use the accelerate module.

To install the compatibility layer, you must run conda update numbapro after installing Accelerate, even if you have an earlier version of NumbaPro already installed. For full details of the compatibility layer, please see the release notes for the accelerate package

Future Improvements

The Anaconda platform has a regular update cadence, thus the accelerate capabilities will have regular updates and releases. New features planned on the roadmap include:

  • NumPy ufuncs accelerated with Intel’s Vector Math Library
  • Access to Nvidia’s cuSOLVER library alongside other CUDA libraries
  • Improved APIs for existing libraries.
     

About the Author

Q. What is your superpower(s)?

A. Developer

Q. What is your technical specialty or area of research?

A. I work in the High Performance Analytics group, which works to get the b …

Read more

Join the Disucssion