How GPU acceleration works
GPU acceleration requires the author of a project to implement GPU-specific code paths for algorithms that can be executed on the GPU. A GPU-accelerated project calls out to vendor-specific libraries for standard algorithms or uses a GPU compiler to compile custom GPU code. Only the algorithms specifically modified by the project author for GPU usage are accelerated; the rest of the project still runs on the CPU. For most packages, GPU support is either a compile-time or run-time choice, allowing a CPU-only variant of the package to be available. When GPU support is a compile-time choice, Anaconda builds the necessary GPU-enabled variants of the package. The number of variants depends on the GPU support of the packages on each platform. For some packages, conda can automatically select the GPU-enabled variant when your hardware meets the minimum requirements. You can also choose between the CPU-only and GPU-enabled variants yourself.Hardware requirements
Working with GPU packages requires a supported GPU in addition to a supported CPU. GPUs are used when a software package can accelerate tasks through parallel processing. Not all operations in a package use GPUs, so a modern CPU is still required. Most users will have an Intel or AMD 64-bit CPU. Anaconda recommends having at least two to four times more CPU memory than GPU memory, and at least four CPU cores to support data preparation before model training.- NVIDIA
- Apple
NVIDIA officially released CUDA in 2007. Since then, all NVIDIA GPUs support CUDA.The CUDA support level of each GPU is identified by its Compute Capability, which defines the hardware features and supported instructions for each NVIDIA GPU architecture.GPU hardware is always complemented by a GPU driver. See Software requirements for driver details.
Software requirements
Anaconda builds packages for 64-bit versions of Windows, Linux, and macOS operating systems. Not every GPU-acceleration framework is available for all platforms. CUDA Toolkit is only available for Windows and Linux, and Metal is only available for macOS.- NVIDIA
- Apple
Each CUDA release requires a minimum version of the GPU driver. Anaconda recommends installing the latest driver supported by your hardware. You can find version details in the GPU driver release notes and download the latest driver from NVIDIA’s driver download page.You do not need NVIDIA’s CUDA installer or your system package manager (such as
apt install cuda-toolkit). Instead, install the CUDA Toolkit into a conda environment with conda install cuda. Each environment can have its own CUDA version. Only the GPU driver needs to be installed on your system.Ubuntu and some other Linux distributions ship with a third-party open-source driver for NVIDIA GPUs called Nouveau. CUDA requires replacing the Nouveau driver with the official NVIDIA GPU driver.NVIDIA GPU drivers are usually backward compatible. You can run applications compiled against an older CUDA version using the latest GPU driver. However, you might not be able to run applications compiled against a newer CUDA version using an older GPU driver. In this case, you need to update the GPU driver. If driver updates are not possible, NVIDIA provides CUDA forward compatibility packages.Anaconda provides the CUDA Toolkit and CUDA-enabled libraries as conda packages.CUDA Toolkit releases: Anaconda ships every CUDA Toolkit release from NVIDIA on the main channel. See CUDA development for installation instructions.CUDA-enabled packages: Packages that require CUDA compilation, such as PyTorch or CuPy, are built against a single pinned CUDA version for each supported major release series. Anaconda currently ships CUDA-enabled variants for both the CUDA 12.x and CUDA 13.x series. Using the same pinned versions across the distribution keeps packages binary compatible with each other.Anaconda chooses these pinned versions based on GPU driver adoption across our user base. Pinning to an older minor release within a series lets the packages run on more systems. Anaconda moves the pin forward once driver support for a newer release is widespread.Available packages
The Anacondamain channel includes many GPU-accelerated packages. The following are some of the most widely used.
PyTorch
PyTorch is a machine learning library for deep learning and general tensor computation. It supports NVIDIA CUDA on Linux and Windows, and Apple MPS on macOS. PyTorch detects GPU availability at runtime.TensorFlow
TensorFlow is a machine learning platform for building and training models. The GPU variant is selected by installing thetensorflow-gpu meta-package.
tensorflow package name use whichever variant is installed, making it straightforward to switch between CPU and GPU in an environment.
JAX
JAX provides composable transformations of Python and NumPy programs—differentiate, vectorize, parallelize, and JIT-compile to GPU.jaxlib, which conda installs automatically as a dependency. CUDA-enabled builds of jaxlib are available on Linux.
Numba
Numba is a general-purpose JIT compiler for Python functions, sponsored by Anaconda. It provides a way to implement custom GPU algorithms in Python syntax.XGBoost
XGBoost is a machine learning library that implements gradient-boosted decision trees. GPU-accelerated training is available through the CUDA-enabled build oflibxgboost, which conda selects automatically when your hardware supports it.
FAISS
FAISS is a library for efficient similarity search and clustering of dense vectors. The GPU variant is selected by installing thefaiss-gpu meta-package.
ONNX Runtime
ONNX Runtime is a cross-platform inference engine for models in the ONNX format. Both CPU and CUDA builds are available.LightGBM
LightGBM is a gradient boosting framework for fast, distributed, high-performance training. Both CPU and CUDA builds are available.DeepSpeed (Linux only)
DeepSpeed is a deep learning optimization library for distributed training and inference at scale.FlashAttention (Linux only)
FlashAttention provides fast, memory-efficient attention kernels for transformer models.bitsandbytes (Linux only)
bitsandbytes provides CUDA-accelerated functions for quantization, enabling large language models to run with reduced memory usage.
vLLM (Linux only)
vLLM is a high-throughput inference and serving engine for large language models.llama-cpp-python
llama-cpp-python provides Python bindings for the llama.cpp library, enabling inference with large language models. It supports CUDA acceleration on Linux and Windows, and Metal acceleration on macOS.
CuPy
CuPy is a GPU-accelerated replacement for NumPy and SciPy. You can use it as a drop-in replacement for NumPy with minimal changes to your code.CUDA development
CUDA programming was historically limited to C, C++, and Fortran. Recent CUDA Toolkit releases make GPU programming available in languages like Python and Rust through specialized packages.CUDA C/C++ SDK
Anaconda provides the CUDA C/C++ SDK in themain channel.
To install the latest version:
CUDA-Python
CUDA-Python provides Python bindings for the CUDA driver and runtime APIs. CUDA-Python releases are linked to CUDA Toolkit major and minor releases. To install the latest version:As of today, CUDA-Rust is an experimental, research-grade package. Anaconda will provide it in the
main channel as it becomes more stable.