anaconda-ai is a CLI and Python SDK for managing AI models curated by Anaconda. With it, you can list and download quantized LLMs, launch and manage inference servers, and work with a vector database, all from the command line.
Installation
Backends
anaconda-ai communicates with a backend to manage models and servers. The following backends are supported:
Anaconda Desktop is currently in beta. Anaconda AI Navigator provides many of the same capabilities and is available to all users.
The
The
ai-catalyst backend supports multi-site configuration, allowing you to work with multiple Anaconda Platform sites. For setup and usage, see the anaconda-auth site configuration documentation and use the anaconda sites CLI commands.--backend option.
Configuration
Configuration settings are defined in~/.anaconda/config.toml under [plugin.ai]. Each parameter can also be set through an environment variable.
Example configuration
Model reference format
Quantized models follow this reference format:<AUTHOR>: Publisher name (optional).<MODEL>: Model name.<QUANT>: Quantization method (Q4_K_M,Q5_K_M,Q6_K,Q8_0).<EXT>: File extension, usually.gguf(optional).
The model name and quantization method must be separated by either
/ or _:server/<server-id>. See the SDK servers documentation for details.
Basic usage
List available models
View details for a specific model
Download a model
Launch an inference server
Ctrl+C to stop and remove the server.
To leave the server running in the background, use the
--detach flag:List running servers
Stop a running server
Vector database
Launch a pgvector database for storing and querying embeddings:Vector database functionality requires a compatible backend. Not all backends support vector database operations.