Skip to main content
anaconda-ai is a CLI and Python SDK for managing AI models curated by Anaconda. With it, you can list and download quantized LLMs, launch and manage inference servers, and work with a vector database, all from the command line.

​
Installation

conda install anaconda-ai

​
Backends

anaconda-ai communicates with a backend to manage models and servers. The following backends are supported:
Anaconda Desktop is currently available through a limited early access program. Anaconda AI Navigator provides many of the same capabilities and is available to all users.
The ai-catalyst backend supports multi-site configuration, allowing you to work with multiple Anaconda Platform sites. For setup and usage, see the anaconda-auth site configuration documentation and use the anaconda sites CLI commands.
To change the default backend, run: 
anaconda ai config --backend <BACKEND> --yes
You can also override the backend per command with the --backend option.

​
Configuration

Configuration settings are defined in ~/.anaconda/config.toml under [plugin.ai]. Each parameter can also be set through an environment variable.
Example configuration
# ~/.anaconda/config.toml
[plugin.ai]
backend = "anaconda-desktop"
stop_server_on_exit = true
server_operations_timeout = 60
show_blocked_models = false

​
Model reference format

Quantized models follow this reference format: 
<AUTHOR>/<MODEL>/<QUANT>.<EXT>
  • <AUTHOR>: Publisher name (optional).
  • <MODEL>: Model name.
  • <QUANT>: Quantization method (Q4_K_M, Q5_K_M, Q6_K, Q8_0).
  • <EXT>: File extension, usually .gguf (optional).
The model name and quantization method must be separated by either / or _:
OpenHermes-2.5-Mistral-7B/Q4_K_M
meta-llama/llama-2-7b-chat-hf/Q4_K_M
TinyLlama/TinyLlama-1.1B-Chat-v1.0_Q4_K_M.gguf
Framework integrations (LangChain, LiteLLM, etc.) also support server references in the format server/<server-id>. See the SDK servers documentation for details.

​
Basic usage

​
List available models

anaconda ai models

​
View details for a specific model

anaconda ai models OpenHermes-2.5-Mistral-7B

​
Download a model

anaconda ai download OpenHermes-2.5-Mistral-7B/Q4_K_M

​
Launch an inference server

anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M
The server starts and remains attached to your terminal. Press Ctrl+C to stop and remove the server.
To leave the server running in the background, use the --detach flag:
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --detach

​
List running servers

anaconda ai servers

​
Stop a running server

anaconda ai stop <SERVER_ID>

​
Vector database

Launch a pgvector database for storing and querying embeddings: 
anaconda ai launch-vectordb
List, create, and manage database tables: 
anaconda ai list-tables
anaconda ai create-table my_table '{"columns": [{"name": "id", "type": "SERIAL PRIMARY KEY"}, {"name": "embedding", "type": "vector(384)"}]}'
anaconda ai drop-table my_table
Stop the vector database when finished: 
anaconda ai stop-vectordb
Vector database functionality requires a compatible backend. Not all backends support vector database operations.
For the full command reference, see the command index. For the Python SDK and framework integrations, see the Anaconda AI SDK documentation.