> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting started

export const GCell = ({children, className}) => <div className={`grid-table-cell ${className || ""}`} role="cell">
    {children}
  </div>;

export const GTH = ({children, className}) => <div className={`grid-table-th ${className || ""}`} role="columnheader">
    {children}
  </div>;

export const GRow = ({children}) => <div className="grid-table-row" role="row">{children}</div>;

export const GBody = ({children}) => <div className="grid-table-body" role="rowgroup">{children}</div>;

export const GHead = ({children}) => <div className="grid-table-head" role="rowgroup">{children}</div>;

export const GTable = ({children, className, cols}) => <div className={`grid-table not-prose overflow-hidden rounded-2xl ${className || ""}`} style={{
  "--grid-table-cols": cols
}} role="table">
    {children}
  </div>;

export const Comments = ({children}) => {
  return <div class="my-4 px-5 py-4 overflow-hidden rounded-2xl flex gap-3 border border-zinc-500/20 bg-zinc-50/50 dark:border-zinc-500/30 dark:bg-zinc-500/10" data-callout-type="comments">
      <div class="w-4">
        <svg width="14" height="14" viewBox="0 0 640 640" fill="currentColor" xmlns="http://www.w3.org/2000/svg" class="w-5 h-5" aria-label="Comments">
            <path d="M320 112C434.9 112 528 205.1 528 320C528 434.9 434.9 528 320 528C205.1 528 112 434.9 112 320C112 205.1 205.1 112 320 112zM320 576C461.4 576 576 461.4 576 320C576 178.6 461.4 64 320 64C178.6 64 64 178.6 64 320C64 461.4 178.6 576 320 576zM280 400C266.7 400 256 410.7 256 424C256 437.3 266.7 448 280 448L360 448C373.3 448 384 437.3 384 424C384 410.7 373.3 400 360 400L352 400L352 312C352 298.7 341.3 288 328 288L280 288C266.7 288 256 298.7 256 312C256 325.3 266.7 336 280 336L304 336L304 400L280 400zM320 256C337.7 256 352 241.7 352 224C352 206.3 337.7 192 320 192C302.3 192 288 206.3 288 224C288 241.7 302.3 256 320 256z" />
        </svg>
      </div>
      <div class="text-sm prose min-w-0 w-full">
        {children}
      </div>
    </div>;
};

`anaconda-ai` is a CLI and Python SDK for managing AI models curated by Anaconda. With it, you can list and download quantized LLMs, launch and manage inference servers, and work with a vector database, all from the command line.

## Installation

```bash theme={null}
conda install anaconda-ai
```

## Backends

`anaconda-ai` communicates with a backend to manage models and servers. The following backends are supported:

<Note>
  Anaconda Desktop is currently in beta. [Anaconda AI Navigator](/docs/legacy/ai-navigator/getting-started-ai-navigator) provides many of the same capabilities and is available to all users.

  ***

  The `ai-catalyst` backend supports multi-site configuration, allowing you to work with multiple Anaconda Platform sites. For setup and usage, see the [anaconda-auth site configuration](/docs/cli-reference/anaconda-auth/getting-started#site-configuration) documentation and use the [`anaconda sites`](/docs/cli-reference/anaconda-auth/commands/sites) CLI commands.
</Note>

<GTable cols="20% 40% 40%">
  <GHead>
    <GRow>
      <GTH>Backend</GTH>
      <GTH>Description</GTH>
      <GTH>Supported Features</GTH>
    </GRow>
  </GHead>

  <GBody>
    <GRow>
      <GCell>`ai-navigator`</GCell>
      <GCell>Backend for [Anaconda AI Navigator](/docs/legacy/ai-navigator/main) (default)</GCell>
      <GCell>Models, Servers, Server Parameters, Vector Database</GCell>
    </GRow>

    <GRow>
      <GCell>`ai-catalyst`</GCell>
      <GCell>Backend for Anaconda Platform's AI features</GCell>
      <GCell>Models, Servers, Multi-Site</GCell>
    </GRow>

    <GRow>
      <GCell>`anaconda-desktop`</GCell>
      <GCell>Backend for Anaconda Desktop's [AI features](/docs/anaconda-desktop/getting-started-desktop#downloading-and-interacting-with-models)</GCell>
      <GCell>Models, Servers, Server Parameters, Vector Database</GCell>
    </GRow>
  </GBody>
</GTable>

To change the default backend, run:

```bash theme={null}
anaconda ai config --backend <BACKEND> --yes
```

<Comments>
  Replace \<BACKEND> with the backend you want to use.
</Comments>

You can also override the backend per command with the `--backend` option.

## Configuration

Configuration settings are defined in `~/.anaconda/config.toml` under `[plugin.ai]`. Each parameter can also be set through an environment variable.

<GTable cols="20% 30% 34% 16%">
  <GHead>
    <GRow>
      <GTH>Parameter</GTH>
      <GTH>Environment variable</GTH>
      <GTH>Description</GTH>
      <GTH>Default</GTH>
    </GRow>
  </GHead>

  <GBody>
    <GRow>
      <GCell>`backend`</GCell>
      <GCell>`ANACONDA_AI_BACKEND`</GCell>
      <GCell>Inference backend to use</GCell>
      <GCell>`ai-navigator`</GCell>
    </GRow>

    <GRow>
      <GCell>`stop_server_on_exit`</GCell>
      <GCell>`ANACONDA_AI_STOP_SERVER_ON_EXIT`</GCell>
      <GCell>Automatically stop servers when the Python interpreter exits</GCell>
      <GCell>`true`</GCell>
    </GRow>

    <GRow>
      <GCell>`server_operations_timeout`</GCell>
      <GCell>`ANACONDA_AI_SERVER_OPERATIONS_TIMEOUT`</GCell>
      <GCell>Timeout in seconds for server start and stop operations</GCell>
      <GCell>`60`</GCell>
    </GRow>

    <GRow>
      <GCell>`show_blocked_models`</GCell>
      <GCell>`ANACONDA_AI_SHOW_BLOCKED_MODELS`</GCell>
      <GCell>Display models that are unavailable on the current system</GCell>
      <GCell>`false`</GCell>
    </GRow>
  </GBody>
</GTable>

```toml Example configuration theme={null}
# ~/.anaconda/config.toml
[plugin.ai]
backend = "anaconda-desktop"
stop_server_on_exit = true
server_operations_timeout = 60
show_blocked_models = false
```

## Model reference format

Quantized models follow this reference format:

```text theme={null}
<AUTHOR>/<MODEL>/<QUANT>.<EXT>
```

* `<AUTHOR>`: Publisher name (optional).<br />
* `<MODEL>`: Model name.<br />
* `<QUANT>`: Quantization method (`Q4_K_M`, `Q5_K_M`, `Q6_K`, `Q8_0`).<br />
* `<EXT>`: File extension, usually `.gguf` (optional).

<Note>
  The model name and quantization method must be separated by either `/` or `_`:

  ```text theme={null}
  OpenHermes-2.5-Mistral-7B/Q4_K_M
  meta-llama/llama-2-7b-chat-hf/Q4_K_M
  TinyLlama/TinyLlama-1.1B-Chat-v1.0_Q4_K_M.gguf
  ```
</Note>

Framework integrations (LangChain, LiteLLM, and so on) also support server references in the format `server/<server-id>`. See the [SDK servers documentation](/docs/cli-reference/anaconda-ai/sdk/servers) for details.

## Basic usage

### List available models

```bash theme={null}
anaconda ai models
```

### View details for a specific model

```bash theme={null}
anaconda ai models OpenHermes-2.5-Mistral-7B
```

### Download a model

```bash theme={null}
anaconda ai download OpenHermes-2.5-Mistral-7B/Q4_K_M
```

### Launch an inference server

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M
```

The server starts and remains attached to your terminal. Press `Ctrl+C` to stop and remove the server.

<Note>
  To leave the server running in the background, use the `--detach` flag:

  ```bash theme={null}
  anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --detach
  ```
</Note>

### List running servers

```bash theme={null}
anaconda ai servers
```

### Stop a running server

```bash theme={null}
anaconda ai stop <SERVER_ID>
```

<Comments>
  Replace \<SERVER\_ID> with the ID of the server to stop. Use `anaconda ai servers` to find the server ID.
</Comments>

### Vector database

Launch a pgvector database for storing and querying embeddings:

```bash theme={null}
anaconda ai launch-vectordb
```

List, create, and manage database tables:

```bash theme={null}
anaconda ai list-tables
anaconda ai create-table my_table '{"columns": [{"name": "id", "type": "SERIAL PRIMARY KEY"}, {"name": "embedding", "type": "vector(384)"}]}'
anaconda ai drop-table my_table
```

Stop the vector database when finished:

```bash theme={null}
anaconda ai stop-vectordb
```

<Note>
  Vector database functionality requires a compatible backend. Not all backends support vector database operations.
</Note>

For the full command reference, see the [command index](/docs/cli-reference/anaconda-ai/commands).

For the Python SDK and framework integrations, see the [Anaconda AI SDK documentation](/docs/cli-reference/anaconda-ai/sdk/overview).