> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# anaconda ai launch

export const GCell = ({children, className}) => <div className={`grid-table-cell ${className || ""}`} role="cell">
    {children}
  </div>;

export const GTH = ({children, className}) => <div className={`grid-table-th ${className || ""}`} role="columnheader">
    {children}
  </div>;

export const GRow = ({children}) => <div className="grid-table-row" role="row">{children}</div>;

export const GBody = ({children}) => <div className="grid-table-body" role="rowgroup">{children}</div>;

export const GHead = ({children}) => <div className="grid-table-head" role="rowgroup">{children}</div>;

export const GTable = ({children, className, cols}) => <div className={`grid-table not-prose overflow-hidden rounded-2xl ${className || ""}`} style={{
  "--grid-table-cols": cols
}} role="table">
    {children}
  </div>;

export const Comments = ({children}) => {
  return <div class="my-4 px-5 py-4 overflow-hidden rounded-2xl flex gap-3 border border-zinc-500/20 bg-zinc-50/50 dark:border-zinc-500/30 dark:bg-zinc-500/10" data-callout-type="comments">
      <div class="w-4">
        <svg width="14" height="14" viewBox="0 0 640 640" fill="currentColor" xmlns="http://www.w3.org/2000/svg" class="w-5 h-5" aria-label="Comments">
            <path d="M320 112C434.9 112 528 205.1 528 320C528 434.9 434.9 528 320 528C205.1 528 112 434.9 112 320C112 205.1 205.1 112 320 112zM320 576C461.4 576 576 461.4 576 320C576 178.6 461.4 64 320 64C178.6 64 64 178.6 64 320C64 461.4 178.6 576 320 576zM280 400C266.7 400 256 410.7 256 424C256 437.3 266.7 448 280 448L360 448C373.3 448 384 437.3 384 424C384 410.7 373.3 400 360 400L352 400L352 312C352 298.7 341.3 288 328 288L280 288C266.7 288 256 298.7 256 312C256 325.3 266.7 336 280 336L304 336L304 400L280 400zM320 256C337.7 256 352 241.7 352 224C352 206.3 337.7 192 320 192C302.3 192 288 206.3 288 224C288 241.7 302.3 256 320 256z" />
        </svg>
      </div>
      <div class="text-sm prose min-w-0 w-full">
        {children}
      </div>
    </div>;
};

## Usage

```bash theme={null}
anaconda ai launch <MODEL> [OPTIONS] 
```

<Comments>
  Replace \<MODEL> with the model name and quantization.<br />
  Additional server options can be passed as `--key=value` or `--key` boolean flags.
</Comments>

## Description

Launches an OpenAI-compatible inference server for a quantized model. If the model has not been downloaded, it is downloaded automatically before the server starts.

By default, the server runs in the foreground and is stopped and removed when you press `Ctrl+C`. Use `--detach` to leave the server running in the background after the command exits.

## Arguments

<GTable cols="25% 25% 50%">
  <GHead>
    <GRow>
      <GTH>Argument</GTH>
      <GTH>Default</GTH>
      <GTH>Description</GTH>
    </GRow>
  </GHead>

  <GBody>
    <GRow>
      <GCell>`MODEL`</GCell>

      <GCell />

      <GCell>Model name with quantization (positional, required)</GCell>
    </GRow>
  </GBody>
</GTable>

## Options

<GTable cols="25% 25% 50%">
  <GHead>
    <GRow>
      <GTH>Option</GTH>
      <GTH>Default</GTH>
      <GTH>Description</GTH>
    </GRow>
  </GHead>

  <GBody>
    <GRow>
      <GCell>`--rm` / `--detach`</GCell>
      <GCell>`--rm`</GCell>
      <GCell>`--rm` stops and removes the server on `Ctrl+C`. `--detach` leaves the server running.</GCell>
    </GRow>

    <GRow>
      <GCell>`--show`</GCell>
      <GCell>`false`</GCell>
      <GCell>Open the server URL in a browser when the server starts</GCell>
    </GRow>

    <GRow>
      <GCell>`--backend <BACKEND>`</GCell>
      <GCell>config value</GCell>
      <GCell>Select the inference backend</GCell>
    </GRow>

    <GRow>
      <GCell>`--at <SITE>`</GCell>

      <GCell />

      <GCell>Target a specific site defined in your configuration</GCell>
    </GRow>

    <GRow>
      <GCell>`--json`</GCell>
      <GCell>`false`</GCell>
      <GCell>Print output as JSON</GCell>
    </GRow>

    <GRow>
      <GCell>`-h, --help`</GCell>

      <GCell />

      <GCell>Show help for this command</GCell>
    </GRow>
  </GBody>
</GTable>

### Server options

Additional server options can be appended as `--key=value` pairs or `--key` boolean flags. These options are passed directly to the backend server.

<Note>
  Server options are backend-specific. The options below apply to the `ai-navigator` and `anaconda-desktop` backends and map to [llama-server parameters](https://github.com/ggml-org/llama.cpp/tree/master/tools/server#usage). The `ai-catalyst` backend may support different options.
</Note>

Common server options for ai-navigator and anaconda-desktop include:

<GTable cols="30% 70%">
  <GHead>
    <GRow>
      <GTH>Option</GTH>
      <GTH>Description</GTH>
    </GRow>
  </GHead>

  <GBody>
    <GRow>
      <GCell>`--ctx_size=<N>`</GCell>
      <GCell>Context window size</GCell>
    </GRow>

    <GRow>
      <GCell>`--n_gpu_layers=<N>`</GCell>
      <GCell>Number of layers to offload to GPU</GCell>
    </GRow>

    <GRow>
      <GCell>`--jinja`</GCell>
      <GCell>Enable Jinja template processing</GCell>
    </GRow>
  </GBody>
</GTable>

## Examples

**Launch a server in the foreground**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M
```

**Launch in the background (detached)**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --detach
```

The command returns to the terminal immediately while the server runs in the background.

**Launch with custom server options**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --ctx_size=512 --jinja
```

**Launch and open the server in a browser**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --show
```