> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Servers
export const GCell = ({children, className}) =>
{children}
;
export const GTH = ({children, className}) =>
{children}
;
export const GRow = ({children}) => {children}
;
export const GBody = ({children}) => {children}
;
export const GHead = ({children}) => {children}
;
export const GTable = ({children, className, cols}) =>
{children}
;
The `.servers` accessor provides methods for creating, listing, starting, and stopping servers.
Some methods like `.match()` and the parameter classes below are backend-specific and may not be available on all backends.
Method
Return
Description
`.list()`
`List[Server]`
List running servers
`.get('')`
`Server`
Look up a running server by name or UUID
`.match()`
`Server`
Find a running server matching configuration
`.create(model, extra_options=None)`
`Server`
Create new server configuration
`.start('')`
`None`
Start the API server
`.status('')`
`str`
Get server status
`.stop('')`
`None`
Stop a running server
`.delete('')`
`None`
Remove server configuration
## Creating servers
The `.create()` method creates a new server configuration. By default, it downloads the model file (if not already downloaded) and selects a random, unused port for the server.
If a server with the specified configuration is already running, the existing configuration is returned, and no new server is created.
Create servers using one of the following approaches:
```py theme={null}
from anaconda_ai import AnacondaAIClient
client = AnacondaAIClient()
server = client.servers.create('OpenHermes-2.5-Mistral-7B/Q4_K_M')
```
```py theme={null}
from anaconda_ai import AnacondaAIClient
client = AnacondaAIClient()
quant = client.models.get('OpenHermes-2.5-Mistral-7B').get_quantization('Q4_K_M')
server = client.servers.create(quant)
```
To look up an existing running server rather than create one, use `.get()`. Pass the server's name or UUID as a string:
```py theme={null}
server = client.servers.get('my-server-name')
```
## Server configuration parameters
The `.create()` method accepts an `extra_options` parameter to customize server behavior:
```py theme={null}
def create(
model: str,
download_if_needed: bool = True,
extra_options: Optional[Dict[str, Any]] = None,
show_progress: bool = True,
console: Optional[Console] = None
) -> Server:
```
Server configuration options are backend-specific. The parameters below apply to the `ai-navigator` and `anaconda-desktop` backends.
Common server options include:
Parameter
Type
Description
`host`
`str`
Server host address (default: "127.0.0.1")
`port`
`int`
Server port (0 for random unused port)
`ctx_size`
`int`
Context window size
`n_gpu_layers`
`int`
Number of layers to offload to GPU
`main_gpu`
`int`
Main GPU device ID
`temp`
`float`
Sampling temperature
`top_k`
`int`
Top-k sampling parameter
`top_p`
`float`
Top-p (nucleus) sampling parameter
`batch_size`
`int`
Batch size for processing
`embedding`
`bool`
Enable embedding mode for embedding models
`jinja`
`bool`
Enable Jinja template processing
For example, to create a server with custom configuration:
```py theme={null}
server = client.servers.create(
'OpenHermes-2.5-Mistral-7B/Q4_K_M',
extra_options={
'host': '127.0.0.1',
'port': 9999,
'ctx_size': 4096,
'n_gpu_layers': 20,
'temp': 0.7,
'top_p': 0.9
},
download_if_needed=False
)
```
## Managing servers
New servers are not automatically started when their configuration is created. You can start or stop a server using the following methods:
```python theme={null}
server.start()
server.stop()
```
```python theme={null}
# Start or stop the server object returned from .create
client.servers.start(server)
client.servers.stop(server)
```
This approach automatically starts and stops the server:
```python theme={null}
with client.servers.create('OpenHermes-2.5-Mistral-7B/Q4_K_M') as server:
openai_client = server.openai_client()
# Make requests to the server
```
## Server attributes
Attribute
Description
`.url`
Full server URL (example: `http://127.0.0.1:8000`)
`.openai_url`
URL with `/v1` for OpenAI compatibility
`.openai_client()`
Pre-configured OpenAI client
`async_openai_client()`
Pre-configured async OpenAI client