> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# API server
export const Comments = ({children}) => {
return
;
The built-in API server enables you to run open-source large language models (LLMs) locally on your own machine. It facilitates direct interaction with your models through API calls, providing a powerful tool for testing and deploying models without the need for external cloud services. By hosting your own API server locally, you have full control over the model's behavior and maintain the privacy and security of your data.
## Understanding the API server
The API server is the core component of Anaconda AI Navigator that enables you to interact with your locally downloaded LLMs through API calls.
Hover over a tooltip to view information about the API Server's configurable parameters.
### Server address
Your local server address, often referred to as `localhost` or `127.0.0.1`, is the default address for the API server. It is a loopback address, meaning any network communication sent to this address will stay within the same machine, keeping your data secure and private. This is common for developing applications.
If you are working in an office network and would like to make your server accessible to other devices within the same network, you can set the server address to your machine's local IP address. These are typically private networks, meaning they're not routable on the public internet.
Setting the server address to `0.0.0.0` configures it to accept connections on all network interfaces. This can expose your server to the public internet if your network is configured to allow external connections and route traffic from external sources to your server.
### Server port
The server port tells the built-in API server where to listen for incoming traffic (that is, where to listen and pick up API requests).
For more information, see [Cloudflare's article on ports](https://www.cloudflare.com/learning/network-layer/what-is-a-computer-port/).
### API Key
Establish an API Key that must be passed as an `Authentication: Bearer` token in the header of requests made to your server.
To test an API Key:
1. Select **API Server** from the left-hand navigation.
2. Enter your API Key in AI Navigator.
3. [Load a model into the API server](/tools/ai-navigator/getting-started-ai-navigator#loading-a-model-into-the-api-server).
You must use a `text-generation` type model.
4. Open [Anaconda Prompt](/reference/glossary#anaconda-prompt) (Terminal on macOS/Linux) and run the following command:
```sh Anaconda Prompt theme={null}
curl --request POST ^
--url http://localhost:8080/completion ^
--header 'Authorization: Bearer ' ^
--header "Content-Type: application/json" ^
--data '{
"prompt": "Hello, how are you?"
}'
```
```sh Anaconda PowerShell Prompt theme={null}
curl --request POST `
--url http://localhost:8080/completion `
--header 'Authorization: Bearer ' `
--header "Content-Type: application/json" `
--data '{
"prompt": "Hello, how are you?"
}'
```
```sh Terminal theme={null}
curl --request POST \
--url http://localhost:8080/completion \
--header 'Authorization: Bearer ' \
--header "Content-Type: application/json" \
--data '{
"prompt": "Hello, how are you?"
}'
```
Replace \ with the value you entered into AI Navigator.
This command will reach out to the local API servers `/completion` endpoint to interact with the model you have loaded. If you receive a response, your API Key is working. If you receive a 401 error, double check your command and try again.
### Model won't load (exit code 3221225781)
The model you are trying to load requires a C++ runtime environment, but Microsoft does not include one in their operating systems by default.
Install the Latest [Microsoft Visual C++ Redistributable Version](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version).
## Viewing API server logs
To view your server logs:
1. Open Anaconda AI Navigator.
2. Select **API Server** from the left-hand navigation.
The server logs provide information for the following metrics:
* **System information**: Provides information about your system's hardware capabilities.
* **Build information**: Provides information about the version of the API server you are using.
* **Chat template**: Shows the sequence of messaging defined for the system.
* **Server listening**: Displays the server address and port number being listened to.
* **Slot information**: Displays the number of available slots for the server. Each slot is able to manage one user API request at a time.
* **Task information**: Displays information such as time spent processing and responding to a request, the request task ID, and which slot fulfilled the task.
The API server records all incoming traffic for the currently running server and displays relevant information in the **Server Logs**.
Once the server is stopped, the log is saved to `~/Library/Logs/ai-navigator` with a timestamped filename (for example `
_api-server.log`), so you can efficiently locate specific server logs if necessary.