Skip to main content
The PydanticAI integration provides Chat and Embedding classes that automatically manage downloading models and starting servers.

​
Chat model usage

Here is a minimal setup example for using PydanticAI chat models with Anaconda AI: 
from pydantic import BaseModel
from anaconda_ai.integrations.pydantic_ai import AnacondaChatModel

class UserInfo(BaseModel):
    name: str
    age: int

model = AnacondaChatModel(
    "OpenHermes-2.5-Mistral-7B_Q4_K_M.gguf",
    extra_options={
        'ctx_size': 4096,
        'n_gpu_layers': 20,
        'temp': 0.7
    }
)
To use an already running server, pass server/<server-name> as the model name: 
model = AnacondaChatModel("server/my-server")

​
Embedding model usage

from anaconda_ai.integrations.pydantic_ai import AnacondaEmbeddingModel

embed = AnacondaEmbeddingModel(
    "sentence-transformers/bge-small-en-v1.5/q4_k_m"
)

result = await embed.embed("cat", input_type="document")
To use an already running server, pass server/<server-name> as the model name: 
embed = AnacondaEmbeddingModel("server/my-server")
For more information on using PydanticAI, see the official documentation.