Skip to main content
Use Panel’s ChatInterface callback to build a chatbot that uses one of Anaconda’s models by serving it through the SDK.
The ChatInterface callback requires panel, httpx, and numpy to be installed in your environment:
    conda install panel httpx numpy
Here’s an example Panel chatbot application: 
import panel as pn
from anaconda_ai.integrations.panel import AnacondaModelHandler

pn.extension('echarts', 'tabulator', 'terminal')

llm = AnacondaModelHandler(
    'TinyLlama/TinyLlama-1.1B-Chat-v1.0_Q4_K_M.gguf', 
    display_throughput=True,
    extra_options={'ctx_size': 2048, 'temp': 0.7}
)

chat = pn.chat.ChatInterface(
    callback=llm.callback,
    show_button_name=False)

chat.send(
    "I am your assistant. How can I help you?",
    user=llm.model_id, avatar=llm.avatar, respond=False
)
chat.servable()
AnacondaModelHandler supports the following keyword arguments: For more information on using Panel, see the official documentation.