While xpander.ai offers a serverless solution where you can use AI agents without managing LLM infrastructure, you might want to use your own LLM deployment for:

  • Running models locally for lower latency
  • Using specific model versions or custom fine-tuned models
  • Implementing local function calling
  • Maintaining full control over your LLM stack
  • Handling sensitive data locally
1

Choose Your LLM Provider

To integrate your LLM with xpander’s tools and local functions, you’ll need to select one LLM provider and model combination. Import the necessary packages and initialize your client:

2

Get Tools to the LLM

Configure your LLM client with the appropriate tools based on your provider:

3

Run Tools for the LLM

Process the LLM response and execute the tools:

4

Understanding extract_tool_calls

The extract_tool_calls method is a crucial utility that handles the conversion between different LLM function/tool calling formats. Here’s how it works:

  1. Purpose: It standardizes various LLM response formats into a unified format that xpander can process:
# Example of how extract_tool_calls processes different formats
# OpenAI format
openai_response = {
    "tool_calls": [
        {
            "function": {"name": "get_weather", "arguments": '{"location": "London"}'}
        }
    ]
}

# Bedrock/Claude format
bedrock_response = {
    "function_calls": [
        {"name": "get_weather", "arguments": {"location": "London"}}
    ]
}

# Both get converted to a standardized internal format
standardized_calls = [
    {
        "name": "get_weather",
        "arguments": {"location": "London"}
    }
]
  1. Provider-Specific Handling: The method automatically detects and processes different response structures:
# The llm_provider parameter tells extract_tool_calls how to parse the response
tools_to_run = XpanderClient.extract_tool_calls(
    llm_response=llm_response,
    llm_provider=LLMProvider.OPENAI  # or AMAZON_BEDROCK, OLLAMA, NVIDIA_NIM
)
  1. Automatic API Translation: Once standardized, these tool calls can be executed uniformly:
# The standardized format allows consistent execution across providers
tool_responses = agent1.run_tools(tools_to_run)

# Each tool_response contains the result of the API call
# Example response:
# {
#     "name": "get_weather",
#     "result": {"temperature": 20, "condition": "sunny"}
# }

Need help? Visit our Discord community or documentation.

Build AI agentsLocal Tools Guide