Streamlining AI Agent Functionality: Understanding Server-Side Tools and Their Benefits

Every AI agent faces a common challenge: while the model can process information, it requires external tools to execute actions. Typically, the execution of these tools—fetching search results, querying databases, or calling APIs—is handled by the underlying code.

Most development teams tackle this by having the model issue a tool call, which the code intercepts, executes, and formats before the result is returned. This cycle continues until the model gathers sufficient information to respond. While effective, this approach means the team must manage the entire tool layer, including connections, credentials, retry logic, error handling, and observability. These components are essential yet not part of the core product.

An alternative is integrating tool execution into the inference layer, allowing tools to operate within the API call. This method simplifies the code while transferring the tool execution responsibility to the server-side.

Key Takeaways

• Simplified Code: While server-side execution simplifies your code, you must still manage failures, performance, and retry safety.
• Development vs. Production: Client-side tools are ideal for development due to their visibility and local debugging capabilities, whereas server-side tools are suited for production by reducing infrastructure overhead.
• Cold Start Management: Server-side execution requires maintaining warm MCP servers to avoid startup delays in responses.
• Differentiating Failures: It's crucial to distinguish between tool failures, which are infrastructure issues, and agent failures, which are prompt-related.
• Tracing Setup: Implement tracing early to identify which tool calls cause failures during production.
• Public MCP Servers: For server-side execution, MCP servers must be publicly accessible, unlike private network setups that use client-side MCP.

What Server-Side Tools Are

Server-side tools for inference engines allow tool execution as part of the inference request, using an existing Model Access Key without additional credentials or API changes. These tools support several external capabilities:

1. Web Search: Powered by a neural search index, it allows real-time searches controlled by the model.
2. Web Fetch: Extracts and cleans content from URLs, optimizing token processing.
3. Knowledge Base Retrieval: Enables querying of private data repositories.
4. Customer-owned MCP Servers: Connects models to remote MCP servers, managing connections and tool executions.
5. Tool Search: Efficiently loads tool definitions, reducing token costs when numerous tools are in use.

## Example of Tool Search with Anthropic (Messages API)
import anthropic

client = anthropic.Anthropic(
    base_url="https://inference.do-ai.run/v1",
    api_key="your-model-access-key"
)

response = client.messages.create(
    model="anthropic-claude-opus-4.8",
    max_tokens=2048,
    messages=[{"role": "user", "content": "What is the weather in zip code 94107?"}],
    tools=[
        {
            "type": "tool_search_tool_regex_20251119",
            "name": "tool_search_tool_regex"
        },
        {
            "name": "get_weather_by_zip",
            "description": "Return current weather conditions for a US zip code.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "zip_code": {"type": "string"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                },
                "required": ["zip_code"]
            },
            "defer_loading": True
        }
    ]
)

In server-side execution, tasks traditionally executed in a multi-step process are handled within a single API call, streamlining operations and improving efficiency.

Use Cases

Research Agents

These agents handle tasks requiring real-time information such as pricing updates, product changes, and news, leveraging web search and fetching capabilities.

response = client.chat.completions.create(
    model="openai-gpt-4o",
    messages=[{
        "role": "user",
        "content": "Summarize the key changes to Kubernetes networking in the last 6 months and their implications for teams running microservices."
    }],
    tools=[{
        "type": "web_search",
        "max_uses": 5,
        "max_results": 5
    }]
)

AI Apps Connecting to Internal Systems

Applications utilizing internal MCP servers can easily integrate these with inference requests, allowing tools like project management and customer relationship management systems to be accessed seamlessly.

response = client.chat.completions.create(
    model="openai-gpt-4o",
    messages=[{
        "role": "user",
        "content": "Pull the open support tickets for our enterprise tier and draft a weekly digest for the team."
    }],
    tools=[{
        "type": "mcp",
        "server_label": "internal-crm",
        "server_url": "https://tools.yourcompany.com/mcp",
        "authorization": "Bearer your-mcp-token",
        "allowed_tools": ["list-tickets", "get-ticket-details", "get-customer-info"]
    }]
)

Multi-Step Agents Combining Tools

Complex agents can combine multiple tools to conduct thorough analyses or due diligence, searching for news, product details, and internal knowledge base data in one inference call.

The Latency Tradeoff

When executing tools client-side, the application runs the tool, requiring additional steps. Server-side execution, however, completes the process within a single request, which may add network latency. The choice between client-side and server-side depends on whether you prioritize infrastructure management or response speed.

Client-side Execution

• Fast for local processes but requires managing each step.
• Examples include validation or formatting tasks, typically under 1ms.

Server-side Execution

• Integrated tool execution with added network latency.
• Suitable for applications where infrastructure management is a higher priority than latency, such as background tasks.

When Tool Latency Adds Up

The latency of tool calls can accumulate, particularly if multiple calls are made sequentially, which can affect user-facing applications where prompt responses are critical.

Building Production-Ready Agents with Server-Side Tools

When adopting server-side tools, consider these strategies:

1. Explicit Instructions for Failures: Define how to handle tool failures in the system prompt to prevent incomplete responses.
2. Safe Retry Operations for MCP: Design write operations with unique request IDs for safe retries.
3. Identify Failed Tool Layers: Differentiate between infrastructure and model-related failures to streamline troubleshooting.

When to Use Client-Side vs. Server-Side

Use Server-Side Tools When:

• You prefer not to manage tool infrastructure.
• Credentials need to be kept closer to their respective tools.
• Multiple users or agents require access to the same tools.
• Real-time retrieval or web search is necessary.
• You are already utilizing an inference service.

Use Client-Side Tools When:

• Development requires complete transparency and control.
• Tools are fast and local, without network dependencies.
• Latency budgets are strict.
• Tools are dependent on local state or data.
• Control over retry behaviors is essential.

Practical Approach

Start with client-side tools to gain insight into failure modes, then transition to server-side tools in production when infrastructure management becomes burdensome.

Comparison with Other Approaches

Compared to other frameworks or APIs, server-side tools provide managed execution, reducing infrastructure responsibilities while maintaining tool flexibility across different models. The choice between managed and client-side tools depends on the need for infrastructure control versus execution visibility.

Server-Side MCP vs. Client-Side MCP

Client-side MCP

• Application manages the connection and execution.
• Offers full visibility and control over tool calls and results.
• Ideal when tools and data are local or behind firewalls.

Server-side MCP

• Managed by the inference service, reducing individual setup requirements.
• Requires public accessibility for MCP servers.
• Suitable for shared tool access among multiple agents.

Enhancing Observability

To maintain visibility when tools execute outside your process:

1. Implement Tracing: Use tracing APIs to monitor tool execution details.
2. Separate Latency Metrics: Track tool latency separately from total response times.
3. Differentiate Failures: Log distinct metrics for tool and agent failures to streamline troubleshooting.

FAQs

• Available Server-Side Tools: Include web search, web fetch, knowledge base retrieval, MCP servers, and tool search.
• Cost Considerations: Certain tools like web search incur additional costs.
• Mixing Tool Approaches: Combining client-side and server-side tools in a single agent is possible.
• MCP Server Requirements: For server-side execution, MCP servers must be publicly accessible.
• Handling Tool Failures: Models continue processing despite tool failures; clear instructions are necessary.
• Debugging Slow Responses: Use tracing APIs to identify bottlenecks in tool execution.
• Tool Search Benefits: Particularly useful for agents with numerous tools, reducing token usage.

Conclusion

Shifting tool execution to server-side changes the complexity landscape, transferring infrastructure responsibilities to a managed service while maintaining the core challenges of agent design. This approach is beneficial when infrastructure overhead becomes a limiting factor, whereas client-side execution offers greater visibility and control during development.