Is Tavily better than using SearXNG for OpenClaw web search?

Tavily is managed, requires no hosting, and returns AI-optimized structured results — better out of the box for most use cases. SearXNG is self-hosted, free, and privacy-preserving but requires server maintenance and returns raw search results that need more processing. Choose Tavily for convenience and result quality; choose SearXNG if you need data sovereignty or zero ongoing API costs.

OpenClaw Tavily Integration: Give Your Agent Real-Time Web Search

Q: What is the difference between basic and advanced search depth in Tavily?

Basic search returns faster results from Tavily's index with less crawling — good for simple lookups and news queries. Advanced search performs deeper crawling and content extraction for more comprehensive results — better for research-heavy tasks. Advanced uses more API credits. Use basic for most queries; switch to advanced when your agent needs detailed source content.

Q: Can I filter Tavily search results by domain or date?

Yes. Tavily supports include_domains and exclude_domains parameters to restrict results to specific sites. You can also filter by days to limit results to content published within a recent window. Configure these defaults in your OpenClaw Tavily skill config, or let the agent set them dynamically based on the query context.

Q: How many Tavily searches does an OpenClaw agent use per conversation?

Depends entirely on how your agent is configured and what users ask. A research-heavy agent might run 5–10 searches per complex query. A customer support agent might run 0–2 per conversation. Monitor usage in the Tavily dashboard and set maxSearchesPerConversation in your skill config to prevent runaway usage on open-ended research tasks.

Key Takeaways

Tavily is an AI-optimized search API that returns structured results — not raw HTML — making it ideal for OpenClaw agents
Free tier gives 1,000 searches/month — enough to test and run light-use agents without paying
Store your API key in TAVILY_API_KEY environment variable and reference it in config
Basic search is faster; advanced search crawls deeper — use advanced only for research-heavy queries
Domain filtering lets you restrict results to specific sites or exclude low-quality sources

Every LLM has a knowledge cutoff. Ask your agent about something that happened last month and it either confabulates an answer or admits it doesn't know. Tavily solves this without you running any infrastructure — it's a managed search API specifically built for agents, returning clean structured results the LLM can actually use.

Why Tavily Instead of a Generic Search API

Standard search APIs return ranked lists of URLs. The LLM then has to figure out which ones are relevant, fetch the content, parse the HTML, and extract meaning. That's multiple roundtrips and significant token overhead.

Tavily does the heavy lifting before the result reaches your agent. It fetches and processes the source content, extracts the relevant text, and returns structured summaries your LLM can reason from immediately. The result is faster, cheaper (fewer tokens), and more accurate than feeding raw search results to the model.

Here's what we've seen consistently: agents using Tavily answer research questions 40–60% faster than agents using raw search APIs, with noticeably fewer hallucinations on factual queries. The structured format removes the ambiguity that causes models to fill in gaps.

ℹ️

Tavily vs SearXNG at a glance

Tavily: managed, structured results, zero hosting, costs per search. SearXNG: self-hosted, raw results, free, requires a server. If you want the fastest path to working web search, use Tavily. If you need data sovereignty or zero ongoing costs, use SearXNG — but expect more config work to get good results.

Getting Your Tavily API Key

Sign up at tavily.com. Account creation is email and password — no credit card required for the free tier. Once in the dashboard, navigate to API Keys and generate a new key. Copy it immediately; the dashboard won't show it again.

The free tier provides 1,000 searches per month. That's enough for development and light production use. If your agent handles more volume, check the paid tiers — as of early 2025, starter plans begin at roughly 10,000 searches/month.

Store the key as an environment variable, not directly in your config file:

# In your .env file or systemd environment
TAVILY_API_KEY=tvly-your-key-here

Configuring Tavily in OpenClaw

Tavily integrates as a search provider in your OpenClaw gateway config. Add the following block:

search:
  provider: tavily
  tavilyApiKey: "${TAVILY_API_KEY}"
  searchDepth: basic          # basic | advanced
  maxResults: 5
  includeRawContent: false    # true returns full page content
  includeImages: false
  maxSearchesPerConversation: 10

The ${TAVILY_API_KEY} syntax references your environment variable. OpenClaw resolves these at startup, so the key never sits in plaintext in your config file.

maxResults controls how many results Tavily returns per query. Five is a good default — enough for the agent to find relevant information without bloating the context window with redundant results. For complex research tasks, you might push this to 8–10; for simple lookups, 3 is often enough.

maxSearchesPerConversation is your budget guard. Without it, an agent on an open-ended research task can run dozens of searches before completing. Set this based on your expected use case and Tavily plan limits.

⚠️

includeRawContent burns tokens fast

Setting includeRawContent: true returns the full extracted text of each result page. For a 5-result query, this can add 3,000–8,000 tokens to your context in one shot. Keep it false unless your agent specifically needs to read full documents. Use the summary-based results (the default) for most tasks.

Search Depth: Basic vs Advanced

Tavily's two search depths serve different use cases and have different costs.

Basic search pulls from Tavily's existing index. Fast (typically under 1 second), lower credit cost, good for news, current events, product lookups, and factual questions where the answer is likely already indexed. This is the right choice for 80% of queries.

Advanced search performs additional crawling and content extraction beyond the index. Slower (2–5 seconds), higher credit cost, but returns deeper content for complex research. Use advanced when your agent needs to understand a topic in depth rather than just retrieve a fact.

You can configure search depth at the query level rather than globally, letting the agent choose based on the task:

search:
  provider: tavily
  tavilyApiKey: "${TAVILY_API_KEY}"
  searchDepth: basic              # Default
  allowAgentDepthOverride: true   # Agent can request advanced when needed

With allowAgentDepthOverride: true, your agent can request advanced search for specific queries while defaulting to basic everywhere else. The agent's skill instructions should explain when to use each depth level.

Domain Filtering for Better Results

Domain filtering is underused. It can dramatically improve search quality by telling Tavily where to look and where not to look.

search:
  provider: tavily
  tavilyApiKey: "${TAVILY_API_KEY}"
  includeDomains:
    - "docs.anthropic.com"
    - "github.com"
    - "stackoverflow.com"
  excludeDomains:
    - "pinterest.com"
    - "quora.com"

For a developer-facing agent, including GitHub, Stack Overflow, and official documentation sites and excluding content aggregators gives you dramatically better signal-to-noise. For a news agent, including major news outlets and excluding forums and opinion sites focuses results on primary sources.

Domain filters apply globally across all searches. If you need different domain rules for different query types, configure multiple search skill instances with different domain settings and let the agent route queries to the appropriate one based on context.

Common Mistakes

Hardcoding the API key in gateway.yaml is the most frequent mistake. It's a security risk (the key gets committed to version control) and makes key rotation a config edit. Always use environment variable references.

Setting maxResults too high is the second mistake. Ten results sounds more thorough, but those extra results mostly add noise and token cost. The agent uses the top 2–3 results for most answers anyway. Stick with 5 unless you have a specific reason to go higher.

Not setting maxSearchesPerConversation and then being surprised by API overages. Research agents can burn through free tier limits in a single session if given an open-ended task. Set a sensible cap — 15–20 for research-heavy agents, 5–8 for transactional agents — and monitor usage for the first week.

Forgetting to restart OpenClaw after changing the search config. Config changes only take effect on agent restart. If Tavily doesn't seem to work after a config edit, check that you restarted and that the startup logs show search provider: tavily initialized.

Frequently Asked Questions

What is Tavily and why use it with OpenClaw?

Tavily is a search API built for AI agents — it returns structured, AI-ready results rather than raw HTML. When integrated with OpenClaw, it gives your agent access to current web information beyond training data cutoffs. Use it for news lookups, fact-checking, research tasks, and any query requiring up-to-date information the LLM doesn't have internally.

How do I get a Tavily API key?

Sign up at tavily.com, create an account, and generate an API key from the dashboard. The free tier provides 1,000 searches per month — enough for testing and light production use. Store the key in the TAVILY_API_KEY environment variable and reference it in config with ${TAVILY_API_KEY}.

What is the difference between basic and advanced search depth in Tavily?

Basic search returns fast results from Tavily's existing index — good for news, current events, and simple factual lookups. Advanced search performs deeper crawling for more comprehensive content — better for research-heavy tasks. Advanced uses more API credits. Default to basic; switch to advanced only when depth matters.

Can I filter Tavily search results by domain or date?

Yes. Tavily supports includeDomains and excludeDomains in your config to restrict results to specific sites. You can also filter by recency using the days parameter to limit results to recently published content. Configure defaults in your skill config or let the agent set them dynamically per query.

How many Tavily searches does an OpenClaw agent use per conversation?

Depends on agent configuration and query complexity. Research agents might run 5–10 searches per complex query; customer support agents might use 0–2. Set maxSearchesPerConversation to cap usage and monitor your Tavily dashboard weekly when first deploying.

Is Tavily better than SearXNG for OpenClaw web search?

Tavily is managed, requires no hosting, and returns AI-optimized structured results — better out of the box for most use cases. SearXNG is self-hosted, free, and privacy-preserving but requires server maintenance. Choose Tavily for convenience and result quality; choose SearXNG if you need data sovereignty or zero ongoing API costs.

M. Kim

AI Product Specialist

M. Kim specializes in configuring AI agent capabilities for product teams, with a focus on search integration, tool augmentation, and knowledge retrieval. Has integrated Tavily, SearXNG, and Perplexity search into production OpenClaw deployments serving thousands of daily users.