tavily

Name: tavily
Author: intellectronica

web searchapi integrationai agentllm tooldata extractioninternet accessresearchbackend

⭐ 273📄 CC0-1.0🕒 2026-04-25Source ↗

Install this skill

npx skills add intellectronica/agent-skills

Works across Claude Code, Cursor, Codex, Copilot & Antigravity

The Tavily skill provides an interface to an AI-optimized search and extraction engine. Unlike standard web scrapers, it processes incoming data specifically for LLM consumption, delivering structured summaries, clean markdown content, and citation-ready search results. The skill manages four core API endpoints: search for retrieving live information, extract for parsing content from specific URLs, map for building site URL topologies, and crawl for deep traversal of domain structures. By integrating these REST calls directly into your agent, you can bypass noisy HTML or raw text responses, instead receiving refined data packets suitable for multi-step reasoning or automated research tasks. The service handles authentication via Bearer tokens and supports fine-grained control over search depth, time ranges, and domain filtering, ensuring the retrieved data remains relevant to your current project objectives.

When to Use This Skill

•Aggregating recent industry news to synthesize a weekly report
•Parsing technical documentation across multiple sub-pages for specific answers
•Researching competitive intelligence by crawling target domain structures
•Cleaning raw website content into formatted context for LLM prompt injection

How to Invoke This Skill

Example prompts that trigger this skill in Claude Code, Cursor, or Antigravity:

“Search the web for the latest updates on
“Extract the main technical specifications from this URL
“Find all pages related to X on this website
“Map the structure of the documentation site at
“Research the current sentiment regarding X using recent articles

Pro Tips

💡Always specify `search_depth` in your Tavily requests (e.g., 'basic', 'advanced') to control the thoroughness and cost of your search.
💡Leverage Tavily's `include_answer` and `include_raw_content` parameters to get both a concise summary and the full context for deeper analysis by your LLM.
💡For complex research, chain Tavily searches with internal knowledge retrieval or other tools to synthesize comprehensive answers.

What this skill does

•Execute web searches with LLM-optimized answer summaries
•Extract and convert website content into clean Markdown
•Discover URL structures and site maps recursively
•Crawl multiple pages using natural language instructions
•Filter search results by domain, date range, or topic
•Retrieve metadata such as favicons and image descriptions

When not to use it

✕Accessing content behind login-protected or highly interactive JavaScript walls
✕Performing simple lookups that can be satisfied by the LLM's internal knowledge base
✕Executing high-frequency scraping that violates destination site robots.txt policies

Example workflow

Identify a target domain for technical documentation research
Trigger the map endpoint to catalog the directory structure
Filter the results to locate specific sub-pages of interest
Call the extract tool on the relevant identified URLs
Pass the resulting Markdown content into the LLM context for synthesis

Prerequisites

–A valid Tavily API key
–Environment variable TAVILY_API_KEY configured

Pitfalls & limitations

!Over-using search depth settings can increase latency and API costs
!Crawler instructions increase processing time and financial expense per request
!Site-specific scraping may fail if the target implements strict bot detection
!Misconfiguring include_raw_content can lead to excessive context window usage

FAQ

How does Tavily differ from a standard search engine API?

Tavily processes results specifically for AI agents, providing cleaned markdown, synthesized summaries, and deduplicated source lists rather than raw search engine response objects.

Can I use this for deep crawling?

Yes, the crawl endpoint supports recursive traversal. You can define depth and breadth constraints to find specific information within a large domain structure.

What happens if I forget my API key?

The agent will identify the missing TAVILY_API_KEY environment variable and prompt you to input it before it can execute any search or extraction tasks.

Is the output always in Markdown?

The format is configurable. You can specify either markdown or text formats depending on your needs for parsing and display.

How it compares

While manual scraping requires building custom regex and DOM parsers for every site, this skill uses Tavily’s abstraction to handle site-specific quirks and formatting automatically.

Source & trust

⭐ 273 stars📄 CC0-1.0🕒 Updated 2026-04-25

View original skill on GitHub →

📄 Full skill instructions — original source: intellectronica/agent-skills

# Tavily

## Purpose

Provide a curl-based interface to Tavily’s REST API for web search, extraction, mapping, crawling, and optional research. Return structured results suitable for LLM workflows and multi-step investigations.

## When to Use

- Use when a task needs live web information, site extraction, mapping, or crawling.
- Use when web searches are needed and no built-in tool is available, or when Tavily’s LLM-friendly output (summaries, chunks, sources, citations) is beneficial.
- Use when a task requires structured search results, extraction, or site discovery from Tavily.

## Required Environment

- Require TAVILY_API_KEY in the environment.
- If TAVILY_API_KEY is missing, prompt the user to provide the API key before proceeding.

## Base URL and Auth

- Base URL: https://api.tavily.com
- Authentication: Authorization: Bearer $TAVILY_API_KEY
- Content type: Content-Type: application/json
- Optional project tracking: add X-Project-ID: <project-id> if project attribution is needed.

## Tool Mapping (Tavily REST)

### 1) search → POST /search

Use for web search with optional answer and content extraction.

Recommended minimal request:

curl -sS -X POST "https://api.tavily.com/search" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{
    "query": "<query>",
    "search_depth": "basic",
    "max_results": 5,
    "include_answer": true,
    "include_raw_content": false,
    "include_images": false
  }'

Key parameters (all optional unless noted):
- query (required): search text
- search_depth: basic | advanced | fast | ultra-fast
- chunks_per_source: 1–3 (advanced only)
- max_results: 0–20
- topic: general | news | finance
- time_range: day|week|month|year|d|w|m|y
- start_date, end_date: YYYY-MM-DD
- include_answer: false | true | basic | advanced
- include_raw_content: false | true | markdown | text
- include_images: boolean
- include_image_descriptions: boolean
- include_favicon: boolean
- include_domains, exclude_domains: string arrays
- country: country name (general topic only)
- auto_parameters: boolean
- include_usage: boolean

Expected response fields:
- answer (if requested), results[] with title, url, content, score, raw_content (optional), favicon (optional)
- response_time, usage, request_id

### 2) extract → POST /extract

Use for extracting content from specific URLs.

curl -sS -X POST "https://api.tavily.com/extract" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{
    "urls": ["https://example.com/article"],
    "query": "<optional intent for reranking>",
    "chunks_per_source": 3,
    "extract_depth": "basic",
    "format": "markdown",
    "include_images": false,
    "include_favicon": false
  }'

Key parameters:
- urls (required): array of URLs
- query: rerank chunks by intent
- chunks_per_source: 1–5 (only when query provided)
- extract_depth: basic | advanced
- format: markdown | text
- timeout: 1–60 seconds
- include_usage: boolean

Expected response fields:
- results[] with url, raw_content, images, favicon
- failed_results[], response_time, usage, request_id

### 3) map → POST /map

Use for generating a site map (URL discovery only).

curl -sS -X POST "https://api.tavily.com/map" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{
    "url": "https://docs.tavily.com",
    "max_depth": 1,
    "max_breadth": 20,
    "limit": 50,
    "allow_external": true
  }'

Key parameters:
- url (required)
- instructions: natural language guidance (raises cost)
- max_depth: 1–5
- max_breadth: 1+
- limit: 1+
- select_paths, select_domains, exclude_paths, exclude_domains: arrays of regex strings
- allow_external: boolean
- timeout: 10–150 seconds
- include_usage: boolean

Expected response fields:
- base_url, results[] (list of URLs), response_time, usage, request_id

### 4) crawl → POST /crawl

Use for site traversal with built-in extraction.

curl -sS -X POST "https://api.tavily.com/crawl" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{
    "url": "https://docs.tavily.com",
    "instructions": "Find all pages about the Python SDK",
    "max_depth": 1,
    "max_breadth": 20,
    "limit": 50,
    "extract_depth": "basic",
    "format": "markdown",
    "include_images": false
  }'

Key parameters:
- url (required)
- instructions: optional; raises cost and enables chunks_per_source
- chunks_per_source: 1–5 (only with instructions)
- max_depth, max_breadth, limit: same as map
- extract_depth: basic | advanced
- format: markdown | text
- include_images, include_favicon, allow_external
- timeout: 10–150 seconds
- include_usage: boolean

Expected response fields:
- base_url, results[] with url, raw_content, favicon
- response_time, usage, request_id

## Optional Research Workflow (Deep Investigation)

Use when a query needs multi-step analysis and citations.

### create research task → POST /research

curl -sS -X POST "https://api.tavily.com/research" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{
    "input": "<research question>",
    "model": "auto",
    "stream": false,
    "citation_format": "numbered"
  }'

Expected response fields:
- request_id, created_at, status (pending), input, model, response_time

### get research status → GET /research/{request_id}

curl -sS -X GET "https://api.tavily.com/research/<request_id>" \
  -H "Authorization: Bearer $TAVILY_API_KEY"

Expected response fields:
- status: completed
- content: report text or structured object
- sources[]: { title, url, favicon }

### streaming research (SSE)

Set "stream": true in the POST body and use curl with -N to stream events:

curl -N -X POST "https://api.tavily.com/research" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TAVILY_API_KEY" \
  -d '{"input":"<question>","stream":true,"model":"pro"}'

Handle SSE events (tool calls, tool responses, content chunks, sources, done).

## Usage Notes

- Treat search, extract, map, and crawl as the primary endpoints for discovery and content retrieval.
- Return structured results with URLs, titles, and summaries for easy downstream use.
- Default to conservative parameters (search_depth: basic, max_results: 5) unless deeper recall is needed.
- Reuse consistent request bodies across calls to keep results predictable.

## Error Handling

- If any request returns 401/403, prompt for or re-check TAVILY_API_KEY.
- If timeouts occur, reduce max_depth/limit or use search_depth: basic.
- If responses are too large, lower max_results or chunks_per_source.

By intellectronica

How to Use This Skill Unit

Option A: Project-Specific (Recommended)

Click "Download" above
In your project, create the directory: .agent/skills/tavily/
Save the file as SKILL.md
The agent will automatically discover the skill based on its description.

Option B: Global Installation (All Agents)

Save the file to these locations to make it available across all projects:

Claude Code: ~/.claude/skills/intellectronica/agent-skills/tavily/SKILL.md
Cursor: ~/.cursor/skills/intellectronica/agent-skills/tavily/SKILL.md
Antigravity: ~/.gemini/antigravity/skills/intellectronica/agent-skills/tavily/SKILL.md

🚀 Install with CLI:
npx skills add intellectronica/agent-skills

Read the Master Guide: Mastering Agent Skills →

Recommended Rules

View more rules →

Recommended Workflows

View more workflows →

Fix CORS Issues

APICORSBackend

--- description: Resolve Cross-Origin Resource Sharing errors in API calls --- 1. **Understand the Error**: - CORS errors occur when frontend (htt...

Setup Redis Caching

BackendCachingPerformance

--- description: Implement Redis caching (Upstash or self-hosted) --- 1. **Option A: Upstash Redis (Serverless Recommended)**: - Best for Vercel/E...

Automatic commit message generator

GitAIAutomation

--- description: Automatic commit message generator and fast AI-powered commit for all current changes --- // turbo-all This workflow automatically ...

Recommended MCP Servers

View more MCP servers →

yfinance

Community

💹The MCP YFinance Stock Server provides real-time and historical stock data in a standard format, powering dashboards, AI agents,and research tools with seamless financial insights.