Search API on Producthunt daily

open_deep_research

Tue, 22 Jul 2025 15:33:16 +0800

langchain-ai/open_deep_research

Open Deep Research

Deep research has broken out as one of the most popular agent applications. This is a simple, configurable, fully open source deep research agent that works across many model providers, search tools, and MCP servers.

Read more in our blog
See our video for a quick overview

🚀 Quickstart

Clone the repository and activate a virtual environment:

git clone https://github.com/langchain-ai/open_deep_research.git
cd open_deep_research
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Install dependencies:

`1`	`uv pip install -r pyproject.toml`

Set up your .env file to customize the environment variables (for model selection, search tools, and other configuration settings):

`1`	`cp .env.example .env`

Launch the assistant with the LangGraph server locally to open LangGraph Studio in your browser:

1
2

# Install dependencies and start the LangGraph server
uvx --refresh --from "langgraph-cli[inmem]" --with-editable . --python 3.11 langgraph dev --allow-blocking

Use this to open the Studio UI:

1
2
3

- 🚀 API: http://127.0.0.1:2024
- 🎨 Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024
- 📚 API Docs: http://127.0.0.1:2024/docs

Ask a question in the messages input field and click Submit.

Configurations

Open Deep Research offers extensive configuration options to customize the research process and model behavior. All configurations can be set via the web UI, environment variables, or by modifying the configuration directly.

General Settings

Max Structured Output Retries (default: 3): Maximum number of retries for structured output calls from models when parsing fails
Allow Clarification (default: true): Whether to allow the researcher to ask clarifying questions before starting research
Max Concurrent Research Units (default: 5): Maximum number of research units to run concurrently using sub-agents. Higher values enable faster research but may hit rate limits

Research Configuration

Search API (default: Tavily): Choose from Tavily (works with all models), OpenAI Native Web Search, Anthropic Native Web Search, or None
Max Researcher Iterations (default: 3): Number of times the Research Supervisor will reflect on research and ask follow-up questions
Max React Tool Calls (default: 5): Maximum number of tool calling iterations in a single researcher step

Models

Open Deep Research uses multiple specialized models for different research tasks:

Summarization Model (default: openai:gpt-4.1-nano): Summarizes research results from search APIs
Research Model (default: openai:gpt-4.1): Conducts research and analysis
Compression Model (default: openai:gpt-4.1-mini): Compresses research findings from sub-agents
Final Report Model (default: openai:gpt-4.1): Writes the final comprehensive report

All models are configured using init_chat_model() API which supports providers like OpenAI, Anthropic, Google Vertex AI, and others.

Important Model Requirements:

Structured Outputs: All models must support structured outputs. Check support here.
Search API Compatibility: Research and Compression models must support your selected search API:
- Anthropic search requires Anthropic models with web search capability
- OpenAI search requires OpenAI models with web search capability
- Tavily works with all models
Tool Calling: All models must support tool calling functionality
Special Configurations:
- For OpenRouter: Follow this guide
- For local models via Ollama: See setup instructions

Example MCP (Model Context Protocol) Servers

Open Deep Research supports MCP servers to extend research capabilities.

Local MCP Servers

Filesystem MCP Server provides secure file system operations with robust access control:

Read, write, and manage files and directories
Perform operations like reading file contents, creating directories, moving files, and searching
Restrict operations to predefined directories for security
Support for both command-line configuration and dynamic MCP roots

Example usage:

`1`	`mcp-server-filesystem /path/to/allowed/dir1 /path/to/allowed/dir2`

Remote MCP Servers

Remote MCP servers enable distributed agent coordination and support streamable HTTP requests. Unlike local servers, they can be multi-tenant and require more complex authentication.

Arcade MCP Server Example:

{
  "url": "https://api.arcade.dev/v1/mcps/ms_0ujssxh0cECutqzMgbtXSGnjorm",
  "tools": ["Search_SearchHotels", "Search_SearchOneWayFlights", "Search_SearchRoundtripFlights"]
}

Remote servers can be configured as authenticated or unauthenticated and support JWT-based authentication through OAuth endpoints.

Evaluation

A comprehensive batch evaluation system designed for detailed analysis and comparative studies.

Features:

Multi-dimensional Scoring: Specialized evaluators with 0-1 scale ratings
Dataset-driven Evaluation: Batch processing across multiple test cases

Usage:

1
2

# Run comprehensive evaluation on LangSmith datasets
python tests/run_evaluate.py

Key Files:

tests/run_evaluate.py: Main evaluation script
tests/evaluators.py: Specialized evaluator functions
tests/prompts.py: Evaluation prompts for each dimension

Deployments and Usages

LangGraph Studio

Follow the quickstart to start LangGraph server locally and test the agent out on LangGraph Studio.

Hosted deployment

You can easily deploy to LangGraph Platform.

Open Agent Platform

Open Agent Platform (OAP) is a UI from which non-technical users can build and configure their own agents. OAP is great for allowing users to configure the Deep Researcher with different MCP tools and search APIs that are best suited to their needs and the problems that they want to solve.

We’ve deployed Open Deep Research to our public demo instance of OAP. All you need to do is add your API Keys, and you can test out the Deep Researcher for yourself! Try it out here

You can also deploy your own instance of OAP, and make your own custom agents (like Deep Researcher) available on it to your users.

Updates 🔥

Legacy Implementations 🏛️

The src/legacy/ folder contains two earlier implementations that provide alternative approaches to automated research:

1. Workflow Implementation (`legacy/graph.py`)

Plan-and-Execute: Structured workflow with human-in-the-loop planning
Sequential Processing: Creates sections one by one with reflection
Interactive Control: Allows feedback and approval of report plans
Quality Focused: Emphasizes accuracy through iterative refinement

2. Multi-Agent Implementation (`legacy/multi_agent.py`)

Supervisor-Researcher Architecture: Coordinated multi-agent system
Parallel Processing: Multiple researchers work simultaneously
Speed Optimized: Faster report generation through concurrency
MCP Support: Extensive Model Context Protocol integration

See src/legacy/legacy.md for detailed documentation, configuration options, and usage examples for both legacy implementations.

exa-mcp-server

Mon, 28 Apr 2025 15:56:41 +0800

exa-labs/exa-mcp-server

Exa MCP Server 🔍

A Model Context Protocol (MCP) server lets AI assistants like Claude use the Exa AI Search API for web searches. This setup allows AI models to get real-time web information in a safe and controlled way.

Demo video https://www.loom.com/share/ac676f29664e4c6cb33a2f0a63772038?sid=0e72619f-5bfc-415d-a705-63d326373f60

What is MCP? 🤔

The Model Context Protocol (MCP) is a system that lets AI apps, like Claude Desktop, connect to external tools and data sources. It gives a clear and safe way for AI assistants to work with local services and APIs while keeping the user in control.

What does this server do? 🚀

The Exa MCP server:

Enables AI assistants to perform web searches using Exa’s powerful search API
Provides structured search results including titles, URLs, and content snippets
Caches recent searches as resources for reference
Handles rate limiting and error cases gracefully
Supports real-time web crawling for fresh content

Prerequisites 📋

Before you begin, ensure you have:

Node.js (v18 or higher)
Claude Desktop installed
An Exa API key
Git installed

You can verify your Node.js installation by running:

`1`	`node --version # Should show v18.0.0 or higher`

Installation 🛠️

NPM Installation

`1`	`npm install -g exa-mcp-server`

Using Smithery

To install the Exa MCP server for Claude Desktop automatically via Smithery:

`1`	`npx -y @smithery/cli install exa --client claude`

Manual Installation

Clone the repository:

1
2

git clone https://github.com/exa-labs/exa-mcp-server.git
cd exa-mcp-server

Install dependencies:

`1`	`npm install`

Build the project:

`1`	`npm run build`

Create a global link (this makes the server executable from anywhere):

`1`	`npm link`

Configuration ⚙️

1. Configure Claude Desktop to recognize the Exa MCP server

You can find claude_desktop_config.json inside the settings of Claude Desktop app:

Open the Claude Desktop app and enable Developer Mode from the top-left menu bar.

Once enabled, open Settings (also from the top-left menu bar) and navigate to the Developer Option, where you’ll find the Edit Config button. Clicking it will open the claude_desktop_config.json file, allowing you to make the necessary edits.

OR (if you want to open claude_desktop_config.json from terminal)

For macOS:

Open your Claude Desktop configuration:

`1`	`code ~/Library/Application\ Support/Claude/claude_desktop_config.json`

For Windows:

Open your Claude Desktop configuration:

`1`	`code %APPDATA%\Claude\claude_desktop_config.json`

2. Add the Exa server configuration:

{
  "mcpServers": {
    "exa": {
      "command": "npx",
      "args": ["/path/to/exa-mcp-server/build/index.js"],
      "env": {
        "EXA_API_KEY": "your-api-key-here"
      }
    }
  }
}

Replace your-api-key-here with your actual Exa API key from dashboard.exa.ai/api-keys.

3. Available Tools & Tool Selection

The Exa MCP server includes the following tools:

web_search_exa: Performs real-time web searches with optimized results and content extraction.
research_paper_search: Specialized search focused on academic papers and research content.
twitter_search: Dedicated Twitter/X.com search that finds tweets, profiles, and conversations.
company_research: Comprehensive company research tool that crawls company websites to gather detailed information about businesses.
crawling: Extracts content from specific URLs, useful for reading articles, PDFs, or any web page when you have the exact URL.
competitor_finder: Identifies competitors of a company by searching for businesses offering similar products or services.
linkedin_search: Search LinkedIn for companies and people using Exa AI. Simply include company names, person names, or specific LinkedIn URLs in your query.

You can choose which tools to enable by adding the --tools parameter to your Claude Desktop configuration:

Specify which tools to enable:

{
  "mcpServers": {
    "exa": {
      "command": "npx",
      "args": [
        "/path/to/exa-mcp-server/build/index.js",
        "--tools=web_search_exa,research_paper_search,twitter_search,company_research,crawling,competitor_finder,linkedin_search"
      ],
      "env": {
        "EXA_API_KEY": "your-api-key-here"
      }
    }
  }
}

For enabling multiple tools, use a comma-separated list:

{
  "mcpServers": {
    "exa": {
      "command": "npx",
      "args": [
        "/path/to/exa-mcp-server/build/index.js",
        "--tools=web_search_exa,research_paper_search,twitter_search,company_research,crawling,competitor_finder,linkedin_search"
      ],
      "env": {
        "EXA_API_KEY": "your-api-key-here"
      }
    }
  }
}

If you don’t specify any tools, all tools enabled by default will be used.

4. Restart Claude Desktop

For the changes to take effect:

Completely quit Claude Desktop (not just close the window)
Start Claude Desktop again
Look for the 🔌 icon to verify the Exa server is connected

Using via NPX

If you prefer to run the server directly, you can use npx:

# Run with all tools enabled by default
npx exa-mcp-server

# Enable specific tools only
npx exa-mcp-server --tools=web_search_exa

# Enable multiple tools
npx exa-mcp-server --tools=web_search_exa,research_paper_search

# List all available tools
npx exa-mcp-server --list-tools

Usage 🎯

Once configured, you can ask Claude to perform web searches. Here are some example prompts:

`1`	`Can you search for recent developments in quantum computing?`

`1`	`Find and analyze recent research papers about climate change solutions.`

`1`	`Search Twitter for posts from @elonmusk about SpaceX.`

`1`	`Find tweets from @samaltman that were published in the last week about AI safety.`

`1`	`Research the company exa.ai and find information about their pricing and features.`

`1`	`Extract the content from this research paper: https://arxiv.org/pdf/1706.03762`

`1`	`Find competitors for a company that provides web search API services, excluding exa.ai from the results.`

`1`	`Find the LinkedIn profile for Anthropic company.`

`1`	`Search for data scientists at OpenAI on LinkedIn.`

The server will:

Process the search request
Query the Exa API with optimal settings (including live crawling)
Return formatted results to Claude
Cache the search for future reference

Testing with MCP Inspector 🔍

You can test the server directly using the MCP Inspector:

`1`	`npx @modelcontextprotocol/inspector node ./build/index.js`

This opens an interactive interface where you can explore the server’s capabilities, execute search queries, and view cached search results.

Troubleshooting 🔧

Common Issues

Server Not Found
- Verify the npm link is correctly set up
- Check Claude Desktop configuration syntax
- Ensure Node.js is properly installed
API Key Issues
- Confirm your EXA_API_KEY is valid
- Check the EXA_API_KEY is correctly set in the Claude Desktop config
- Verify no spaces or quotes around the API key

Connection Issues

Restart Claude Desktop completely
Check Claude Desktop logs:

# macOS
tail -n 20 -f ~/Library/Logs/Claude/mcp*.log

# Windows
type "%APPDATA%\Claude\logs\mcp*.log"

Acknowledgments 🙏

Exa AI for their powerful search API
Model Context Protocol for the MCP specification
Anthropic for Claude Desktop