How to combine local language models with web search capabilities while keeping your data private
Setting up AI tools for business use often means choosing between convenience and privacy. Cloud-based solutions are easy to use but send your data elsewhere. Local solutions keep everything on your machine but can be tricky to set up and extend with additional capabilities.
Today, we’ll walk through building a practical AI setup that gives you both: a local language model that can search the web when needed, all while keeping your conversations and processing on your own hardware. This approach aligns perfectly with our philosophy of self-hosted solutions—similar to how we’ve shown you can run your own n8n automation platform for complete workflow control.
The Tools We’re Using
Let’s start by understanding what each piece does:
Ollama: Your Local AI Engine
Ollama is software that runs large language models on your own computer. Think of it as a way to have ChatGPT-like capabilities without sending anything to OpenAI or other cloud providers. It handles downloading models, managing memory, and serving them through a simple API.
The key advantage? Everything stays local. Your questions, the AI’s responses, and any processing happen entirely on your machine. This matters for sensitive business discussions, proprietary information, or simply when you want to avoid monthly AI subscription fees.
Model Context Protocol (MCP): Adding Capabilities
Language models are great at conversation and reasoning, but they can’t browse the web, read files, or interact with other systems by themselves. Model Context Protocol (MCP) solves this by providing a standardized way to give AI models access to external tools.
Think of MCP as a universal translator between your AI model and other software. Want your AI to search the web? There’s an MCP server for that. Need it to read databases? Another MCP server. The beauty is that once you set up the protocol, adding new capabilities becomes much simpler.
mcphost: The Reliable Connection
Here’s where it gets practical. While MCP is the standard, you need software to actually connect your local AI model to MCP servers. mcphost does exactly this, and it does it well.
We tried other options first (like ollmcp), but ran into constant connection timeouts and configuration headaches. mcphost uses explicit configuration files instead of trying to auto-discover everything, which means it actually works reliably in real-world scenarios.
DuckDuckGo MCP Server: Private Web Search
For web search capabilities, we’re using a DuckDuckGo MCP server. This lets your local AI search the web through DuckDuckGo’s API, which doesn’t track users or store search histories. It’s the perfect complement to a privacy-focused local setup.
uv: Python Environment Management
uv is a fast Python package manager that handles creating isolated environments and managing dependencies. We use it to run the DuckDuckGo MCP server cleanly without interfering with other Python projects on your system.
How It All Works Together
Here’s the flow when you ask your local AI to search for something:
- You type a question like “Search for recent developments in e-commerce regulations”
- mcphost receives your message and forwards it to your local Ollama model
- The model recognizes it needs to search and calls the DuckDuckGo MCP server through the MCP protocol
- The MCP server searches DuckDuckGo and returns results
- Your local model processes the search results and gives you a comprehensive answer
- Everything stays on your machine except the actual web search query
Setting This Up: The Practical Steps
Getting the Foundation Ready
First, you’ll need Ollama running with a model that supports tool calling. We recommend qwen3:30b-a3b for a good balance of capability and speed:
# Install and start Ollama
ollama pull qwen3:30b-a3b
ollama serve
Installing mcphost
mcphost is written in Go, so you’ll need that first:
# On macOS
brew install go
# On Ubuntu/Debian
sudo apt install golang-go
# Install mcphost
go install github.com/mark3labs/mcphost@latest
# Make sure it's in your PATH
export PATH=$PATH:$(go env GOPATH)/bin
Setting Up DuckDuckGo Search
Clone and set up the DuckDuckGo MCP server:
git clone https://github.com/example/duckduckgo-mcp-server
cd duckduckgo-mcp-server
pip install uv # if you don't have it
uv sync
Configuration That Actually Works
Create a configuration file at ~/.mcphost.json:
{
"mcpServers": {
"duckduckgo-search": {
"command": "uv",
"args": [
"run",
"--directory",
"/path/to/your/duckduckgo-mcp-server",
"python",
"src/duckduckgo_mcp_server/server.py"
]
}
}
}
Replace /path/to/your/duckduckgo-mcp-server with the actual path where you cloned the repository.
Using Your New AI Assistant
Once everything’s running, you can interact with your AI in several ways:
Interactive Chat
# Start a conversation
mcphost -m ollama:qwen3:30b-a3b
This gives you a chat interface where you can have ongoing conversations and ask for web searches when needed.
One-Off Questions
# Get a quick answer
mcphost -m ollama:qwen3:30b-a3b -p "Search for information about EU data privacy laws"
Automated Scripts
# Save results to a file
mcphost -m ollama:qwen3:30b-a3b -p "Search for competitors in the bowling equipment market" --quiet > research.txt
What You Can Actually Do With This
Research and Fact-Checking
Ask your AI to search for current information on topics relevant to your business. Since it’s using DuckDuckGo, you get decent web results without the tracking.
Content Research
“Search for recent trends in sustainable packaging” gives you current information that the model can then analyze and summarize based on its training.
Competitive Intelligence
“Search for news about [competitor name] and summarize their recent activities” provides up-to-date information while keeping your research interests private.
Technical Problem Solving
“Search for solutions to API rate limiting issues” can help you find current best practices and code examples.
The n8n Connection: Workflow Automation Meets AI
Here’s where things get really interesting. Just as we’ve shown you how to self-host n8n for workflow automation, this local AI setup can be integrated into automated workflows.
Imagine triggering your local AI through n8n workflows:
- Automated Research: Daily workflows that search for industry news and compile summaries
- Content Generation: Triggered content creation based on calendar events or form submissions
- Data Analysis: Automated processing of uploaded documents with AI insights
- Customer Support: Local AI processing of support tickets before human review
The combination of self-hosted n8n and local AI creates a powerful automation stack where your data never leaves your infrastructure. You get the convenience of automation with complete privacy control.
Future Integration Possibilities
We’re exploring several ways to connect these systems:
- HTTP endpoints from your local AI that n8n can call
- File-based integration where n8n triggers AI processing on documents
- Database workflows where AI results feed into automated business processes
- Email automation where your local AI helps draft and personalize responses
Just like our WordPress automation tools streamline content management, combining local AI with n8n workflows can automate complex business processes while maintaining data sovereignty.
The Real Benefits
Privacy That Actually Matters
Your business conversations, internal discussions, and strategic thinking stay on your hardware. Only your web search queries go out (through privacy-focused DuckDuckGo), and those don’t include context about why you’re searching.
No Monthly Bills
After the initial setup, there are no ongoing AI subscription costs. You’re using your own hardware to run everything—similar to how self-hosting n8n saves you from expensive SaaS workflow tools.
Reliable Performance
With proper configuration, this setup is remarkably stable. No more “the AI service is down” or “we’ve hit our monthly limit” problems.
Customizable and Extensible
Want to add database search? File reading? Custom business logic? The MCP ecosystem makes adding new capabilities straightforward, and eventual integration with your n8n workflows opens up endless automation possibilities.
Common Issues and Solutions
“Command Not Found” Errors
Make sure your Go binary directory is in your PATH:
echo 'export PATH=$PATH:$(go env GOPATH)/bin' >> ~/.zshrc
source ~/.zshrc
Connection Timeouts
Usually means the MCP server isn’t starting properly. Test it manually:
cd /path/to/duckduckgo-mcp-server
uv run python src/duckduckgo_mcp_server/server.py
Model Not Responding to Tool Calls
Make sure you’re using a model that supports function calling. Not all Ollama models have this capability built-in.
Slow Performance
Consider using a smaller model (like qwen3:7b) if your hardware is limited, or tune the temperature settings for faster responses:
mcphost -m ollama:qwen3:7b --temperature 0.3 -p "your question"
Infrastructure Considerations
If you’re already running self-hosted services like n8n, you might want to consider hosting this on the same infrastructure. The setup we described for n8n on Hetzner Cloud can easily accommodate Ollama as well—you’ll just need more RAM and CPU for the language models.
A typical setup might include:
- Hetzner CCX42 or similar for the resources needed by larger models
- Docker containerization for easy management alongside n8n
- Traefik reverse proxy for secure access (if you want remote access)
- Shared storage for models and workflow data
Is This Worth the Setup Time?
If you regularly use AI for business tasks and care about privacy, absolutely. The initial setup takes a few hours, but you end up with a capable AI assistant that:
- Works offline (except for web searches)
- Doesn’t send your data to third parties
- Costs nothing to operate after setup
- Can be extended with additional capabilities as needed
- Integrates with your existing self-hosted infrastructure
For teams handling sensitive information or operating in regulated industries, this setup provides AI capabilities without the compliance headaches of cloud services.
The combination of local processing with selective web access strikes a practical balance between capability and privacy that’s hard to achieve with either pure cloud or pure local solutions. And when you add workflow automation into the mix, you’re building a foundation for truly sophisticated business automation that stays under your control.
What’s Next?
We’re actively working on deeper integrations between local AI setups and workflow automation platforms. Keep an eye out for future posts covering:
- Docker configurations for combined n8n + Ollama setups
- Custom MCP servers for business-specific data sources
- Workflow templates that leverage local AI for common business processes
- Security hardening for production deployments
The future of business automation isn’t just about connecting existing services—it’s about having intelligent systems that can reason, research, and act on your behalf while keeping everything under your control.
If you run into issues during setup or want to extend this configuration for your specific use case, we’re happy to help troubleshoot through our standard support channels.