Open-source retrieval infrastructure

Web search for every LLM.

One self-hostable search API for agents and RAG, with no-key sources, citations, and partial-failure resilience.

Structured evidence, ready for an agent

{
  "query": "open source rerankers",
  "category": "academic",
  "results": [{
    "title": "Example work",
    "url": "https://doi.org/...",
    "source": "crossref",
    "score": 0.016393
  }],
  "providers": [{
    "provider": "crossref",
    "ok": true
  }]
}

SearXNG Wikipedia GitHub Crossref Hacker News

Search by intent, not vendor.

SearchForge routes each query to relevant sources, normalizes the evidence, removes duplicate URLs, and preserves every contributing citation.

Web

Run private metasearch through SearXNG with Wikipedia as a no-key fallback.

category: "web"

Code

Find repositories and implementation references through GitHub search.

category: "code"

Academic

Retrieve papers, DOI metadata, authors, and publication details from Crossref.

category: "academic"

Community

Search technical discussions and launch posts through Hacker News.

category: "community"

Read URL

Turn a public page into bounded, LLM-ready Markdown with SSRF protections.

read_url("https://example.com")

One retrieval core. Four ways in.

MCP

Expose web_search, read_url, and search_status to compatible agents.

REST

Call a versioned HTTP contract from any language or RAG service.

CLI

Search, read, and diagnose providers directly from a terminal.

TypeScript

Embed providers and the routing core inside an existing application.

Running in four commands.

Node.js 20 or newer. No account is required for the default adapters.

git clone https://github.com/divyanshu-iitian/SearchForge.git
cd SearchForge
npm install && npm run build
node dist/cli.js search "agent frameworks" --category code

Built for evidence, not generated answers.

SearchForge stays below the reasoning layer. Your model receives structured sources, provenance, health data, and predictable failure behavior.

Read the architecture

No telemetry: Queries stay with the providers you configure.
Partial results: One provider outage cannot erase healthy evidence.
Stable contracts: Versioned response shapes across REST, MCP, and TypeScript.
Self-hosted path: A private SearXNG stack is included for broad web metasearch.

Open web retrieval for your next agent.

MIT licensed. No telemetry. No mandatory paid search provider.

View on GitHub