Search API or SERP scraper? The question is whether your agent reads
A SERP proxy costs about a dollar per thousand queries. A search API costs five. The gap looks enormous until you notice the two products return different things — and that the cheap one stops being cheap the moment your agent has to open a page.
Both products answer the sentence "find me pages about X". They diverge completely on the sentence after it.
The pricing gap is real and large. Serper sells Google results at roughly a dollar per thousand queries, dropping toward thirty cents at volume. SerpApi runs $9 to $25 per thousand depending on tier. Search APIs built for agents — Exa, Tavily, Brave, our own — sit between $5 and $8. If you are looking at those numbers alone, the SERP proxy wins and it is not close.
The numbers are not alone.
What each one actually hands back
A SERP proxy is a parser sitting in front of somebody else's search engine. You send a query, it runs that query on Google or Bing, scrapes the result page, and returns it as structured JSON. What comes back is the same thing a person sees: a title, a URL, and a snippet of forty to sixty words that the engine picked to look good on a results page.
A search API queries an index. Because it holds the index, it can hand back more than the pointer — the extracted body text, an authority signal, a freshness date, a flag for whether the page is likely paywalled.
The distinction matters because of what an agent does next. A person reads the snippet and decides whether to click. An agent cannot cite a snippet: forty words are not enough to answer with, so it has to open the page. That fetch is the step the price comparison quietly omits.
The second bill
Fetching pages yourself is not free, and it is not one line of code:
- You are now running a crawler. Retries, timeouts, redirect chains, content-type sniffing, per-host politeness.
- You are now doing extraction. Boilerplate stripping, main-content detection, PDF handling. This is a solved problem, but solved by a library you now have to keep working.
- You are now handling refusals. Bot walls, consent interstitials, paywalls. A meaningful share of any result set will not open for a datacentre IP.
- Or you buy that too. An extraction API adds its own per-page charge, which is where a $0.002 query becomes a $0.012 one.
None of this argues that SERP proxies are bad. It argues that the two prices are not comparable, because one of them is for a smaller product.
Three things that bite later
Availability is inherited. A SERP proxy's uptime is a function of how the upstream engine feels about being proxied this week. When a detection rule tightens, error rates move on your side without anything changing in your code. An index-backed API's availability is a function of its own capacity, which is at least a thing its operator controls.
Latency has a floor. A proxy cannot be faster than the engine it proxies plus a round trip. Our web lane answers from our own index in about 80 ms at p50 because there is no second hop. If search sits inside a loop the agent runs fifteen times, that difference is a second of wall clock per task.
The terms are somebody else's. Automated retrieval of a search engine's result pages generally sits outside that engine's terms of service. Whether that matters to you is a question for your own counsel and your own risk appetite — but it is a dependency you are taking on, and it is worth taking on deliberately rather than discovering it during diligence.
How to choose, in one line each
| If you need… | Use |
|---|---|
| Ranked links at very high volume, no page content | A SERP proxy. Genuinely cheaper. |
| Rank tracking, SEO monitoring, SERP-feature analysis | A SERP proxy — you want the result page itself. |
| Grounding for a model, with citations it can quote | A search API. |
| An agent that reads before it answers | A search API with extraction on the same call. |
| Predictable latency inside an agent loop | A search API with its own index. |
For the third and fourth rows, that is what our deep lane is: one call, page text included, $0.005. The full price comparison against every provider in this market is in what a web search API should cost in 2026.
And if the agent doing the reading is Claude or ChatGPT, you can skip the integration entirely — connect it over MCP instead.
Frequently asked questions
What is the difference between a search API and a SERP API?
A SERP API fetches and parses a search engine's results page on demand, returning titles, URLs and snippets. A search API queries an index directly and can return the retrievable content behind those URLs. SERP APIs are cheaper per call; search APIs remove the second step of fetching and extracting the pages.
Is a SERP scraper cheaper than a search API?
Per call, yes — roughly $1.00 per 1,000 against $5.00. But a SERP call returns a snippet, so any agent that needs the page content has to buy extraction separately. Once you add a scraping or extraction API to the bill, the two converge and often cross over.
Can I use a SERP API for retrieval-augmented generation?
You can, provided you add your own fetching, extraction and error handling for blocked or paywalled pages. Snippets alone are usually too thin to ground a generation — they are a few dozen words chosen to be read by a human scanning a results page, not to be read by a model.
