Best web scraping APIs

The best web scraping API, ranked by live price.

These catalog providers take a URL and return the page as markdown. Ranked by price per page, read live from the catalog.

Give this to your agent

set up looot for Best web scraping API - https://looot.ai/best/web-scraping-api/skill.md

Jina AI
Olostep
ScrapeGraphAI
Context.dev
Serper
Steel API
Keenable

The short answer

The best web scraping API for an agent returns the page as markdown, because a model reads text, not HTML. For plain pages the cheapest call per page wins, and this page ranks the looot catalog providers by that live price. For crawls, batches or browser features, pick the provider that has those endpoints.

Cheapest today

Jina AI

$0.0002

Per page for a web page as markdown from a url, read live from the catalog.

$0.20 per 1,000 pages

The ranking

7 providers, one job

A web page as markdown from a URL. Bars show price per page, to scale.

Sort by

01
Jina AI
Returns the main content as markdown with the title, metadata and links. It is one reader endpoint, with no crawl or batch endpoint in the catalog.
- 5 endpoints
- Web search
$0.0002 / page$0.20 per 1,000 pages
02
Olostep
Scrapes one page to markdown and also runs batches, crawls and sitemap reads. Batches and crawls are separate jobs you poll for results.
- 21 endpoints
- Crawl
- Batch
- Structured
- Sitemap
$0.0018 / page$1.80 per 1,000 pages
03
ScrapeGraphAI
Scrapes a page to markdown and has a stealth variant for pages that block plain requests. The stealth variant is its own endpoint with its own price, so check it before you switch.
- 11 endpoints
- Crawl
- Structured
- Screenshot
$0.0020 / page$2.00 per 1,000 pages
04
Serper
A Google search API with one page scrape endpoint on the side. It has no crawl or batch endpoint, so it suits reading the pages a search returned.
- 13 endpoints
- Google results
$0.0020 / page$2.00 per 1,000 pages
05
Context.dev
Can keep only the main content, keep or drop links and images, wait for the page and fetch it from a chosen country. It also has an HTML scrape and a crawl.
- 28 endpoints
- Raw HTML
- Crawl
- Screenshot
$0.0025 / page$2.50 per 1,000 pages
06
Keenable
Fetches a page as markdown, next to a web search endpoint. Those two endpoints are all it has in the catalog.
- 2 endpoints
- Web search
$0.0040 / page$4.00 per 1,000 pages
07
Steel API
Loads the page in a hosted browser and returns markdown or HTML, and can add a screenshot or a PDF. The catalog lists no crawl endpoint for it.
- 3 endpoints
- Screenshot
- PDF
$0.0050 / page$5.00 per 1,000 pages

Choosing one

Pick by what you need

If you needBatches, crawls and sitemaps from one providerPick Olostep
If you needPages that block plain requestsPick ScrapeGraphAI
If you needMain content only, or a page fetched from one countryPick Context.dev
If you needRead the pages a Google search returnedPick Serper
If you needA browser render with a screenshot or a PDFPick Steel API

How we rank

What the ranking is built on

Every provider here has an endpoint in the looot catalog that takes a URL and returns the page as markdown.
We rank by the price per page of that endpoint, read live from the catalog. A cheaper page ranks higher.
Endpoints billed per result instead of per page are left out, because the two units do not compare.
The catalog does not publish a measured success rate for these endpoints, so this page does not rank by one and quotes none.
Extras are other web endpoints the provider has, such as crawl or batch. More extras does not mean better pages.
Providers outside the looot catalog are not ranked.

Web scraper API cost per 1,000 pages

Each row in the ranking shows the live price per page and the price per 1,000 pages next to it. Both come from the same catalog read, so they never disagree.

A page that comes back empty still costs a call on most providers. If a site blocks plain requests, the cheapest row can cost more per good page than a pricier one, so run ten of your own URLs before a big job.

URL to markdown API, with each endpoint's options

Scraping one page or crawling a site

A scrape reads one URL. A crawl starts at one URL and follows links, so it returns many pages and runs as a job you poll. The ranking prices the single page scrape.

For a whole docs site or a blog archive, pick a provider with a crawl tag in the ranking. For a list of URLs you already have, a batch endpoint or one scrape per URL both work.

The prompt

Let your agent pick the cheapest.

Paste the prompt. The agent finds the page to markdown endpoints, checks each price and runs the cheapest one that fits.

One run is one page.
Runs on demand, whenever you ask.
Checking on a timer needs scheduled runs, not available yet.

Prompt: Best web scraping API

Use looot to read these 10 URLs as markdown: call the cheapest page to markdown endpoint for each, save each page as a markdown file named after its domain, and tell me the total cost. 1. Set up https://looot.ai/skill.md. 2. Search the catalog for each step and check its price before you call. 3. Run the cheapest endpoint that fits. 4. Show the results as a table and the total cost.

Give this to your agent

Setup command

set up looot for Best web scraping API - https://looot.ai/best/web-scraping-api/skill.md

FAQ

Questions people ask

It depends on the pages. This page ranks the looot catalog providers by live price per page for turning a URL into markdown. Pick the cheapest for plain pages, or the provider with crawl, batch or browser endpoints when you need them.

The first row of the ranking above, read live from the catalog. Prices can change, so the page reads them on each refresh and shows no number when the catalog is unreachable.

This page ranks priced endpoints. Free tiers, where a vendor has one, sit on the vendor's own plan page and are outside looot.

Firecrawl is in the looot catalog, with scrape, crawl and structured extraction endpoints. The catalog files its scrape under structured extraction, so this list, which prices one capability, does not rank it. Every provider above returns a page as markdown, and your agent calls any of them with one key.

A service that takes a URL, loads the page and returns its content as data. For an agent the useful form is markdown, the text of the page without the HTML around it.

A model reads text. HTML carries scripts, styles and navigation that cost tokens and say nothing. Markdown keeps the headings, lists, links and tables and drops the rest.

Some providers load the page in a browser first. Steel does, and the URL to markdown use case lists the options of each endpoint. Run one of your own pages before you commit to a provider.

Use a crawl endpoint. Olostep, ScrapeGraphAI and Context.dev have one, shown as a crawl tag in the ranking. Crawling on a timer needs scheduled runs, not available yet, so run it by hand when you want a fresh copy.

It depends on the site, the data and where you are. Read the site's terms and robots rules, and do not collect personal data you have no basis to hold. This page is not legal advice.

Paste the prompt on this page into your agent. It sets up looot, finds the page to markdown endpoints, checks the price and runs the cheapest one that fits.

The catalog does not publish a measured success rate for these endpoints. We would rather rank by a number we can read than quote one we cannot.

No. looot bills per call, and nothing runs while your agent is idle.

Keep going

Use case
URL to markdown API
Send a page URL. Read markdown text and page title back. Your agent picks the provider, and you pay per call.
Use case
Web search API
Send a search query. Read ranked links and titles back. Your agent picks the provider, and you pay per call.
Recipe
Docs scraped to markdown
Scrapes a docs site and saves every page as clean markdown, with one folder per section.
Recipe
Pricing page monitor
Reads competitor pricing pages and reports when a plan name, price or limit changes. Your agent needs another tool for alerts.
Recipe
Landing page teardown
Reads a landing page and lists its tech stack, headline, offer, proof and call to action, then suggests what to test.
Best list
Best SERP API
The best SERP API for your agent, ranked by live price per call and per 1,000 searches from the looot catalog, with what each provider returns.
Best list
Best news API
The best news API for your agent: providers that return news articles for a keyword, ranked by live price per call from the looot catalog.
Best list
Best LinkedIn API
The best LinkedIn API for your agent: providers that return a public profile from a profile URL, ranked by live price per call from the looot catalog.
Best list
Best TikTok API
The best TikTok API for your agent: providers that return a creator's public profile, ranked by live price per call from the looot catalog.
Best list
Best Reddit API
The best Reddit API for your agent: providers that return the posts of a subreddit, ranked by live price per call from the looot catalog.
Best list
Best B2B data API
The best B2B data API for your agent, ranked by live price per call for company enrichment from the looot catalog, with what each provider returns beyond it.
Best list
Best lead finder API
The best lead finder API for your agent: people search providers that return leads by title, company and location, ranked by live price per lead.