LiteLLM for data APIs: fallback for your backend
LiteLLM for data APIs: how provider fallback works for lookups like email finding, with REST, Python and Node code and a cost cap per call.
Luisa · · 6 min read
On this page
LiteLLM lets your backend call many language models through one interface, and it moves a failed call to another model. A LiteLLM for data APIs would do the same for lookups, such as finding a work email or enriching a company. That is harder than it sounds, because data APIs have much less in common than model APIs do. They take different inputs, return different shapes and use HTTP status codes in different ways, so your code cannot even assume that a failure will look like a failure.
This guide is for the developer who writes that code. It walks through the wrapper you would build yourself, then the same call through looot in REST, Python and Node, and it ends with the cases where you should skip a gateway. If you are still choosing between products, Is there an OpenRouter for data APIs? compares what each kind of router does. For the concepts, such as what triggers a fallback and how the order is picked, read how data provider fallback works.
Why is a LiteLLM for data APIs harder to build?
LiteLLM calls more than 100 models in the OpenAI format (LiteLLM docs, read 2026-11-16), and if a call still fails after its retries, it falls back to another model group (LiteLLM fallbacks, read 2026-11-16). That works because model providers share one request shape and a failed call comes back as an error. Data providers share neither.
Take email finders. Their own descriptions in the looot public catalog show the spread. QuickEnrich answers a not-found with an empty `data` list and a reason field. Limadata answers it with HTTP 404. AI Ark answers 404 on its older version and 200 with `data: null` on the newer one. Icypeas puts `NOT_FOUND` in a status field. Some of them bill a search that finds nothing and some do not.
So retry code written for model errors would stop on a miss that deserved another try, and it would keep going on a bad input that every provider will reject. Here is how the two problems compare.
| Concern | Model router | Data API fallback |
|---|---|---|
| What moves to the next provider | An error or a timeout | An error, a timeout or an empty answer |
| What the HTTP status tells you | Usually whether it worked | Often not enough, so you read the body |
| Shape across providers | Similar | Different inputs and fields |
| What a retry risks | A second token bill | A second charge from each provider |
| What can be retried | Any call | Lookups only, never a write |
What does the DIY wrapper look like?
The loop itself takes under 20 lines, and it is the easy part. This is a sketch: the provider objects and their methods are the part you would write.
import requests
def find_email(person, providers, max_cost):
spent = 0.0
for p in providers:
if spent + p.price > max_cost:
continue
try:
answer = requests.post(p.url, json=p.map_input(person),
headers=p.headers, timeout=10)
except requests.RequestException:
continue
if p.bills(answer):
spent += p.price
if answer.ok and p.has_email(answer.json()):
return p.read_email(answer.json())
return NoneThe real cost sits in `map_input`, `bills` and `has_email`, because you need one of each per provider and they break when a provider changes its answer. You also keep an account, a key and a balance with every provider. If your process dies between a paid call and saving its answer, this code pays again when it restarts. That is still a fair trade if you use two providers that rarely change, and in that case you should write it yourself.
How do you get LiteLLM-style fallback for data APIs in one request?
With looot you name the job instead of a provider, turn on `fallback` and set a cost cap. looot then walks the other providers of that job until one finds an answer or the cap is reached. Here is the same call in three languages. The Node version needs Node 18 or later and an `.mjs` file, or `"type": "module"` in `package.json`, because it uses top-level `await`.
curl -s -X POST "https://api.looot.ai/v1/runs?wait=20" \
-H "Authorization: Bearer $LOOOT_TOKEN" \
-H "content-type: application/json" \
-d '{"endpointId":"job:people.email.find",
"input":{"first_name":"Jane","last_name":"Doe","domain":"example.com"},
"idempotencyKey":"lead-4821-email",
"fallback":{"maxAttempts":3,"maxCostUsd":0.10}}'import os, requests
run = requests.post(
"https://api.looot.ai/v1/runs", params={"wait": 20}, timeout=30,
headers={"Authorization": f"Bearer {os.environ['LOOOT_TOKEN']}"},
json={
"endpointId": "job:people.email.find",
"input": {"first_name": "Jane", "last_name": "Doe", "domain": "example.com"},
"idempotencyKey": "lead-4821-email",
"fallback": {"maxAttempts": 3, "maxCostUsd": 0.10},
},
).json()
if run["outcome"] == "hit":
save(run["result"], cost=run["actualCost"])
elif run["outcome"] == "miss":
mark_not_found()const res = await fetch("https://api.looot.ai/v1/runs?wait=20", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.LOOOT_TOKEN}`,
"content-type": "application/json",
},
body: JSON.stringify({
endpointId: "job:people.email.find",
input: { first_name: "Jane", last_name: "Doe", domain: "example.com" },
idempotencyKey: "lead-4821-email",
fallback: { maxAttempts: 3, maxCostUsd: 0.1 },
}),
});
const run = await res.json();`save` and `mark_not_found` stand for your own code.
Your code should read `outcome` and not the HTTP status, because looot has already judged the answer from the data that came back. The placeholder person in the snippets shows why. The request succeeds with HTTP 201, but no provider has an email for Jane Doe at example.com, so the run comes back with `outcome: "miss"` and an `actualCost` of zero. The `route` block lists each attempt. Here is a trimmed example response from a lookup of a real person, where the first provider found nothing and did not bill, and the second found the email and did:
{"outcome":"hit",
"route":{"servedBy":"tomba-email-finder","outcome":"hit",
"attempts":[
{"n":1,"endpointId":"filtrat-finder-search","provider":"filtrat","outcome":"miss","status":200,"billed":false},
{"n":2,"endpointId":"tomba-email-finder","provider":"tomba","outcome":"hit","billed":true}]}}This gives you one way to read the outcome, but it does not make every provider return the same fields. The result still holds what that provider sent, and only some runs also carry a `normalized` block, so check the fields you need on each endpoint.
`maxCostUsd` caps the whole route, whatever each provider does. A miss is free unless its provider bills for it, and the fallback guide linked above explains who pays when it does.
How do you retry without paying twice?
Send an `idempotencyKey` built from your own record, one per lookup. The same key with the same input returns the first run instead of starting a second one, and the same key with a different input is refused. How to build a key that holds up, and what to do with each kind of answer, is in idempotency keys for paid APIs.
When should you not use a gateway?
Call the provider directly if you use only one, your volume is high and you already have a contract with it, since a gateway adds little there. The same goes for a field that only one provider returns, where you can pin that endpoint and skip the walk. Fallback also never applies to a call that changes data, because a write runs once on the endpoint you named. If your data rules name specific processors, check who would handle the input before you send it.
For the price of each email finder on one page, see the email finder API page. For the model side of your stack, read the LLM router post.
Try it: read a route
Prompt for your agent
Run one lookup with fallback
Questions
Keep reading
Is there an OpenRouter for data APIs?
OpenRouter routes language models. Search, enrichment and scraping calls need a router for data APIs: what exists, how the two compare and how fallback works.
The looot team · · 6 min read
Data provider fallback: automatic fallback for data APIs
How data provider fallback works when a data API returns nothing or fails: the order, the cost cap, what stops a route and who pays for a miss.
The looot team · · 6 min read
LLM router: what it routes and what it leaves out
An LLM router picks a model for each request. It does not route the data and tool calls an agent makes. Where each kind of routing fits, and how fallback works.
Walid Boulanouar · · 7 min read
Try it with the agent you already use
Start your workspace, top up, and paste one prompt into your agent.