API reference

Everything here is implemented. Parameters the gateway does not read are not listed, and pnpm docs:check fails the build if this page and the code disagree.

There is also a machine-readable description at /openapi.json. It is generated from the same outcome taxonomy the router uses, so its status codes and enums are the real ones. Point a client generator at it, or open it in any OpenAPI viewer.

Endpoints

PUT and DELETE on /v1 return 404. They are not treated as GET.

Parameters

url

Rejected at the edge if it resolves to a private range, a denylisted host, or a cloud metadata address. That returns TARGET_FORBIDDEN.

render

Only true and 1 enable rendering. Every other value, including leaving the parameter out, means false.

This is deliberate. Presence alone never counts as true. Otherwise render=false would render the page and cost about five times as much.

provider

A benchmarking escape hatch. It pins one provider, so there is no failover. If that provider cannot serve the request, you get NO_PROVIDER_AVAILABLE rather than a silent substitution.

timeout

The deadline for the whole request, in milliseconds, including every failover hop. Default is the server's PROXLANE_DEADLINE_MS.

You can ask for less time than the server budgeted. You cannot ask for more: the ceiling is what bounds how long one request holds a slot, and the gateway's memory sizing depends on it.

The floor is 8000. Below that a single attempt cannot finish, so the request would time out without having tried anything — that is a 400, not a 504.

When the deadline runs out the outcome is BUDGET_EXCEEDED.

POST requests

The body is forwarded as text, byte for byte. Proxlane does not parse it. Guessing between JSON and form encoding would corrupt one of them.

curl -X POST -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  --data '{"q":"example"}' \
  "https://your-gateway/v1?url=https://example.com/search"

res = requests.post(
    "https://your-gateway/v1",
    params={"url": "https://example.com/search"},
    headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
    data='{"q":"example"}',
)

const res = await fetch(
  `https://your-gateway/v1?url=${encodeURIComponent("https://example.com/search")}`,
  {
    method: "POST",
    headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
    body: JSON.stringify({ q: "example" }),
  },
);

Request bodies use the same size cap as responses. Over it, you get RESPONSE_TOO_LARGE.

Response headers

X-Ignored-Params is worth wiring into your logs. We don't reject a parameter we don't recognise — ScraperAPI accepts a dozen we don't implement, and rejecting them would break the hostname change this whole thing is built on. So the request runs, and the header names what went nowhere.

The one that catches people is render. js_render=true is ScrapingBee's spelling and js=true is Scrapfly's; ours is render=true. Send the wrong one and you get HTTP 200, a real page, and no JavaScript — at about a fifth of the cost, which is the tell:

X-Ignored-Params: js_render
X-Cost-Estimate: 1.000000     # rendered, this would be ~5

Two of those are easy to misread.

A request rejected before a provider was chosen — no url, a bad premium, a wrong key — carries X-Outcome-Class, X-Attempts: 0 and X-Cost-Estimate: 0.000000. It has no X-Outcome, because the taxonomy describes what happened to a scrape and that request never became one. This is the reason to branch on the class.

X-Cost-Source tells you whose number it is. Three of the four providers report what they charged on the response itself, and the adapter passes that straight through. estimated means they said nothing and we applied our own table for that provider — a figure worth treating with more suspicion than a reported one, because it is our model of their pricing rather than their answer. mixed means a chain used both, which a failover across providers routinely does.

X-Cost-Estimate covers every attempt, not just the one that worked. A failover that burned two charged hops reports both.

X-Chain is the one that says who failed. X-Provider-Used names the winner, so a request that failed over and then succeeded returns a clean 200 that hides the provider which cost you 22 seconds:

X-Chain: scraperapi:PROVIDER_TIMEOUT>scrapfly:OK
X-Provider-Used: scrapfly
X-Attempts: 2

A single-attempt request has a one-element chain, which matters: an absent header could not otherwise say "nothing failed". It is omitted, never empty, when no provider was tried at all — a bad URL, a refused target, every provider cooling. X-Attempts: 0 says that already.

Retry-After is never guessed. If it is absent, Proxlane does not know when to retry. A number you can trust is worth more than a number that is always present.

Branching on results

Branch on X-Outcome-Class, not X-Outcome.

X-Outcome is open and gains members as adapters land. X-Outcome-Class has six values and does not grow. Code written against the class keeps working when the vocabulary expands.

switch (res.headers.get("x-outcome-class")) {
  case "ok":       return res;           // you have the page
  case "blocked":  return retryLater();  // every provider was blocked
  case "target":   return giveUp();      // the site itself said no
  case "client":   throw new Error("fix the request");
  case "provider": // fallthrough: transient, safe to retry
  case "gateway":  return retryLater();
}

outcome_class = res.headers["X-Outcome-Class"]

if outcome_class == "ok":
    return res.text                      # you have the page
if outcome_class == "target":
    return None                          # the site itself said no
if outcome_class == "client":
    raise ValueError("fix the request")
return retry_later()                     # blocked, provider or gateway

Errors

When there is no page to return, the body is JSON instead of an empty success.

{
  "requestId": "01JD8F2K9WQ3",
  "error": {
    "code": "NO_PROVIDER_AVAILABLE",
    "class": "gateway",
    "message": "No adapter matches the capability, or the chain is exhausted",
    "docs": "https://github.com/proxlane/proxlane#outcomes"
  },
  "attempts": [
    { "provider": "scraperapi", "outcome": "SOFT_BLOCK", "detectRuleId": "cf-challenge" }
  ]
}

error.code is the outcome. One vocabulary covers failures at a provider and failures before one was reached.

UNAUTHORIZED is the exception. It is not an outcome, because outcomes describe what happened to a scrape, and a request rejected at the door never became one.

attempts lists what was tried and what each provider said. That is the grain you need when debugging a failover.

Backpressure

Past its concurrency ceiling the gateway returns 429 GATEWAY_BUSY with Retry-After.

It sheds rather than queues. A queued scrape holds your socket open while its own deadline runs down, so you get a timeout with no explanation instead of a 429 you can act on.

The class is gateway, not provider. No provider throttled you. /health is never shed, so a busy gateway does not fail its own health check.