unlob Docs
Console
Browse documentation

Logging and monitoring calls to unlob

Six fields turn a request log into an early warning system, before a user reports anything.

Nothing here requires a dashboard product or a new dependency — every field is already in the response you get back, and most of what is worth alerting on is a threshold on a number your client already receives. The gap on most integrations is not missing data; it is that the data is read once per call and never logged anywhere a person or an alert can see it later.

Per-request: six fields worth a log line

FieldWhereWhy it is worth logging
x-ratelimit-remainingResponse header, every callFalling toward zero is your early warning before the first 429
partialResponse bodyA true here means the answer was incomplete, not that the topic is thin
independent_sourcesEach hitA near-zero average across a period suggests queries returning single-source results more than usual
HTTP statusEvery call429 with no retry-after, 402, and the two 503s are worth their own counters, distinct from a generic error rate
retry-afterOn a 429 or a 503The real wait in seconds. Log what you were told and whether you honoured it — a client that substitutes its own sleep is the usual cause of a retry storm
Caller-observed latencyTimed around the callRound-trip time from your own process, not a figure unlob publishes — see the note below

None of these need the response body parsed beyond what your client already does to use the result. The only change most integrations need is writing three or four of these values to a structured log line alongside the query, rather than discarding them once the call succeeds.

log.info("unlob.search", extra={
    "query": query,
    "status": response.status_code,
    "ratelimit_remaining": response.headers.get("x-ratelimit-remaining"),
    "partial": body.get("partial", False),
    "avg_independent_sources": mean(h.get("independent_sources", 0) for h in body["results"]) if body["results"] else None,
    "latency_ms": elapsed_ms,
})

The account-level picture

GET /account is free — it costs no credits — and reports credits_used_this_period against monthly_credits alongside rate_per_min and metered_billing. Polling it on a schedule — hourly is more than enough — and logging the result gives you the monthly trend line that per-request logging cannot: x-ratelimit-remaining tells you about the current minute, credits_used_this_period tells you where you are in the month. Every call also says what it was billed in x-credits-charged, so summing that header per workload shows which calls spend the month. See Pricing and plans for what each plan’s allowance actually is, and Rate limits and credits for what happens at each limit.

Separating the two 503s

Both a saturated server and an unreachable index answer 503, and a single error counter merges two conditions with opposite responses. Split them on the one thing that distinguishes them — the presence of retry-after, confirmed by the body text:

if response.status_code == 503:
    kind = "at_capacity" if "retry-after" in response.headers else "no_shard"
    log.warning("unlob.unavailable", extra={"kind": kind, "body": response.text[:120]})

at_capacity means the request was refused before it ran, because the service was too busy to take it. It is transient, it costs no credits, and the header says when to come back. A brief cluster of them during a spike is normal; a sustained rate means your own concurrency is above what the service will take from you at once, and the fix is to slow down rather than to retry harder.

no_shard means the data plane could not serve the request. That one is not yours to wait out, and it is the one worth alerting on.

Four alerts worth setting up

  1. x-ratelimit-remaining under a fixed floor for several consecutive calls. Not a single low reading — the header reports the current minute’s remaining allowance, so one low value during a burst is normal. A floor held for multiple consecutive calls is the signal that a workload has outgrown its rate limit rather than merely spiked.

  2. A 429 with no retry-after at all. This one is binary and worth paging on rather than merely graphing: it means the key is hard-capped and out of credits for that call, and every call that costs as much will fail the same way until the period rolls over or the plan changes.

  3. partial: true rate climbing over a period. An occasional partial result is normal — some shard was briefly unreachable. A rising rate across many queries over hours, rather than one blip, is worth investigating rather than silently degrading every answer your agent gives during that window.

  4. A 503 with no retry-after on a search route. That is the data plane being unavailable rather than busy, and it is the one 503 a retry policy cannot fix. There is no public endpoint that reports the serving fleet’s internals, so this status on an ordinary search route is the signal to use.

What not to try to measure

Do not try to infer index freshness, corpus size, or serving-side latency from client-side observations — none of those are things a caller can measure correctly from outside, and this API deliberately does not publish index-time or storage figures for you to compare against (see the framework and API pages’ repeated point that what you observe as a caller — prices, response shapes, filter semantics, your own measured latency — is the whole contract; how the index is built and served is not). Your own round-trip latency is fair game to log and alert on, since it is genuinely something your process observed; a guess at what is happening server-side from that number is not.

Worked example: a weekly credits check

A small script, run on a schedule, that logs the account snapshot and flags the two conditions worth a human looking at:

import httpx, os

r = httpx.get(
    "https://api.unlob.com/account",
    headers={"x-api-key": os.environ["UNLOB_API_KEY"]},
    timeout=10,
)
acct = r.json()
used = acct["credits_used_this_period"]
allowance = acct["monthly_credits"]  # 0 means unlimited
plan = acct["plan"]

log.info("unlob.account", extra=acct)

if not acct["metered_billing"] and allowance and used > 0.8 * allowance:
    alert(f"unlob: {plan} key at {used}/{allowance} credits this period, "
          f"no payment method on file — will hard-cap at 429 when it runs out")

The metered_billing check matters: a key with a payment method attached keeps working past its allowance rather than hard-capping, so the same 80%-of-allowance threshold means something different depending on that one field.

FAQ

Does unlob provide a webhook or push notification for usage events? Not documented on this site. Polling GET /account on a schedule, as above, is the supported way to track usage — it costs no credits to call.

Should I log the full response body? For debugging a specific failure, yes, temporarily. For routine operation, the fields in the table above are what actually change what you would do next; logging every hit’s full snippet text on every call is usually more storage than signal.

Is caller-observed latency comparable between requests? Only roughly — it includes your own network path, not just server time, so it is a trend to watch over time rather than a number to compare directly against anyone else’s published benchmark figures.

Bottom line

Five fields, most of them already in hand on every call, cover request-level and account-level observability without adding a dependency: the rate-limit header for the current minute, partial and independent_sources for answer quality, status codes for the two 429 cases and 402, and a scheduled poll of GET /account for the monthly trend.

Next