unlob Docs
Browse documentation

The Coverage Graph

Signals on every hit, and six calls that use the structure instead of a ranking over it.

Two ways to use it

There is a graph over the corpus — passages linked to hosts, to near-duplicate stories, to topics and to entities. You can use it two ways, and the cheap way covers most of what you need.

The cheap way: signals on every hit. centrality, independent_sources, community_id and in_degree come back on every search result, and /search filters and sorts on them:

# corroborated results only, most central first
curl -sG -H "x-api-key: $UNLOB_API_KEY" https://api.unlob.com/search \
  --data-urlencode 'q=merger terms' -d min_independent_sources=3 -d sort=centrality

No extra call, no extra cost. For most questions this is enough.

The structural way: six calls. When you need the shape of the connections rather than a ranking derived from it.

The six

CallReturnsThe loop it replaces
corroborateDistinct hosts carrying a story, grouped and ranked by authorityCross-searching by hand to see how widely a claim is carried
dossierAn entity’s mentions, top sources, and co-mentioned entitiesAbout ten searches and a manual merge
authoritiesTop passages on a topic by centrality, not by query matchReading junk into context before you know the vocabulary
relatedThe connected neighbourhood of a passageA search per hop
pathThe shortest edge chain between two passagesGuessing at what connects two things
assemble_contextA deduplicated, trust-ranked, budget-packed reading setThe entire retrieval loop

corroborate is the one to build a habit around

curl -sG -H "x-api-key: $UNLOB_API_KEY" https://api.unlob.com/corroborate \
  -d 'id=https://example.com/article:0'
{
  "story_id": "…",
  "independent_sources": 4,
  "merged_duplicates": 37,
  "sources": [
    { "host": "reuters.com", "host_rank": 0.97, "passages": [ … ] },
    { "host": "ft.com", "host_rank": 0.95, "passages": [ … ] }
  ]
}

Forty rows in a result list and independent_sources: 1 describe the same corpus. The first looks like consensus; the second is one source and thirty-nine reprints.

One detail in your favour: when a near-duplicate copy is dropped at ingest, the host that published it is still recorded against the passage that survived. So independent_sources counts every host that carried the story, not only the ones whose copy was kept.

One detail to be honest about: hosts are not independent sources. Ten hosts can run one wire dispatch, copy one press release, or belong to one media group, and corroborate counts every one of them. ground is the call that computes independence — it groups a story’s members into origins, assigns owners, and reports independent_origins. Use corroborate to see how widely a story is carried, and ground to know how many places it actually came from. Provenance and independence has the rules.

dossier gives you the leads

curl -sG -H "x-api-key: $UNLOB_API_KEY" https://api.unlob.com/dossier \
  --data-urlencode 'entity=Acme Corporation'

Mentions, top source hosts, and related_entities as [entity, count] pairs. That third list is what earns the call: it is the people, products and organisations you did not know to search for. Feed the interesting ones back in and you have walked a network in three calls.

assemble_context is the whole loop

curl -sG -H "x-api-key: $UNLOB_API_KEY" https://api.unlob.com/assemble_context \
  --data-urlencode 'q=what happened with the Acme acquisition' -d budget=4000

Search, deduplicate, corroborate, rank, pack — done on the server, returned as a context pack where each item carries the reason it was included. budget is a token ceiling; estimated_tokens reports what was actually packed.

Use it when the question is “what should I know about X”. Use search when the question is “find me the page that says Y”.

Node keys

path returns a chain of namespaced nodes, so the chain reads as an explanation rather than a route:

PrefixNode
p:Passage
h:Host
s:Story (near-duplicate cluster)
t:Topic
e:Entity

Two documents linked through a shared entity and two linked only through a shared topic are very different findings. found: false is also an answer: within everything the index holds, these two things are not connected.

A research loop, six calls

  1. authorities on the topic — acquire the vocabulary.
  2. search with min_independent_sources — corroborated hits, in that vocabulary.
  3. corroborate the one you intend to rely on.
  4. dossier on the entity it names — find the leads.
  5. path between two leads that look connected — find out whether they are.
  6. assemble_context on the refined question — hand yourself the reading.

The graph calls cost 2 credits each, assemble_context 3 and a search 1. See Graph recipes for the worked version.

Next