GraphRAG Indexing Cost

Indexing is not one model call per chunk. It is four stages of model calls, and the largest of them is not the one you are thinking of. At default settings a 200-document corpus costs about 3,400 calls and sends roughly ten times the corpus's own token count to the model — most of it prompt template and re-sent conversation, not your text.

The stages, in the order the pipeline runs them. extract_graph sends each chunk to the model and asks for entities and relationships. Gleanings then ask the same model, in the same conversation, whether it missed anything — max_gleanings defaults to 1, so this stage exists on every install and doubles the call count of the first. summarize_descriptions makes one call for every entity and every relationship that got described more than once, because each chunk produced its own description and they have to be merged. create_community_reports writes one report per community per level of the Leiden hierarchy — not per community at the level you plan to query.

The panel below runs that pipeline. It chunks a corpus with real overlap arithmetic, draws entity mentions so that repeats accumulate the way they do in real text, builds the extraction conversation message by message the way graph_extractor.py does, and counts what each stage costs. Move max_gleanings first: it is the one people never touch.

Take max_gleanings from 1 to 0. The call count falls by about a quarter and the prompt tokens by nearly 40%, because the gleaning call is the expensive one: it re-sends the 1,620-token template, your whole chunk, and everything the model has already said. Now take it from 1 to 3 and watch the hero number roughly double. The GraphRAG paper reports that gleanings improve entity recall — they do — but the price of that recall is the most superlinear thing in the pipeline, and the default of 1 means you are already paying for one round of it whether or not you decided to.

The gleaning loop is a conversation, not a retry

The thing that makes gleanings expensive is not that they add a call. It is that they add a call to the same conversation. Read what graph_extractor.py does: it builds a message list, appends the model's answer to it, appends a fixed follow-up asking for more entities, and sends the whole list again. Nothing is dropped between rounds.

So with max_gleanings: 1 — the default — a 1,200-token chunk costs two calls whose prompts are 2,820 and roughly 3,750 tokens. Your 1,200 tokens of document were sent twice; the 1,620-token template was sent twice; the model's first answer was sent back to it. Set gleanings to 3 and the conversation grows on every round, so the fourth call carries the template, the chunk, and three previous answers. Prompt tokens for the stage rise faster than the call count does, which is exactly what the hero number shows when you move that slider.

There is a second call hiding in there. From the second gleaning onward the extractor sends a separate LOOP_PROMPT asking the model to answer "Y" or "N" to whether anything is still missing, and stops if the answer is not exactly Y. That check is a full model call over the entire accumulated conversation, and it returns one token. At max_gleanings: 3, up to two of your six calls per chunk exist only to ask whether to keep going.

The arithmetic of the loop is worth memorising, because it is not what the name suggests. max_gleanings: 0 is one call per chunk. 1 is two. 2 is up to four. 3 is up to six. It is not 1 + gleanings; it is 2 × gleanings, capped by the model answering "N" early.

If you are running a model with prompt caching, this is the one stage where it changes the shape of the bill, because every gleaning call shares a long identical prefix with the one before it — the same mechanics described in how a served model reuses a prefix. GraphRAG does not arrange the calls to make this happen, but many providers detect it automatically, and it is worth checking whether yours does before you conclude that gleanings are unaffordable.

The stage nobody budgets for

Extraction runs once per chunk and has no memory of any other chunk. So an entity appearing in forty chunks arrives with forty descriptions, and summarize_descriptions then has to merge them into one. The code makes a model call for every entity with more than one description, and the same for every relationship. On the default corpus in the panel that is over a thousand calls before a single community has been detected.

It gets worse for your most important entities. The merge batches descriptions until max_input_tokens (4,000) is reached, calls the model, then feeds the result back in as the first item of the next batch and continues. An entity with 300 descriptions is not one call; it is a chain of calls, each summarising the previous summary. The entity your corpus is most about is the one whose description has been through the most rounds of lossy compression, and by the last round the model is summarising summaries rather than reading anything you wrote.

This is also where graphrag update quietly changes meaning. Read index/update/communities.py: the incremental path indexes the new documents as a separate delta index, offsets the delta's community ids past the old maximum, and concatenates the two tables. There is no re-clustering of the combined graph. That is why an update is cheap, and it is also why a document you add in month six will never share a community with a document from month one, no matter how obviously related they are. If your corpus grows continuously, the honest options are a periodic full rebuild or accepting a hierarchy that is really several hierarchies stacked.

Community reports: the bill for a tree you do not query

Reports are generated per community per level. Move max_cluster_size in the panel from 10 to 40 and the report count drops by roughly three quarters while the level-0 partition stays exactly as it was, because that parameter never touches the top of the tree — the mechanism is in the community detection lesson. You are not losing top-level structure; you are declining to summarise the deep levels that a query at level 0 or 1 will never read.

Each report call carries a 2,214-token template plus a community context capped at 8,000 tokens, and returns up to 2,000 tokens. So a report is a large call in both directions, and there are thousands of them. This is the stage behind issue 746 — "I'm trying to index around 100 docs… the create_community_report cost me around 10 hours" — and it is invisible in any estimate that starts from "one call per chunk".

Before you run a full index, you can bound this stage exactly. Index a small sample, read len(communities) from the output, and note the ratio of communities to entities. That ratio barely changes with corpus size, because it is set by max_cluster_size, so multiply it by your projected entity count and you have your report call count to within a reasonable margin.

Checking it yourself

Four things to do, in the order that saves the most money.

Read the cache directory. Every completion is stored under a hash of its exact input arguments, in cache/, in subdirectories named after the stage: extract_graph, summarize_descriptions, community_reporting, extract_claims. Count the files in each. That is your true call count per stage, already broken down, with no instrumentation needed, and it answers the question in issue 671 that nobody answered.

Decide about gleanings deliberately. Run two small indexes, one with extract_graph.max_gleanings: 0 and one with the default 1, over the same 20 documents, and compare len(entities) and len(relationships). If the second finds 8% more entities for twice the extraction cost, that is a decision you can now make. Most people have never seen both numbers.

Watch the chunk size. Halving chunks.size at least doubles the number of text units, and more than doubles it once the chunk gets close to the 100-token overlap — a 3,800-token document is 4 units at 1,200 tokens, 8 at 600, and 19 at 300. Every text unit is a full extraction conversation. Small chunks are often chosen for retrieval quality reasons that apply to ordinary vector retrieval and do not transfer to graph extraction, where the model needs enough context to see a relationship at all.

Do not change settings casually mid-project. The cache key is a hash of the whole model-call input, so editing a prompt file, changing the entity type list, or switching model names invalidates every entry. Set the run mode selector to "index again after editing settings.yaml" to see what that costs. If you must experiment, experiment on a 20-document subset.

Next: what all this buys you at query time, and why the query bill is a separate one that also scales with corpus size, in global search fan-out. If the entity count in your index looks far too high, the extraction stage is producing duplicates and the fix is upstream, in entity resolution. And if you are considering building a custom retriever over the graph instead of using the community reports, read why extra hops add noise rather than answers first.