Gleaning responses are concatenated without a record delimiter, silently dropping records at the join #2605

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed. (Closed [Bug]: graph_extractor miss entities while processing llm result #1183 covered part of it and was closed as not planned; see Additional Information.)

• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" section of the Get Started guide.

  Noting this one honestly: I found this against a self-hosted model, but the bug is in string handling after the response arrives and is model-independent. The repro below calls no model at all.

• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

GraphExtractor._process_document concatenates each gleaning response directly onto the previous one:

https://github.com/microsoft/graphrag/blob/main/packages/graphrag/graphrag/index/operations/extract_graph/graph_extractor.py#L109

results += response_text

No record delimiter is inserted. The shipped extraction prompt never asks the model to terminate its final record, and its own example shows the last record ending with ) followed by <|COMPLETE|> on the next line:

("relationship"<|>MARTIN SMITH<|>CENTRAL INSTITUTION<|>Martin Smith is the Chair of the Central Institution<|>9)
<|COMPLETE|>

So when _process_result splits on ##, the previous response's final record and the next response's first record are one record.

The fused record is not rejected. It parses as the first record with extra tuple fields appended, and _process_result reads only fields [1], [2], [3], so the second record's fields are silently discarded:

['"entity"', 'CENTRAL INSTITUTION', 'ORGANIZATION', 'The Central Institution ...)\n<|COMPLETE|>\n("entity"', 'ALICE CHEN', 'PERSON', 'Alice Chen is ...']
   [0]              [1]                  [2]                      [3]                                          [4]           [5]          [6]

Nothing is logged. The entity or relationship the gleaning was called to recover is simply absent from the graph. max_gleanings defaults to 1, so a default configuration can lose up to one record per text unit.

The same prompt shape causes a second, independent problem with no gleanings involved. Because the final record ends ...<|>9)\n<|COMPLETE|>, the string does not end in ), so re.sub(r"^\(|\)$", "", record) cannot strip the closing paren. record_attributes[-1] is then 9)\n<|COMPLETE|>, float() raises ValueError, and the except ValueError branch sets weight = 1.0. Every text unit's last relationship gets weight 1.0 regardless of what the model scored it, which feeds into community detection and ranking.

The record == COMPLETION_DELIMITER guard on line 139 only fires when the delimiter is on its own record, which happens only if the model terminates its last record first.

Steps to reproduce

Run extraction with max_gleanings: 1 (the default) and a model that follows the prompt's example formatting. Entities that appear only in a gleaning continuation go missing, and the last relationship of each text unit has weight 1.0.

Both halves reproduce without an LLM call, using the library's own parser and two responses shaped exactly like the prompt's Example 1 output:

from graphrag.index.operations.extract_graph.graph_extractor import GraphExtractor

BASE = (
    '("entity"<|>CENTRAL INSTITUTION<|>ORGANIZATION<|>The Central Institution is the Federal Reserve of Verdantis)\n'
    "##\n"
    '("relationship"<|>MARTIN SMITH<|>CENTRAL INSTITUTION<|>Martin Smith is the Chair of the Central Institution<|>9)\n'
    "<|COMPLETE|>"
)
GLEANING = (
    '("entity"<|>ALICE CHEN<|>PERSON<|>Alice Chen is a policy analyst at the Central Institution)\n'
    "##\n"
    '("relationship"<|>ALICE CHEN<|>CENTRAL INSTITUTION<|>Alice Chen analyses policy<|>7)\n'
    "<|COMPLETE|>"
)
ge = GraphExtractor.__new__(GraphExtractor)

# (a) weight corruption, no gleanings involved
_, r = ge._process_result(BASE, "tu1", "<|>", "##")
print(r.iloc[0]["weight"])                      # 1.0, model said 9

# (b) what _process_document returns today for one gleaning
e, _ = ge._process_result(BASE + GLEANING, "tu1", "<|>", "##")
print(sorted(e["title"]))                       # ['CENTRAL INSTITUTION'] -- ALICE CHEN is gone

# (c) the same two responses joined by a record delimiter
e, _ = ge._process_result(BASE + "\n##\n" + GLEANING, "tu1", "<|>", "##")
print(sorted(e["title"]))                       # ['ALICE CHEN', 'CENTRAL INSTITUTION']

The difference between (b) and (c) is the whole of the first bug: the two responses carry the same records, and only the join decides whether one of them survives.

Expected Behavior

• An entity or relationship emitted by a gleaning continuation appears in the extracted graph.
• A relationship's weight is the value the model emitted, not the 1.0 fallback.

GraphRAG Config Used

extract_graph:
  model_id: default_chat_model
  prompt: "prompts/extract_graph.txt"
  entity_types: [organization, person, geo, event]
  max_gleanings: 1

Logs and screenshots

No logs are produced. That is the core of the report: both failures are silent. The fused record satisfies record_attributes[0] == '"entity"' and len(record_attributes) >= 4, so it is accepted, and the weight failure is swallowed by except ValueError: weight = 1.0.

Additional Information

• GraphRAG Version: main @ v3.3.0
• Operating System: Linux / macOS
• Python Version: 3.12
• Related Issues: [Bug]: graph_extractor miss entities while processing llm result #1183 reported the concatenation half of this in September 2024 with the correct fix, and was closed as not planned in August 2025 with no discussion. The completion-delimiter half of the problem was not covered there.

I have a fix ready with a regression test and will open a PR referencing this issue.