You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the existing issues and this bug is not already filed.
My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here. (Not applicable: the bug is in the input reader, before any model call.)
I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.
Describe the bug
JSONLinesFileReader.read_file splits the file with text.splitlines() (jsonl.py#L38). Python's str.splitlines() also breaks on U+2028 (LINE SEPARATOR), U+2029 (PARAGRAPH SEPARATOR) and U+0085 (NEXT LINE). JSON Lines uses only \n as the record separator, and those three characters are valid unescaped inside JSON strings. A valid row containing one of them is therefore cut into two fragments, neither of which parses. Each fragment is logged as Skipping malformed JSONL row in <path> at line N, so that row is dropped from the input (the rest of the file is still read), and the only trace is a warning.
(Side note: splitlines() also splits on \x0b, \x0c, \x1c, \x1d and \x1e. Those are control characters that must be escaped in JSON, so they cannot occur unescaped in a valid row and are not part of this bug.)
Steps to reproduce
File x.jsonl, where the first row contains a raw U+2028 between a and b (shown here as <LS>):
{"text":"a<LS>b"}
{"text":"c"}
docs=awaitJSONLinesFileReader(storage=storage).read_files()
print([d.textfordindocs]) # ['c'] (expected ['a b', 'c'])# log: Skipping malformed JSONL row in x.jsonl at line 1# Skipping malformed JSONL row in x.jsonl at line 2
json.loads('{"text":"a b"}') parses fine on its own. The same happens with U+2029 and U+0085.
Expected Behavior
Both rows are read. Split on "\n" only and strip a trailing "\r" for CRLF files.
GraphRAG Config Used
input:
type: jsonl
Logs and screenshots
Skipping malformed JSONL row in x.jsonl at line 1
Skipping malformed JSONL row in x.jsonl at line 2
Additional Information
GraphRAG Version: 3.3.0 (graphrag-input at 5faaaf4)
Suggested fix: for line_number, line in enumerate(text.split("\n"), start=1): line = line.rstrip("\r"), plus a test with U+2028 and U+0085 inside a string value.
PRs here are limited to collaborators, so the fix is on a branch instead: main...nefayran:graphrag:fix/jsonl-unicode-line-separators. It splits on "\n" only (a trailing \r is whitespace to json.loads), adds a test with U+2028, U+2029 and U+0085 rows in a CRLF file, and a patch entry under .semversioner/next-release.
Activity
nefayran commentedon Oct 10, 2026
PRs here are limited to collaborators, so the fix is on a branch instead: main...nefayran:graphrag:fix/jsonl-unicode-line-separators. It splits on
"\n"only (a trailing\ris whitespace tojson.loads), adds a test with U+2028, U+2029 and U+0085 rows in a CRLF file, and a patch entry under.semversioner/next-release.