Repository navigation
Fix JSON parsing offsets at EOF - #3635
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
The sameoffsetvariable is reused for two different semantics in the same loop (string parse… · New Since the behavior is being changed specifically to record offsets even at EOF, it would help to… · Newarray_to_single_repeated = trueis not required for most of the test cases here and broadens the… · New
What changed in this PR
Fixes incorrect parsed_offset reporting when JSON parsing stops exactly at EOF with allow_remaining_bytes_after_parsing, preventing callers from reprocessing the same bytes and potentially looping indefinitely.
Changes:
- Update RapidJSON iterative parsing to always record the consumed offset when
kParseStopWhenDoneFlagis used, even at EOF. - Add unit tests covering offset behavior at EOF for both string and stream-based parsing.
- Add coverage for parsing multiple concatenated JSON values and resuming correctly at EOF.
| File | Description |
|---|---|
| test/brpc_protobuf_json_unittest.cpp | Adds regression tests for parsed offset correctness at EOF and for concatenated JSON parsing behavior. |
| src/butil/third_party/rapidjson/reader.h | Ensures parseResult_.Offset() is updated at EOF when stop-when-done iterative parsing completes. |
Review effort: Lite
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
2 open issues
3 resolved since last review
Review effort: Lite
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
This is a vendored third-party header, but the comment is project-specific (JsonToProtoMessage())… · New UsingSetParseError(kParseErrorNone, ...)to update a successful parse offset is non-obvious… · New The test nameparse_json_offset_at_eofis misleading because the cases include non-EOF-stop… · New
2 resolved since last review
Review effort: Lite
|
@wwbmmm These are optional readability/style improvements of copilot, so no further changes are needed. This PR is ready for maintainer review. |


What problem does this PR solve?
Issue Number: Related to #3050
Problem Summary:
When
allow_remaining_bytes_after_parsingis enabled, parsing a JSON value thatends exactly at EOF succeeds but returns
parsed_offset == 0. For example, parsing"{}"returns zero instead of two.Callers that resume parsing with
json.substr(parsed_offset)therefore repeatedly processthe final message and may enter an infinite loop.
What is changed and the side effects?
Changed:
Always record the consumed offset when iterative parsing finishes a JSON value with
kParseStopWhenDoneFlag, including at EOF. This matches the existing non-iterativebehavior.
Side effects:
Performance effects:
Breaking backward compatibility:
Check List: