Skip to content

Fix JSON parsing offsets at EOF - #3635

Merged
wwbmmm merged 3 commits into
apache:masterfrom
chenBright:fix_rapidjson_offset
Oct 22, 2026
Merged

wwbmmm merged 3 commits into
apache:masterfrom
chenBright:fix_rapidjson_offset

Conversation

@chenBright

@chenBright chenBright commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: Related to #3050

Problem Summary:

When allow_remaining_bytes_after_parsing is enabled, parsing a JSON value that
ends exactly at EOF succeeds but returns parsed_offset == 0. For example, parsing
"{}" returns zero instead of two.

Callers that resume parsing with json.substr(parsed_offset) therefore repeatedly process
the final message and may enter an infinite loop.

What is changed and the side effects?

Changed:

Always record the consumed offset when iterative parsing finishes a JSON value with
kParseStopWhenDoneFlag, including at EOF. This matches the existing non-iterative
behavior.

Side effects:

  • Performance effects:

  • Breaking backward compatibility:


Check List:

@chenBright
chenBright requested a balanced review from Copilot October 21, 2026 03:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues
What changed in this PR

Fixes incorrect parsed_offset reporting when JSON parsing stops exactly at EOF with allow_remaining_bytes_after_parsing, preventing callers from reprocessing the same bytes and potentially looping indefinitely.

Changes:

  • Update RapidJSON iterative parsing to always record the consumed offset when kParseStopWhenDoneFlag is used, even at EOF.
  • Add unit tests covering offset behavior at EOF for both string and stream-based parsing.
  • Add coverage for parsing multiple concatenated JSON values and resuming correctly at EOF.
File Description
test/​brpc_protobuf_json_unittest.cpp Adds regression tests for parsed offset correctness at EOF and for concatenated JSON parsing behavior.
src/​butil/​third_party/​rapidjson/​reader.h Ensures parseResult_.Offset() is updated at EOF when stop-when-done iterative parsing completes.

Review effort: Lite

Comment thread test/brpc_protobuf_json_unittest.cpp Resolved
Comment thread test/brpc_protobuf_json_unittest.cpp Resolved
@chenBright
chenBright requested a balanced review from Copilot October 21, 2026 06:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues
3 resolved since last review

Review effort: Lite

Comment thread test/brpc_protobuf_json_unittest.cpp Resolved
@chenBright
chenBright requested a balanced review from Copilot October 21, 2026 07:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues
2 resolved since last review

Review effort: Lite

Comment thread test/brpc_protobuf_json_unittest.cpp Resolved
@chenBright

chenBright commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

@wwbmmm These are optional readability/style improvements of copilot, so no further changes are needed. This PR is ready for maintainer review.

@chenBright
chenBright requested a review from wwbmmm October 21, 2026 07:55
@wwbmmm
wwbmmm merged commit e6fd4c5 into apache:master Oct 22, 2026
25 checks passed
@chenBright
chenBright deleted the fix_rapidjson_offset branch October 22, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants