Repository navigation
Conversation
|
run buildall |
|
/review |
There was a problem hiding this comment.
Static review of PR #70546 at head ad96168 found no new findings. The only change adds six fixed-name ALTER TABLE ... MODIFY COLUMN ... SET STATS ('row_count'='3') statements after creating and loading three-row OLAP fixtures. Source tracing confirms each statement validates synchronously, writes and synchronizes the column statistic, marks the base-index table row count as user-injected, and is consumed by Nereids when selecting the bucket-shuffle basic child. The injected count matches the data, and count-only stats leave NDV untrustable, so no result or unrelated join-plan coverage defect was substantiated. The change is focused and follows existing regression-test syntax and setup conventions; no concurrency, lifecycle, configuration, protocol, persistence, or source-data path is introduced. Existing inline threads: none. User review focus: none. Static validation covered the complete authoritative diff, changed-path list, existing review context, and upstream/downstream statistics flow; no builds or runtime tests were executed under the static-review contract. Review converged in round 1 because both assigned review passes returned NO_NEW_VALUABLE_FINDINGS and the only risk focus was dismissed with evidence.
Review converged after 1 round(s).
|
run compile |
### Background The bucket-shuffle set-operation suite checks physical plan shapes as well as query results. INTERSECT input ordering and bucket-shuffle anchor selection depend on estimated row counts. ### What problem does this PR solve? Issue Number: None Related PR: apache#65129 (enabled the suite and added bucket-shuffle coverage) PR Type: Test Problem Summary: The three initial tables each contain three rows, but asynchronous tablet statistics updates can leave one estimated at three rows and another at one row. This changes INTERSECT input order or the UNION anchor and causes intermittent shape mismatches. Inject row_count=3 for both columns of these tables before checking plans. Keep the existing queries, assertions, and golden results. ### Details - **Failure:** INTERSECT children can swap, or the join above UNION ALL can change from colocated to bucket shuffle. - **Cause:** The plan checks relied on asynchronously reported row counts immediately after creating and loading the tables. - **Stability evidence:** The complete suite passed 10 consecutive runs with default settings and another 10 with enable_spill=true. Only row counts are specified: adding detailed column statistics changes an unrelated join choice and bypasses the parent-hash-request coverage. All existing shape and result checks remain intact, including the fixtures with larger injected row counts later in the suite. ### Testing On a local macOS instance with one FE and one ASAN BE, built from b325dcd: - Single complete suite run passed. - `./run-regression-test.sh --run -d query_p0/set_operations -s bucket_shuffle_set_operation -parallel 1 -times 10`: 10/10 passed with default settings. - The same command with global enable_spill=true: 10/10 passed. - Regression framework compilation and its 50 unit tests passed. - FE build passed with zero Checkstyle violations; BE build passed. - `git diff --check` passed. No golden output changes. ### Release note None ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Codex - Generated: Test setup and PR description. - Human review: Pending maintainer review.
ad96168 to
af151e9
Compare
|
run buildall |
|
/review |
|
Codex automated review failed and did not complete. Error: Review setup stopped before context preparation ran; inspect the first failed or cancelled setup step. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
|
Codex automated review failed and did not complete. Error: Review setup stopped before context preparation ran; inspect the first failed or cancelled setup step. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
|
/review |
There was a problem hiding this comment.
Static review of PR #70546 at af151e9 found no new findings.
The change pins the three initial fixtures at their actual three-row cardinalities before any plan or result check. Source tracing confirms that ALTER stats publishes user-injected base-index counts, which Nereids uses ahead of tablet-reported counts. The incomplete column records become UNKNOWN through ColStatsData.isValid(), preserving unknown detailed column statistics rather than introducing false NDV/range estimates. Cache reloads apply the same validation.
Critical checkpoint conclusions:
- Goal and proof: the setup addresses the identified row-count race. Existing shape assertions and ordered-result goldens remain intact; runtime stability was not independently tested here.
- Scope and clarity: ten setup lines in one regression file, using the existing statistics-injection facility.
- Concurrency and locks: SQL statements complete sequentially. Injected counts take precedence over asynchronous tablet reports; no threads, locks or lock-order changes are introduced.
- Lifecycle: fixtures are recreated before use, statistics are keyed by table IDs, and the suite reuses its connection. ForwardWithSync waits for metadata replay on a follower; no new resource lifetime is introduced.
- Configuration: no configuration item or session-variable assignment is added.
- Compatibility: no function symbols, storage formats or production behavior change.
- Thrift/protobuf: no gensrc changes; field-ID comparisons are inapplicable.
- Parallel paths: all three initial fixtures and both columns are covered. Id-only, value-only, full-row, unequal-bucket, local-shuffle and planner-switch cases remain present. Later right-basic and fill-up tables retain their independent larger counts.
- Conditional checks: no new conditional or fallback; fixed loops cover the intended six table/column pairs.
- Test coverage: the complete suite was inspected, including UNION/INTERSECT/EXCEPT, joins, windows, parent-hash requests, right-basic selection, missing-bucket/equivalent-key cases, multi-column fallback and nullable keys. No new kernel feature requires unit tests.
- Test results: no golden changes. All existing output blocks were inspected against unchanged fixture data and queries; no executed result is claimed.
- Observability: existing SQL logging and explain/result failure diagnostics cover this setup addition; no production metrics are needed.
- Transactions and persistence: existing statistics persistence, metadata journaling and forwarding are reused without implementation or failover-protocol changes.
- Data writes and atomicity: statistics DDL follows completed fixture inserts. Source rows are unchanged; SQL errors propagate and fail the suite.
- FE/BE variables: no transmitted variable or protocol field is added.
- Performance: six bounded metadata statements per suite; no production hot path, large allocation or unbounded work is added.
- Other issues and core invariants: no change to data visibility, visible versions, delete bitmaps, memory ownership or nullability. No further defect was substantiated after rechecking column-cache validation and downstream optimizer consumers.
Existing inline threads and independently confirmed P0/P1 blockers: none. Additional user review focus: none.
Review converged in round 1: both the complete-review agent and the separate statistics-risk agent returned NO_NEW_VALUABLE_FINDINGS. Main-agent verification dismissed both risk focuses with evidence; no unresolved candidates or uncovered changed paths remain. Validation was static inspection only: no builds, tests or benchmarks ran, and the author's repeated-run claims were not independently reproduced.
Review converged after 1 round(s).
Background
The bucket-shuffle set-operation suite checks physical plan shapes as well as query results. INTERSECT input ordering and bucket-shuffle anchor selection depend on estimated row counts.
What problem does this PR solve?
Issue Number: None
Related PR: #65129 (enabled the suite and added bucket-shuffle coverage)
PR Type: Test
Problem Summary:
The three initial tables each contain three rows, but asynchronous tablet statistics updates can leave one estimated at three rows and another at one row. This changes INTERSECT input order or the UNION anchor and causes intermittent shape mismatches. Inject row_count=3 for both columns of these tables before checking plans. Keep the existing queries, assertions, and golden results.
Details
Testing
On a local macOS instance with one FE and one ASAN BE, built from b325dcd:
./run-regression-test.sh --run -d query_p0/set_operations -s bucket_shuffle_set_operation -parallel 1 -times 10: 10/10 passed with default settings.git diff --checkpassed. No golden output changes.Release note
None
Check List (For Author)
AI Disclosure