Repository navigation
fix: Improve slow fuzz test leaf_pushdown_fuzz - #26846
Draft
2010YOUY01 wants to merge 1 commit into
Draft
2010YOUY01 wants to merge 1 commit into
2010YOUY01 wants to merge 1 commit into
Conversation
Member
Author
|
Some refactoring is needed to make individual test cases reproducible from a seed. For example, if a test case fails: # Random seed determined at runtime
run_case(357)The error message should print the failing seed, so we can add it to the regression tests: def run_regression():
# All previously failing seeds
run_case(43)
run_case(357)I think this is probably an acceptable maintenance overhead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
case 1 of #26759
Rationale for this change
This section I'll try to explain the idea at high-level.
The simplest way to write a fuzzer is:
Given limited CI resources, we typically want to run as many rounds as possible within a small time budget, e.g. 5 seconds.
However, some coverage goals cannot be reached within that budget. For example:
leaf_pushdown_fuzz_extendedtakes around 4 minutes in total CI time, as only a fuzzing unit test.A better approach would be:
This trades some convenience in reproduction for broader coverage over time, and also uses way less CI resources.
What changes are included in this PR?
Apply the above idea for
leaf_pushdown_fuzztestLocally this test runs from 42s to 2s
What is the testing strategy for this PR?
Are there any user-facing changes?