You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add IMDB queries (a.k.a. JOB - Join Order Benchmark) to DataFusion benchmark suite #12311
It is also used in HyPer, DuckDB, and CedarDB. It is a good benchmark for testing join ordering and join operators. It is also part of DuckDB's regression test suite.
I think if we add this test suite, it will also help with improvements like those discussed in #7955.
Describe the solution you'd like
JOB utilize the IMDB datasets. These datasets are provided in csv.gz format and represent real-world data, making them ideal for testing datafusion.
task
Convert the dataset from csv.gz format to Parquet.
Add the IMDB license to the LICENSE.
add benchmark queries.
Integrate the benchmark suite into dfbench.
Once everything is set up, we will be able to easily run benchmarks using the following command:
cargo run --bin dfbench --imdb --query=5
I would like to work on this!
Can someone help me understand the usual process for adding a third-party license in a Apache project ?
I think adding the join order benchmark would be reasonable.
Can someone help me understand the usual process for adding a third-party license in a Apache project ?
I would personally recommend following the model of the other benchmarks and not try and incorporate the files directly. Instead, download them on demand. If you do this I don't think we need any licensing updates
Activity
austin362667 commentedon Sep 4, 2024
@doupache Thanks. It seems promising to integrate the
Join Order Benchmark. I look forward to take the follow-up tasks.alamb commentedon Sep 5, 2024
I think adding the join order benchmark would be reasonable.
I would personally recommend following the model of the other benchmarks and not try and incorporate the files directly. Instead, download them on demand. If you do this I don't think we need any licensing updates
The benchmarking scripts are here: https://github.com/apache/datafusion/tree/main/benchmarks
I would recommend working on orchestrating the process using
https://github.com/apache/datafusion/blob/main/benchmarks/bench.sh
So a benchmark session might look like something like
TPCH does something similar (convert the
tsvoutput of the tpch data generator to parquet)alamb commentedon Sep 5, 2024
Thanks @doupache -- this sounds very cool
doupache commentedon Sep 17, 2024
Convert the dataset from
csv.gzformat toParquet. (Add JOB benchmark dataset [1/N] (imdb dataset) #12497)add benchmark queries (115 queries)
Integrate the benchmark suite into
dfbench.