Skip to content

Consolidate and clean up tests / make them more uniform / refactor #743

Closed

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

As @jorgecarleitao pointed out many moons ago, in apache/arrow#9936 (review) , the tests in datafusion/src/context.rs are not really unit tests. They are more like SQL integration tests.

There is also a small and languishing set of sql tests in rust/datafusion/tests/sql.rs that use a string comparison style that I personally find quite hard to upgrade.

Among other things, this split makes it hard to understand the state of DataFusion's test coverage.

Since these tests are critical for DataFusion's quality, I propose a small reorganization so it is easier to find existing test coverage and write new ones:

Describe the solution you'd like

Specifically I propose:

  • move the assert_batches_eq! macros to a non part of datafusion #745
  • Update the style of all sql.rs tests to be inline with that in context.rs (using assert_batches_eq!)
  • Move tests that are not specific to ExecutionContext out of context.rs and into sql.rs
  • (maybe) move rust/datafusion/src/test to its own module rust/test_helpers (so that it can be shared with sql.rs)

Then over time I imagine being able to organize the tests within sql.rs better (split into multiple modules, for example)

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
None

Additional context
Original mailing list thread: https://lists.apache.org/thread.html/r77cb6f55952803debdbe10bb615d00ec097942fb88c1a88496b3d69a%40%3Cdev.arrow.apache.org%3E

Activity

  1. changed the title [-]Consolidate datafusion test / make them more uniform[/-] [+]Consolidate and clean up datafusion test / make them more uniform[/+] on Oct 15, 2021
  2. changed the title [-]Consolidate and clean up datafusion test / make them more uniform[/-] [+]Consolidate and clean up tests / make them more uniform[/+] on Oct 15, 2021
  3. matthewmturner commented on Nov 24, 2021

    @matthewmturner
    Contributor

    @alamb i believe we can mark task 2 completed now. If you agree then I can start looking at 3

  4. alamb commented on Nov 24, 2021

    @alamb
    ContributorAuthor

    @alamb i believe we can mark task 2 completed now. If you agree then I can start looking at 3

    Done, and THANK YOU!

  5. changed the title [-]Consolidate and clean up tests / make them more uniform[/-] [+]Consolidate and clean up tests / make them more uniform / refactor[/+] on Dec 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    development-processRelated to development process of DataFusionenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions