You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am currently using this plugin for splitting up my tests.
A lot of my pytests consist of multiple test cases which are depended on one another.
Is there a possibility to split the tests on class basis?
Example: Testclass A: test_01: test_02: Testclass B: test_01: test_02: test_03: Testclass C: test_01:
Splitting those in two would do something like this:
Batch A: A(test_01, test_02) C(test_01)
Batch B: B (test_01, test_02, test_03)
Currently I am getting something like this:
Batch A: A(test_01, test_02) B(test_01)
Batch B: B(test_02, test_03), C(test_01)
Maybe I am missing something. If this is not possible at the moment are you willing to accept a Pull Request?
Greetings,
Andreas
Reacted by Khabibullin Danil, Christian Bundy and Jinzhe Zeng
Happy to take a PR for this 🙂 I this the best would be to have a CLI flag for it such that it could be used with all the current (and future) splitting algorithms. Another option would be to implement a new splitting algorithm which is tailored for this class use case.
FYI there's also demand for keeping tests from a single module within the same group: #108. Thus, I think introducing a CLI flag for both use cases is better than new splitting algorithms. For #108 I suggested --no-module-splits and for this one I think --no-class-splits would make sense. I believe there are plenty of synergies for these two features considering what kind of changes are needed in the current implementation.
My current implementation goes this way, that I want to support the following splitting boundaries:
class Boundary(enum.Enum):
FUNCTION = pytest.Function
CLASS = pytest.Class
MODULE = pytest.Module
PACKAGE = pytest.Package
FILE = pytest.File
DIR = pytest.Dir
And I thought about a parameter called boundary supporting the values
While implementing the PR for this "splitting on boundaries" feature I stumbled over an issue with the test coverage.
I get the warning:
~/git/pytest-split/.venv/lib/python3.12/site-packages/coverage/inorout.py:519: CoverageWarning: Module pytest_split was previously imported, but not measured (module-not-measured)
self.warn(msg, slug="module-not-measured")
And in the coverage report below I see that the first lines of each file are always not covered, which (from my limited understanding) should usually not be the case, because they are only imports and definitions.
I don't have an idea how to resolve this.
From what I read online it might be connected to the fact, that this plugin somehow uses an old version of the pytest plugin testing API.
Any suggestions how to remove this warning (and increase our test coverage)?
@jerry-git What shall I do with this test coverage issue mentioned in #82 (comment) (just above)?
Shall I submit my changes with some minor refactoring in tests or in separate commits?
I recently remembered that I didn't yet created a pull request for my solution to split on class boundaries (and other boundaries).
Shall I base it on the current master or will you integrate the fixes of vokracko soon?
I would love to see the improvements of vokracko in (although this would be some effort for me to rebase my changes to the newer version.
Activity
christianbundy commentedon Apr 5, 2024
Adding my use-case here: Django's
TestCaseruns one transaction per class, so it's optimal to run each class on a single worker.jerry-git commentedon Jun 19, 2024
Happy to take a PR for this 🙂 I this the best would be to have a CLI flag for it such that it could be used with all the current (and future) splitting algorithms. Another option would be to implement a new splitting algorithm which is tailored for this class use case.
long-git-hash commentedon Dec 24, 2024
I am currently working on an implementation for that.
jerry-git commentedon Dec 29, 2024
FYI there's also demand for keeping tests from a single module within the same group: #108. Thus, I think introducing a CLI flag for both use cases is better than new splitting algorithms. For #108 I suggested
--no-module-splitsand for this one I think--no-class-splitswould make sense. I believe there are plenty of synergies for these two features considering what kind of changes are needed in the current implementation.long-git-hash commentedon Jan 2, 2025
My current implementation goes this way, that I want to support the following splitting boundaries:
And I thought about a parameter called
boundarysupporting the valueslong-git-hash commentedon Jan 3, 2025
While implementing the PR for this "splitting on boundaries" feature I stumbled over an issue with the test coverage.
I get the warning:
And in the coverage report below I see that the first lines of each file are always not covered, which (from my limited understanding) should usually not be the case, because they are only imports and definitions.
I don't have an idea how to resolve this.
From what I read online it might be connected to the fact, that this plugin somehow uses an old version of the pytest plugin testing API.
Any suggestions how to remove this warning (and increase our test coverage)?
long-git-hash commentedon Jun 4, 2025
@jerry-git What shall I do with this test coverage issue mentioned in #82 (comment) (just above)?
Shall I submit my changes with some minor refactoring in tests or in separate commits?
mrohleff commentedon Sep 12, 2025
@jerry-git any update on this topic?
long-git-hash commentedon Sep 12, 2025
My custom version splitting on class boundaries is now running for half a year in our test rack.
long-git-hash commentedon Jun 11, 2026
I recently remembered that I didn't yet created a pull request for my solution to split on class boundaries (and other boundaries).
Shall I base it on the current master or will you integrate the fixes of vokracko soon?
I would love to see the improvements of vokracko in (although this would be some effort for me to rebase my changes to the newer version.