Skip to content

Verify quality of test coverage #131

Open

Description

@tomato42

Use mutation testing to verify if the existing test coverage is good enough.

Options:

Activity

  1. added this to the v0.14 milestone on Oct 7, 2019
  2. tomato42 commented on Nov 3, 2019

    @tomato42
    MemberAuthor

    given that we are able to verify something like 20 mutants in 25 minutes in Travis, we would be able to mutation test only the simplest of simple patches – unusable without #127, maybe usable with it

    either way, not for 0.14

    side note: now, just before 0.14 is released (d6cb288), the test suite has 23.93% survival rate (76.07% mutation score)

  3. modified the milestones: v0.14, someday/future on Nov 3, 2019
  4. tomato42 commented on Nov 12, 2020

    @tomato42
    MemberAuthor

    with 0.16 we improved a bit, the survival rate is 21.33%

  5. tomato42 commented on Jul 9, 2022

    @tomato42
    MemberAuthor

    with 0.18.0 we improved a lot, the survival rate is now 16.86%, though I executed the unchanged test suite (not the one with changes from #156) and cosmic-ray changed significantly too

  6. tomato42 commented on Jan 15, 2024

    @tomato42
    MemberAuthor

    And before 0.19 release the survival rate is 21.51% (mutation score 78.49%), but we're able to execute about 33% of all mutations, so the confidence interval for survival rate is quite narrow: 20.80 to 22.22%.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedmaintenanceissues related to making the project usable or testable

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions