You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@scripnichenko we are not using soft assertions in our code and you are the first one who are asking for such issue during last few years. Seems like everyone uses AssertJ instead of TestNG assertions this days. But I'm understand that issue is important for people who support old projects written years ago and have no possibility for refactoring.
BTW I have no idea how to fix this, some research work is required. Probably the easiest way to fix the issue add an aspect. If you have any ideas feel free to share.
Can anyone explain the expected behavior and attach the minimal code to reproduce the issue? As I don't really understand what kind of SA support is required.
I will try from my point of usage:
When in the an of API (for example), I'm checking the JSON response, I need to test all fields that exist as expected.
I can run soft assertion or run a new test to validate one filed.
Or in UI - when by click on an element will be an open window with 3 parameters that I need to validate, I will use SA.
@sskorol mark each failed step (and each of its ancestors) with red color, add assertion message within failed step content, add stack-trace within assertion error content, add summary error-message as test result, e.g. 'N assertions failed: <error 1> (expandable with stack-trace), <error 2> (expandable) ...
@sskorol : if I can I would try to support the feature enhancement. Can you give me some advise about entry points where to start? I guess there is a kind of post-processing for the final test / step state (updating the TestResult information).
public class AllureSelenideCustom {
public static void markOuterStepAsFailedAndStop() {
AllureLifecycle lifecycle = getLifecycle();
lifecycle.updateStep(step -> step.setStatus(Status.FAILED));
lifecycle.stopStep();
}
public static void stepSoftAssert(final String passMessage, final String failMessage,
final Consumer<String> consumer, final boolean markStepAsFailed) {
Allure.step(markStepAsFailed ? failMessage : passMessage
, () -> {
consumer.accept(failMessage);
if (markStepAsFailed) {
markOuterStepAsFailedAndStop();
}
});
}
}
The bad new is you should pass to this method boolean markStepAsFailed. It means you should additionally ("manually") check result before use it.
Its ok for me in cases when I looking for "bad items" and pass as argument something like badItem.exists()
Activity
[-][TestNG] Add support for soft asserions[/-][+]Add support for soft assertions[/+]scripnichenko commentedon Dec 17, 2018
Hi @baev. Any plans to get it implemented? 💡
baev commentedon Dec 17, 2018
@scripnichenko we are not using soft assertions in our code and you are the first one who are asking for such issue during last few years. Seems like everyone uses AssertJ instead of TestNG assertions this days. But I'm understand that issue is important for people who support old projects written years ago and have no possibility for refactoring.
BTW I have no idea how to fix this, some research work is required. Probably the easiest way to fix the issue add an aspect. If you have any ideas feel free to share.
Little-Elephant commentedon Feb 12, 2019
I'll be the second one.
Lyisko commentedon Jun 27, 2019
I'll be the third one.
vlayon commentedon Feb 25, 2020
Hello there.
I want to ask also for implementing SoftAsserts in Allure. Are you going to do it?
..because now the bahavior is : current step, which has failed soft assert in it, does not fail. The case fails when assertAll method is called.
bereg2k commentedon Nov 3, 2020
Yep, that's an important feature.
Guter1988 commentedon Jan 19, 2021
I use it too.
And due to this, add attachments when it failed to find a better solution.
sskorol commentedon Jan 19, 2021
Can anyone explain the expected behavior and attach the minimal code to reproduce the issue? As I don't really understand what kind of SA support is required.
Guter1988 commentedon Jan 21, 2021
I will try from my point of usage:
When in the an of API (for example), I'm checking the JSON response, I need to test all fields that exist as expected.
I can run soft assertion or run a new test to validate one filed.
Or in UI - when by click on an element will be an open window with 3 parameters that I need to validate, I will use SA.
I hope it helped :)
sskorol commentedon Jan 21, 2021
@guterscooter I know how SA works. I wonder what do you expect to see in Allure while using SA?
Guter1988 commentedon Jan 24, 2021
@sskorol To see all parts where it failed.
I see only the first one, while the next can failed will not be shown at Allure.
twixdouble commentedon Feb 17, 2021
@sskorol mark each failed step (and each of its ancestors) with red color, add assertion message within failed step content, add stack-trace within assertion error content, add summary error-message as test result, e.g. 'N assertions failed: <error 1> (expandable with stack-trace), <error 2> (expandable) ...
lbastil commentedon Nov 9, 2021
@sskorol: I think in #529 everything is explained quite well:
What is the current behavior?
current step, which has failed soft assert in it, does not fail. The case fails when assertAll method is called.
What is the expected behavior?
hope it show failed in each step and make the test failed
Example + Expectation:
On a very general level an example would be:
One would expect in this case the following report result:
But it is currently:
Which is misleading and hard to review for the test team which evaluates the allure report details
lbastil commentedon Nov 9, 2021
@sskorol : if I can I would try to support the feature enhancement. Can you give me some advise about entry points where to start? I guess there is a kind of post-processing for the final test / step state (updating the TestResult information).
deisenberg-remedy commentedon Jun 12, 2023
What is the status on implementing Soft Assertion support as described above?
Phlegethonyarre commentedon Jul 21, 2023
Hi guys. A little workaround here I created long time ago, this could be helpful for your own ideas, probably with aspects :)
Achitheus commentedon Nov 10, 2023
Maybe I'm wrong, but it looks like the
run()method never throws errors because it has a soft assertion inside?Achitheus commentedon Nov 10, 2023
I did this for my needs:
The bad new is you should pass to this method
boolean markStepAsFailed. It means you should additionally ("manually") check result before use it.Its ok for me in cases when I looking for "bad items" and pass as argument something like
badItem.exists()