Unique Error Analysis
You can look at the test failure analysis from different points of view: qualitative (passing rate – How many tests have failed?) and quantitative (Why have they failed?). For example, if 1000 test cases are failed, then
-
they could fail for the same reason
-
they could fail for various reasons
While failed tests allow you to understand what is broken, “Unique Error analysis” functionality implemented in version 5.7 will show you why it broke. The main advantage of this solution is that a list of all unique errors of the launch is presented in one place. Moreover, the system automatically groups tests by the same errors: when you expand some error, you see a list of steps where it occurred.
“Unique error auto-analysis” is set ON by default.
“Include/exclude numbers” settings
There are 2 settings: “include/exclude numbers” – it depends if you decide that numbers in error logs have significant value for analysis or not.
Now, Unique error auto-analysis will be started after a launch has been finished.
To see the list of “Unique errors” for the launch, open any item level in the launch and click “Unique errors” tab.
Finally, you can see the list of “Unique errors”.
There you can see a list with groups of error logs. You can expand a group to check what tests belong to the same one and it could give you a hint during error analysis and defects assigning. The groups are formed based on unique error logs, some small error logs can be merged and displayed as one error.
How to run “Unique error analysis” manually
You can also run “Unique error analysis” manually from any item level in case auto analysis is set OFF. Please, follow the steps below:
You can also run “Unique Error analysis” from the menu next to a particular launch.