Flaky test cases table (TOP-50)
Shows the TOP-50 the most flaky test cases within the specified previous launches.
The test case is displayed in the table if test case status has changed at least once from Passed to Failed or from Failed to Passed in the specified previous launches.
For the Flaky Test Cases Table Top 50 widget, only the status of the last retry of a test case is considered. If a test case undergoes multiple retries during different launch executions, and the last retries of each launch yield identical statuses, the test case will not be classified as flaky. Thus, for a test to be identified as flaky, the final retry statuses in each launch must differ.
Widget's parameters:
-
Launches count: 2-100. The default meaning is 30.
-
Launch name: required filed.
-
Include Before and After methods: optional.
Widget view
The widget has a table view with the following data displayed:
- Test Case - link to the Step level of the last launch.
- Switches - count of found results with often switches.
- % of Switches - the percent of the fact switches and the possible.
- Last switch - date and time of the last run, when the test item switches the status, displayed in “time ago” format (i.e. “10 minutes ago”).
On mouse hover, the system will display accurate start times.
In “Switches” column only Passed and Failed statuses are displayed (Passed - green, Failed - red).
The number of switches from one state to another of the test case with the same uniqueID is displayed in the format N from M where:
N is the number of changes of statuses.
M is the number of all possible changes of the item in selection (number of item runs=number of test executions minus number of executions with status Skipped minus 1).
On mouse hover, tooltip appears “N status changes from M possible times”.