Component health check
The widget shows the passing rate of the application components, indicated by the attributes specified for test cases.
For using this widget you need to report (or add manually) attributes to test items.
Widget's parameters:
- Filter
- Parameters: All Launches/Latest Launches
- Checkbox 'Exclude Skipped Tests from Statistics'. Default state of checkbox: unselected.
- Minimum allowable passing rate for the component: Possible value ranges from 50% to 100%. The default value is 100%.
- Attribute Key for the First Level (mandatory)
- Attribute Key for the 2nd to 10th Levels (optional)
When the checkbox 'Exclude Skipped tests from statistics' is selected, the passing rate is calculated based on the total number of test cases, excluding the skipped tests.
When the checkbox 'Exclude Skipped tests from statistics' is not selected, the passing rate is calculated based on the total number of test cases, including the skipped tests.
Widget view
The Component Health Check widget is multi-level (up to 10 level) with the ability to drill down to the list of test cases included in the corresponding group at each attribute key level.
‘ALL LAUNCHES’ option
For the first level, system applies chosen filter to all the launches in ReportPortal and analyzes the last 600 launches from the filter. After combining all the tests cases from these launches, the system searches for the test cases with the attribute key specified for the first widget level (e.g., attribute key ‘function’) and groups the found test cases around unique attribute values (order, team, configure, administrate). The system then calculates the passing rate for each group.
For the second level, the system again analyzes the 600 launches from the filter and searches for the test cases with 1st level attribute key plus value for chosen group (e.g., function:order) and also attribute key specified for the second widget level (e.g., type). The found tests cases are grouped around unique attribute values (backend, API, Unit, UI). The system again calculates the passing rate for each group.
The same flow is applied to the other levels of the widget.
‘LATEST LAUNCH’ option
For the first level, the flow is almost the same as for the ‘ALL LAUNCHES’ option. However, after the test cases are grouped around unique values for the attribute key, system only leaves the tests cases from latest launches executions for each selection.
For example, if you have Launch A with executions #1 and #2 and Launch B with executions #1 and #2 and they correspond to the applied filter, then building the widget based on “LATEST LAUNCHES’ parameter will take into account only the test cases from Launch A executions #2 and Launch B executions #2.
Widget section
The widget is divided into two sections: Passed and Failed.
The Failed section includes all groups (test cases with the same attribute) that have a passing rate lower than the passing rate specified in the widget wizard.
The Passed section includes all groups that have a passing rate higher than the rate specified in the widget wizard.
Each group on the widget has:
- name, which is equivalent to attribute value
- passing rate, calculated as (the number of passed test cases in the group)/(total number of test cases in the group)
- the number of test cases in the group
- a color line, which depends on the passing rate (see section Widget legend)
Users can drill down to view the list of test cases included in the group, filtered by:
- test method: Test
- status: Passed, Failed, Skipped, Interrupted
- attributes (key=Key for corresponding level, value=group name)
Each subsequent level should contain the attributes of previous levels.
When you click on the arrow on the component and the 'Exclude Skipped tests' toggle is OFF, the system redirects you to the test item view. Test items with statuses Passed, Failed, Interrupted, and Skipped are displayed.
When you click on the arrow on the component and the 'Exclude Skipped tests' toggle is ON, the system redirects you to the test item view. Test items with statuses Passed, Failed, and Interrupted are displayed.
Widget legend
The widget legend has two lines - Passed and Failed.
Failed
The failed line has four colors:
- light red
- regular red
- strong red
- dark red
It represents values that are less than the rate specified in the widget minus 1.
Passed
The passing line has only two colors:
- slightly green
- green
And have values - from specified on widget wizard to 100%. Depending on this color scheme each group on the widget has its own color.
Let's say we set 'The min allowable passing rate for the component' to be 90%. The tests cases will be grouped in following color groups:
green: tests with passing rate equals to 100%
slightly green: greater than or equal to 90% (specified on widget wizard) and less than 100%
light red: greater than or equal to 67,5% (calculated as (90%/4*3
)) and less than 90%
strong red: greater than or equal to 45% (calculated as (90%/4*2
)) and less than 67,5%
regular red: greater than or equal to 22,5% (calculated as (90%/4
)) and less than 45%
dark red: less than 22,5%
The widget doesn't contain 'IN PROGRESS" launches.
Use case
Situation:
As a Project Manager or Test Lead, I want to identify the most unstable areas in my product (application).
Solution:
Let’s build Component Health Check Widget based on a particular filter for the launches and the following attributes for test cases:
1st level: key: function, possible values: order, team, configure, administrate
2nd level: key: type, possible values: backend, API, Unit, UI
3d level: key: market state, possible values: open, close
4th level: key: role, possible values: Project Manager, Member, Admin
On the first level of the widget, the user will only see the test cases with attribute key ‘function’.
The test cases will be grouped by attribute value: order, team, configure, administrate.
By clicking on one of the groups (e.g., order), the user will see the second level of the widget.
It will contain only test cases that have attribute function: order and attribute key ‘type’.
Test cases will be grouped by attribute value available for attribute key ‘type’ which are: backend, API, Unit, UI.
1st LEVEL
2nd LEVEL