Skip to main content

Component health check

The widget shows the passing rate of the application components, indicated by the attributes specified for test cases.

note

For using this widget you need to report (or add manually) attributes to test items.

Widget's parameters:

  • Filter
  • Parameters: All launches/ Latest launches
  • The min allowable passing rate for the component: Possible values from 50 - 100%. The default value is 100%.
  • Attribute key for the first level (mandatory)
  • Attribute key for the 2nd – 10th levels (optional)

Use case:

Situation: As a Project Manager or Test Lead, I want to identify the most unstable areas in my product (application).

Solution: Let’s build Component Health Check Widget based on a particular filter for the launches and the following attributes for test cases:

1st level: key: function, possible values: order, team, configure, administrate
2nd level: key: type, possible values: backend, API, Unit, UI
3d level: key: market state, possible values: open, close
4th level: key: role, possible values: Project Manager, Member, Admin

On the first level of the widget, the user will only see the test cases with attribute key ‘function’. The test cases will be grouped by attribute value: order, team, configure, administrate. By clicking on one of the groups (e.g., order), the user will see the second level of the widget. It will contain only test cases that have attribute function:order and attribute key ‘type’. Test cases will be grouped by attribute value available for attribute key ‘type’ which are: backend, API, Unit, UI.

1st LEVEL

2nd LEVEL

Widget view

The Component Health Check widget is multi-level (up to 10 level) with the ability to drill down to the list of test cases included in the corresponding group at each attribute key level.

‘ALL LAUNCHES’ option

For the first level, system applies chosen filter to all the launches in ReportPortal and analyzes the last 600 launches from the filter. After combining all the tests cases from these launches, the system searches for the test cases with the attribute key specified for the first widget level (e.g., attribute key ‘function’) and groups the found test cases around unique attribute values (order, team, configure, administrate). The system then calculates the passing rate for each group.

For the second level, the system again analyzes the 600 launches from the filter and searches for the test cases with 1st level attribute key plus value for chosen group (e.g., function:order) and also attribute key specified for the second widget level (e.g.,. type). The found tests cases are grouped around unique attribute values (backend, API, Unit, UI). The system again calculates the passing rate for each group.

The same flow is applied to the other levels of the widget.

‘LATEST LAUNCH’ option

For the first level, the flow is almost the same as for the ‘ALL LANCHES’ option. However, after the test cases are grouped around unique values for the attribute key, system only leaves the tests cases from latest launches executions for each selection. For example, if you have Launch A with executions #1 and #2 and Launch B with executions #1 and #2 and they correspond to the applied filter, then building the widget based on “LATEST LAUNCHES’ parameter will take into account only the test cases from Launch A executions #2 and Launch B executions #2.

Widget section

The widget is divided into two sections: Passed and Failed.

The Failed section includes all groups (test cases with the same attribute) that have a passing rate lower than the passing rate specified in the widget wizard.

The Passed section includes all groups that have a passing rate higher than the rate specified in the widget wizard.

Each group on the widget has:

  • name, which is equivalent to attribute value;
  • passing rate, calculated as (the number of passed test cases in the group)/(total number of test cases in the group);
  • the number of test cases in the group;
  • a color line, which depends on the passing rate (see section Widget legend)

Users can drill down to view the list of test cases included in the group, filtered by:

  • test method: Test
  • status: Passed, Failed, Skipped, Interrupted,
  • attributes (key=Key for corresponding level, value=group name)
note

Each subsequent level should contain the attributes of previous levels.

Widget legend consists of two lines: Passed and Failed

Failed

The failed line has four colors:

  • light red
  • regular red
  • strong red
  • dark red

It represents values that are less than the rate specified in the widget minus 1.

Passed

The passing line has only two colors:

  • slightly green
  • green

And have values - from specified on widget wizard to 100%. Depending on this color scheme each group on the widget has its own color.

Let's say we set 'The min allowable passing rate for the component' to be 90%.

passed green: groups which have passing rate 100%.
slightly green: groups which passing rate from 99 - specified on widget wizard.
light red: from 3* (90% - 1)/4 to (90% - 1)
strong red: from (90% - 1)/2 to 3* (90% - 1)/4
regular red: from (90% - 1)/4 to 2*(90% - 1)/4
dark red: 0 - ((90% - 1)/4 -1)

note

The widget doesn't contain 'IN PROGRESS" launches.