Back to blog
Nowadays, Quality Gates is a significant stage in the project delivery cycle. These checkpoints play a vital role in the development of high-quality software.
What is Quality Gate?
This is a set of predefined criteria that must be met before a software product can progress through the development lifecycle.
ReportPortal offers Quality Gates as a premium feature that accelerates the CI/CD pipeline by providing quick and continuous feedback to your CI/CD tool. There are five rules to quickly assess the build quality and understand if it can be deployed to a specific environment.
Below, you will find some tips to get the most from Quality Gates.
1. Create separate Quality Gates for different launches
For example, your project may have various automated tests such as Regression Suite, Smoke Tests, and UI Tests. And we can create different rules for these types of tests.
We also need to determine under what conditions the Quality Gate will be considered as Passed. It's recommended that these aspects should be discussed among several interested team members to define quality criteria more accurately. For instance, a team might decide that they agree with a failure rate of 1% if there are at least 2200 tests.
2. Gradually increase the number of rules
We suggest starting with the simplest rule, the "Amount of tests in the run", to track that all tests are running every time. Understand how it all operates before adding a new rule.
Further, you can enhance accuracy by selecting the "Tests with attributes" option. For instance, the following Quality Gate with the "Amount of issues in the run" rule will be Passed for the "dev" environment even if there is one Product Bug with the attribute "dev". However, for the "stage" environment, the Quality Gate with such a rule will be categorized as Failed.
3. Keep track of the relevance of rules
No project is static: new test cases are created, bugs are detected, some test cases are excluded from the run, and Quality Gates should be reviewed in line with requirements.
For example, in the screenshot below Quality Gate failed because the number of tests in the launch has changed, but the Quality Gate rule was not updated.
4. Make the Quality Gate a stage of the CI/CD pipeline
By using Quality Gates in the CI/CD pipeline, the decision to move to the next stage can be made automatically within seconds. For instance, if the tests are successful and the Quality Gate check is Passed, the build is deployed to the environment. This flow prevents the progression of the code if it doesn't meet the necessary criteria. Thus, there is no need to manually go through and analyze the test results – if they meet the Quality Gates, the next step in CI/CD will run automatically.
5. Use Quality Gates to construct test automation metrics
Just like our widgets, Quality Gate rules can also assist in developing metrics to understand the effectiveness of automation efforts.
Pass/Fail rates
Error/failure distribution
Regression test effectiveness
Defect density
Amount of issues in tests with specified attributes
Test case success rate
Failure rate of the run – to get test case success rate, subtract the percentage of failed tests from 100.
By following best practices, you can automate quality control and ensure that only high-quality code progresses, reducing bugs and improving software stability for users. Overall, Quality Gates provide a strategic and robust development stage in the software delivery cycle.