Retail
Reducing regression time by 50%
EPAM helps a Canadian retail company to reverse-engineer their legacy IBM-based store management system to a modern tech stack. As part of this project, ReportPortal was deployed as a centralized test reporting system.
Challenges
Unavailable environments (15VMs) blocked by aggregation scripts
High risk of aggregation fail: 1 in 10 aggregations fails. In case of a fail, the whole regression should be re-run
Constant regression fails move weekly releases for one day
Lack of information for investigation: no screenshots/no history/no structure/no all info
Duplicated analysis efforts: missing history of test cases and known issues
Highlights
Simplified test run reporting by integrating the test framework with ReportPortal
Distributed test execution data for root cause analysis: logs/screenshots/ attachments
Provided a possibility to triage failed items (AI-based and manual)
Provided clear reporting for non-technical stakeholders
Real-time reporting
Save on early reaction: team result analysis right after execution started in real time
Collaborative results analysis
Test Case History helped to identify flaky test cases
Extended Machine Learning Analyzer
Benefits & Results
Test results analysis time reduced
Before | After |
|---|---|
12 hours | 4,2 hours |
Release reporting accelerated
Before | After |
|---|---|
2 hours | 15 min |
Regression time reduced
Before | After |
|---|---|
24 hours | 12 hours |
Regression scope increased
Before | After |
|---|---|
10K tests | 15K tests |
Team size reduced
Before | After |
|---|---|
50 people | 40 people |
Automation stability improved
Before | After |
|---|---|
85% | 96,5% |