Back to blog
If you’re curious about ReportPortal and want to jump in, or if you're already using it and want to make sure you’re getting the most out of every feature – this user guide is here to help! Welcome to the Handbook that provides an overview of the complete user journey within ReportPortal.
Step 1: To begin using ReportPortal, you first need to set it up. There are two options:
Use the Demo instance to explore ReportPortal’s features without any installation efforts.
Deploy your own instance via Kubernetes, Docker, or Google Cloud Marketplace. Be sure to review system's technical requirements. And if you'd prefer not to deal with the infrastructure and other technical stuff, our team is happy to help you.
Step 2: Once your instance is ready, add your team members. Configure the E-mail server under your Admin account to invite them by email.
Step 3: To fully experience ReportPortal’s features, you need to report test data. You can do this in three ways:
Generate demo data: populate your test automation results dashboard with sample data to explore the ReportPortal’s capabilities.
Report real automation results: connect your test framework to send test results to ReportPortal. This enables AI-based defect triage and advanced analytics for you. When reporting tests, stick to the structure launch – suit – test – step, add attributes and use nested steps (Java, Python, JavaScript, .NET) to make error logs more readable. Real time reporting ensures continuous updates and seamless monitoring of test progress.
Import test results: use JUnit or RobotFramework plugins for importing via the UI, or develop a custom import plugin tailored to your specific needs. Additionally, test results can be imported via the API.
Step 4: Configure a rule for notifications to receive updates via E-mail or Slack once the launch is complete.
Once your test results appear on the Launches page in ReportPortal, you may have an extensive list of test executions. Create filters to quickly access specific launches. Using attributes will provide additional filtering options.
To ensure you don't encounter issues with storage exceeding limits due to massive test automation results reporting, make sure to define a launch retention policy tailored to your specific needs.
Step 5: It’s now time for defect triage. Issues flagged as "To Investigate" need your review. Start triaging by examining each case, identifying failure causes, and assigning a defect type in the "Make Decision" modal. All logs, stack traces, attachments, history of actions, and Sauce Labs integration will help you in this process. You can also use ML suggestions, which recommend similar tests and their associated defect types.
Additionally, while in the "Make Decision" modal, you can post issues to your bug tracking system (BTS) and link them to existing ones with just a few clicks using BTS plugins.
Step 6: Pattern Analysis can help expedite test failure analysis. Create rules, and after a launch completes, the system will identify all failed items that match known patterns in the error logs.
Step 7: Run a Unique Error Analysis to view a consolidated list of all unique errors in the launch, along with the number of tests associated with each error. After that, you can get ML suggestions for the group of tests.
Step 8: After manually analyzing the results of the first run, you can take full advantage of Auto-Analysis, which performs automated defect triaging after each launch. Our Analyzer minimizes manual effort by automatically assigning defect types based on your previous decisions and linking issues in the BTS, if available.
Step 9: If your launch contains many tests, you can use Immediate Auto-Analysis and Immediate Pattern Analysis without waiting for the entire launch to finish.
Step 10: Now, you can create dashboards and various widgets to track trends and QA and test automation metrics. For example, you can assess the Passed/Failed ratio using the Passing rate summary widget, view the distribution of failure reasons with the Launch statistics chart widget, and identify high-risk areas with the Component health check widget.
Step 11: Integrate ReportPortal with your CI/CD tool to receive real-time feedback on test outcomes, ensuring that only stable builds progress in the pipeline. Quality Gates help maintain consistent standards and reliability. This automated check reduces manual oversight, accelerates CI/CD, and prevents unstable builds from progressing.
With ReportPortal, managing test automation and enhancing your QA process has never been easier. Ready to transform your testing process? Make the most of everything ReportPortal has to offer!