“All the results are updated automatically, and you can see all the cases that were created for this particular task. It was exactly what we needed to understand what we check and how we're doing it. We significantly boost our good old transparency — and ensure a better understanding of the quality of the features we release.”
Mikhail SidelnikovHead of QA at AdTech Holding
The AdTech team has come a long way: in the beginning, they didn't even have a QA Department, it was only organized in 2015. As the company grew, so did the workload, and with more microservices it became increasingly difficult to keep everything organized and transparent.
The founders of the QA team were all familiar with TestRail, so in the beginning, it was the obvious solution for them. It did cover the most basic needs (such as case history and an acceptable Jira integration). However, as the project grew, issues began to emerge. The volume of test runs and the number of releases were scaling quickly, and TestRail wasn't too good under this pressure.
The most glaring problems were:
The team compiled a list of requirements for a new solution:
The team stumbled upon Allure TestOps. They knew it was based on Allure Report, a tool very familiar to most team members; they knew it was fresh and trendy - and they decided to give it a go. The change was smooth and paid off greatly. So what was it about TestOps that made it so attractive for the AdTech team?
With TestRail, downloading test results means a lot of fuss with the API. With TestOps, thanks to its many integrations, it's much easier. If you are generating an Allure Report, then the results are already where TestOps needs them. If you're getting the results from another tool like TeamCity, TestOps has plenty of integrations. They are elegant and simple to set up:
And once it's done, that's it, the results arrive in TestOps automatically. You don't need to mess with JS, web servers, and API.
It turned out that Allure TestOps has a Jira plugin that allows you to see all tests for a particular task. The plugin includes pagination, search filters, and the latest results of these tests.
The settings are also quick and simple — as well as the interface:
If you've got a large test base keeping it organized is a problem. The AdTech team used annotations in the code to label the tests and keep track of them. Plenty of annotations are provided by default:
And you can also specify custom ones: @LabelAnnotation(name = “any name”). Using the data provided by annotations, you can sort all your tests inside TestOps with the help of e. g. tree maps.
Other important features were of great help:
