Say you're a manager and you want to get statistics on test runs or an overview of tests covering a feature you're responsible for, or you want to launch a test run yourself. Typically, this involves digging into your CI/CD system, messing with the test suite you're running, or going to the people responsible for tests. Either way, lots of work. How do we avoid that?
One of the main ideas of Allure Testops is that it allows you to stay at your preferred level of abstraction, to have the information you need, and to stay away from details you don't.
This is because we recognize that the entire team should have a hand in quality assurance. Testers are the people with the know-how, but its application requires collaboration. Allure Testops is perfect for that since it provides a single space where different team members can access tests, always at their preferred level of abstraction. And under the hood, this is possible because Allure Testops has excellent integrations with all modern CI/CD systems.
So, let's look at the particular uses to which you can put our tool if you're a manager or an analyst.
Analytics
Imagine you've just had some test runs and need to get stats on them. If you'd try and get those stats from Jenkins, you'd have to dig into console output:
Somewhere in the middle of this, you'll find a line about passed and failed tests. Not exactly easy to read - for most people, anyway.
Testops is integrated with Jenkins and receives all this data after each test run, so let's take a look at how it's presented there:
And if you want more, you can click the name of the test run, and the system will show you more detailed statistics:
There, you'll also find test results for each test presented in a tree, an overview of errors, graphs, etc. And all the test results are stored long-term so that you can compare them historically. For that, Testops has a separate Dashboards view:
This overview screen will suffice if you need some basic info. But you can also create custom widgets to display whatever info you want:
Here, you can even specify the data you want with a query language we've developed, AQL. And again, if you don't need to get too technical, you don't have to dig into this - the option is hidden behind the scenes.
Allure Testops gets all this data from the CI servers it is integrated with, but it reduces the effort needed to sift through that data by several orders of magnitude. Also, you never need to go to your Jenkins or Gitlab.
This brings us to another advantage: cost. If you only want to look at the data, then all you need is a free Guest account in Allure Testops. You don't need to pay for a separate user, neither in Testops nor in Jenkins or any other CI server.
Test coverage
Let's say you're responsible for some feature of your product, you want to make sure it's working reliably, so you need to check how well it's covered with tests. Provided your tests have been properly labeled, this is easy to do.
First, you go to Test cases in the black menu on the left. Once you're there, click on the Test cases heading in the upper left corner, and you'll be able to choose a particular view for the test cases; these views can be set up in advance (here's how).
Once you're done with that, you'll see all your test cases sorted by the feature they're covering:
So if you're interested in a particular feature, here you'll see how well it's covered. Similarly, you can sort all the test results the same way - under Launches > Tree:
This is based on data Allure Testops receives from the CI systems it integrates with; however, managing this data is much more convenient in Testops. And you can actually do the test runs from the side of Testops - which brings us to the next point.
Executing tests
If you are a manager, developer, or admin, you probably don't want to be tinkering with a test suite directly, but you might want to run it. You could do it in your CI/CD system, but generally speaking, it's a hassle.
This is something that Testops can do without too much fuss - it provides a magic button for the entire team, something that is very important for setting up a proper DevOps flow. Moreover, you can select just a part of the suite to run and get back results in real-time - there's a step-by-step guide on that.
Oh, and both the automated test cases and their test results all look like they were manual tests from a TMS. This is something we call Smart Test Cases: when test results from a CI come in, they are automatically presented in a familiar form. In other words, you don't have to read code to figure out what's going on.
Automation as a service
As you can see, we've done a lot of work on delivering data from tests to you. The foundation for this is integrating with CI systems - we've covered all the most popular ones, and thanks to the technology we've used, the connection is stable and capable of delivering large volumes of test results at high speed. Those results are presented in a convenient form similar to the usual presentation of manual test cases. Thanks to the dashboards and widgets, you can derive statistical data from them. Finally, you can launch the tests yourself - the CI integrations allow you to control the whole thing from the side of Testops.
In all of this, we use the principle of automation as a service: you apply minimal effort and stay at precisely the level of abstraction you need, only digging into the technical details if you need to.
