If you get into automated testing, one of the first things you'll learn is that it's best to run tests on a remote server - this way, everyone in the team can do a test run without setting up the tests locally and then worrying if their local environment has influenced the test results. When choosing the server, the first option you'll hear about will probably be Jenkins.
Jenkins is an open-source CI/CD server with 20 years of history, a vast community, and probably a thousand developers writing code for it full-time. Naturally, Allure Testops has an integration for Jenkins. Let's talk about the strengths of Jenkins and how Allure Testops can compensate for its weaknesses.
Community
Jenkins and Allure Report follow a similar model: they are both very extensible open-source tools with a solid and helpful community (and an enterprise solution based on that tool - with Jenkins, it's CloudBees CI). Jenkins is easy to get started with, and there is a huge knowledge base and documentation available.
Flexibility
A key strength of Jenkins is that it can do pretty much anything. Of course, some uses are more popular than others - e.g., deploying into production, running automated tests, or driving integration. But generally speaking, anything you can run in the command line of your build agent can be run through Jenkins.
The architecture of Jenkins is based on plugins, and its community has been writing them for over a decade, so today they boast more than 1900. To visually appreciate this number, we can take a look at the "jenkinsgraph" tool that displays dependencies for plugins. First, let's filter everything by "allure":
Now, let's go bigger: show all plugins with the word "test" in their names.
Finally, just show everything:
Doesn't it kinda remind you one of those videos where they show you the relative sizes of objects in the universe for you to better appreciate its awe and majesty?
The point is that pretty much anything you can think of can integrate with Jenkins. If not - well, you can always write the plugin yourself (as we did for Allure Report), or you can set up command line execution.
Downsides
This flexibility comes at a cost. Universal toolboxes that can do anything usually have steep learning curves and aren't easy to use. To be fair, Jenkins is easy to install, but the amount of options and configurations can be overwhelming. In addition, people complain a lot about Jenkins's UI.
This isn't a problem if you've spent much time with the system and know it in and out. But not everyone who wants to run a suite of tests might have this knowledge. If you want automated tests to be a tool that the entire team can use, hiding that complexity could go a long way toward achieving that goal. This is what Allure Testops does: it allows people to use Jenkins without using Jenkins.
Allure Testops
As with any Allure Testops CI integration, the Jenkins integration allows you to do a full cycle of operations you need to run tests:
- retrieve pipelines available for the current user
- retrieve the name and parameters of a pipelines
- initiate the execution of a pipeline from Allure Testops's side
- monitor the execution of a pipeline live and receive the individual test results without waiting for the entire test run to finish
And all of this is done without ever logging in to Jenkins. Of course, somebody will need to set up the integration and the pipeline, but once that's done, everything can be controlled from Testops's side.
Setting up the integration
We've already got a detailed how-to on setting up the integration with Jenkins, so I'll just give you the highlights here. You've got two options: a plugin or allurectl. Whichever you choose, you'll need to:
- authenticate Allure Testops for Jenkins with a token
- add your Allure server instance information to the Jenkins configuration
- configure a build job in Jenkins (whether freestyle or pipeline) and add a command to execute the tests
- set up environment variables on both sides
This would be enough to receive results from Jenkins; to initiate the execution of Jenikns pipelines from Testops, you'll need to:
- add the required integration in Allure TestOps globally (you'll need admin privileges for that)
- add the required integration on the project level and provide credentials
Conclusion
Allure Testops and Jenkins work together very nicely. Jenkins is incredibly versatile, popular, and has excellent community support. Probably its biggest downside is that some people aren't willing to fiddle with it - but with Allure Testops, they don't have to! All the complexity can be dealt with by lads and lasses with deep knowledge of the system. They will set up a workflow that everyone else can run with a few mouse clicks from Testops.
