The variety of modern testing frameworks, libraries, etc., can be overwhelming. Setting everything up takes time, and if you want to try multiple solutions to make an informed decision, you might not have that much time. Don't worry, though - there is a solution.
A menagerie of tools
There was a time more than a decade ago when a testing framework was a rare beast: JUnit appeared in 2002, and for a while, that was pretty much it. Along with Extreme Programming and TDD, it did a lot to help people realize the importance of tight delivery cycles and automated testing.
Because of how good it was, people started making ports of JUnit left and right: CppUnit for C++, NUnit for .NET, and many more. Every language developed its own framework.
After a while, problems with those first frameworks became apparent. TestNG was created to overcome the shortcomings of JUnit. Other instruments appeared, like Cucumber.
During that time, build tools were also rapidly evolving. Maven's first appearance is also in 2002, while Gradle was first released in 2008.
As a result of all this development, we now have a pretty impressive menagerie of tools.
Why it's bad for you
This menagerie presents a problem. Think of how much variation you can have in your stack. There's the programming language (which can have different versions, often not backward-compatible), the build tool, the test framework, and the libraries used for testing (WebDriver, Playwright, or whatever).
Additionally, writing in-depth documentation takes time, and many tools might have very little of it. They might also not have support in IDEs. All of this means that switching technologies is both
- something you have to think about often
- very time-consuming
Here are some examples:
Suppose someone on your team or your user comes to you with a problem. In their particular setup, pytest version x.y.z doesn't work with stuff that functions perfectly well in another setting. Is it just them? To understand that, you'd need to replicate their stack. Even if you've already worked under that exact setup, it will take you a few hours to refresh all the dependencies and ensure everything works. And what if you have to set up everything from scratch?
Suppose you want to test some frameworks and decide which one to adopt. Maybe you're working with Selenide and wish to try Playwright or Cypress. Maybe you want to know what is available for Python or figure out how to write API tests in Ruby. That's natural - the number of frameworks keeps growing, so we all want to keep up with the technology and adopt the good stuff. As things are today, you'll have to spend two weeks fully setting up Cypress, then two weeks with Playwright, and only then will you learn which is best.
Suppose you're sitting on a really old version of Cucumber and want to know how difficult it will be to upgrade to the latest version. How will you know without upgrading?
Suppose you're new to the industry; having just learned to write selectors, you're eager to finish it. Not so fast, buddy! First, you'll spend several frustrating days setting up the IDE and all the dependencies.
How to beat it
To avoid all of this pain, we've created Allure Start. It assembles your tech stack for you!
Here's how it works. Go to https://allurereport.org/start/ and choose a selection of tools that you need for your project. The system then generates an archive with everything you need, including documentation:
If you generate a project for Cucumber with Java, this is what you'll get in your file:
.
├── .mvn
│   └── wrapper
│       ├── maven-wrapper.jar
│       └── maven-wrapper.properties
├── src
│   └── test
│       ├── java
│       │   └── org
│       │       └── example
│       │           └── cucumber
│       │               ├── steps
│       │               │   ├── RestSteps.java
│       │               │   └── WebSteps.java
│       │               └── CucumberTest.java
│       └── resources
│           ├── features
│           │   ├── labels-rest.feature
│           │   ├── labels-web.feature
│           │   └── milestone-web.feature
│           └── allure.properties
├── mvnw
├── mvnw.cmd
└── pom.xml
Soon, this will also include further instructions on developing your project. The stack of tools you get is guaranteed to work: we verify it on a CI before handing it off.
The folder you get from the archive can be opened in your favorite IDE as a new project. Actually, you can think of the whole thing as the "New project" action in IDE; it's just way more configurable.
A similar solution exists for Spring called Spring Initializr. What we're building will do the same thing - but for testing. Basically, Allure Start is a playground where every combination is a few clicks away, and you can experiment to your heart's content.
The advantages of being polyglot
Think again about how many different permutations your tech stack can have - and yet, one position doesn't have to change: the reporter. Allure Report was built to handle this great variety of tools.
No matter what language you're using or what test framework, the representation in Report will be the same: steps with natural-language descriptions, attachments, and stack traces for errors.
A list of everything Report integrates with is available here. The test framework page alone has 35 entries, and before, setting up an integration with any of those on your system would take some elbow grease - fiddling with settings and such. But now, Allure Start takes care of everything for you, and you get everything pre-made.
Conclusion
The modern testing tech stack is incredibly variable, and you can find all kinds of permutations out there. That limits the number of people in the community who have worked on a particular permutation, makes problem-solving more difficult, and complicates experimenting with new tools.
Allure Start helps you try out different combinations of tools error-free and quickly, with just a few clicks. That means:
- Less time spent on decisions to adopt or upgrade technologies
- Way quicker diagnosing of technical problems on uncommon tech stacks
- Lower barrier of entry for people who are just beginning to learn test automation
Try it and tell us what you think!
