We've just refactored the way we're organizing Python tests for Allure Report and Allure Testops. You probably won't be running any of this code yourself - but if our tests are easier to maintain, we can deliver a higher-quality product to you. Also, the new structure is just plain neat! If you'd like a thorough explanation of how everything works right now - take a look at the GitHub page of the update; here, we'll give you a quick overview.
How it was before
Before the refactoring, the tests for each framework were written using the framework itself and were self-contained. This meant that code was hard to reuse, tests were diverse, and every package had to be run separately. If you wanted to check for something, you'd have to implement that check separately for each of the frameworks. Another thing is that writing unit tests for Robot Framework and Behave Framework can be a pain, and you are often forced to write more cumbersome end-to-end tests instead. Finally, under the old system, the examples for frameworks were somewhat messy and hidden inside the packages.
How it works now
Now, all the tests are written with PyTest, and they have been moved into a separate 'tests' folder. All tests can now be run with a single command, and their code is easier to reuse.
The new test structure looks like this:
tests
├── allure_behave
│ ├── acceptance
│ │ ├── allure_api
│ │ └── behave_support
│ ├── defects
│ ├── behave_runner.py
│ └── conftest.py
├── allure_nose2
│ ├── acceptance
│ │ ├── allure_api
│ │ └── nose2_support
│ ├── conftest.py
│ └── nose2_runner.py
├── allure_pytest
│ ├── acceptance
│ ├── externals
│ ├── conftest.py
│ └── pytest_runner.py
├── allure_pytest_bdd
│ ├── acceptance
│ └── conftest.py
├── allure_robotframework
│ ├── acceptance
│ │ ├── allure_api
│ │ └── robotframework_support
│ ├── conftest.py
│ └── robot_runner.py
├── conftest.py
└── e2e.py
Here, general stuff for end-to-end tests resides on the first level, in the e2e .py-file. The framework-specific folders can have an "acceptance" folder with end-to-end tests for a specific framework, a "defects" folder for bug reproduction tests, and an "externals" folder for testing compatibility with 3d party packages.
Also, the examples for different frameworks were rewritten completely as '.rst' files with code blocks inside. Each code block has an ID inside, and a test can invoke that code using that ID so that each example can be covered with a test.
There were plenty of small improvements as well - take a look at the GitHub page for more.
Why this is good for everyone
The update has specific fixes that affect user experience directly - for instance, "--clear-alluredir" now properly removes everything from the allure directory. But there is a more general benefit, too.
A homogenous testing environment means that QA can write more tests with fewer mistakes. Replacing some e2e tests with unit tests means less time spent writing and running the tests. In the end, we're able to fix bugs and roll out new features faster. And if a feature or a fix takes less time to implement, it means we will get to it sooner - cost calculations are always part of deciding what to do next.
