Nowadays, there is a lot of talk about quality gates — milestones between the stages of the project delivery cycle. Quality gates are essential for creating top-quality software. But they are often thought of as meetings where people just sit together and manually go through checklists.
In reality, a quality gate is any check that can stop product delivery if not passed. What checks can do that?
There are all kinds of quality gates out there
- An IDE warning/error. Even something that simple can prevent you from running your code — which means it's a quality gate.
- Style guides. Those are often recommendations rather than delivery-breaking rules - Python's style guide warns us that sometimes style guide recommendations just aren't applicable. And yet, it is possible (and recommended) to get a PEP 8 linter for your code editor.
- Code reviews. Having an experienced pair of eyes (or two) go over your code is always good - at the very least, it ensures that knowledge about code will be spread around the team so that one dev doesn't become a bottleneck.
- Pull requests. Whoever does the merging will review the changes, another check that can stop the process.
- Building. In a way, this is another quality gate - if a build fails, the whole process stops.
- And, of course, tests. Unit tests, integration tests, staging, and manual tests.
Many of these checks suffer from an inherent problem: humans are too smart. The dev, the tester, the manager - everyone's view of the system is different. If they think about QA at all, they've got different approaches. For instance:
When the developer writes an awesome solution for a problem, writing unit tests is a chore; usually, they just confirm that the awesome solution works, testing the happy path. They might ignore yellow warnings from an IDE because, quite frankly, they've got their hands full with the red ones.
Testers are the bedrock of QA, but they do make mistakes. For example, they might mute a flaky test and then forget about it, not realizing that the test has become permanently red.
Automation has an important advantage over humans: it is dumb. Which means it's reliable and transparent.
Automation will never replace humans completely, and manual testers aren't going anywhere. But automated tests provide checks that don't depend on our laziness, lack of attention span, or our brains being busy with more burning issues. As quality gates, automated tests with pipelines are among the most efficient ones.
The Advantages of Using Automated Tests and Pipelines
By placing automated tests into build and deployment pipelines, you get automated quality gates that provide fast and continuous feedback. You can specify whether a deployment fails when it doesn't pass through all the quality gates or just one of them, depending on how strict you want to be with your pipelines.
Due to the automated nature of the tests, once you have a successful pipeline run, you'll be confident that some of your headaches have been taken care of without manually tracking your progress. This is more efficient than manual quality gates and saves lots of time. You can add numerous gates into many pipelines until you've automated almost 100% of your testing life cycle.
Problem: integrating automated pieces
Automating pays off in the end, but it takes work. Don't get stuck at automating the individual pieces of your workflow; returns on your efforts increase once you integrate those pieces. Running your automated test suite on each pull request is one such example.
Writing tests is a lot of work and initially, only the automators can use them. Now, consider a simple workflow in GitHub that runs a test suite on each pull request and sends results to Allure Testops.
With GitHub, the system is more controllable, and it becomes possible to automate the checks. Finally, with Allure Testops, the results of those checks become much more readable so that everyone on the team can use them.
Each new step makes the system more usable and more accessible; more and more people become involved, and return on invested effort increases.
Problem: managing a menagerie
Which brings us to another difficulty. If you want proper automation, you have to manage a menagerie of programming languages, frameworks, issue trackers, CI/CD systems, etc.
We've calculated that just integrating testing will take two people per month - and we're talking about a rather specialized skill set, devs with a deep knowledge of testing and integrating different tools. It's not easy to find such people. Often, companies prefer to train these specialists. Either way, it's not cheap.
Problem: automated tests are dumb
As we've already established, automated checks are dumb. It means they're easier to conduct - but it also means that if you've moved a button 10 pixels to the left, a screenshot test for that button will fail. You might not want it to, but this is exactly what it is supposed to do.
Consequence: it takes patience
If you know about these issues, then you also understand that automation doesn't happen overnight. It's a process that takes time and effort, and if you want to know more about it, you can read our guide on integrating automated testing into DevOps. But despite the effort, automation is worth the expenditure. Humans hurry, get tired, and make mistakes. Code doesn't.
The Best Tools for Quality Gates
- Google Sheets and Microsoft Excel: For simple quality gate checklists, Google Sheets and Microsoft Excel are excellent.
- Trello: For automated updates between teams, use Trello to integrate messaging tools such as Slack.
- Atlassian JIRA: With Atlassian JIRA, you can put your quality gates into a dynamic dashboard to provide broader visibility for your team.
- Atlassian Confluence: You can create a shared space with checklists and documented quality gates for your team with Atlassian Confluence.
- GitHub: GitHub allows you to run whichever checks you wish on actions such as pull requests.
In Summary
Having automated tests as quality gates doesn't mean you need to get rid of all your manual tests. But you can automatically trigger manual test runs and rely on automated tests to give the go-ahead if everything is in order.
Automated tests don't require gathering meetings and going through checklists by hand. In other words, they are automated quality gates.
