This is a story based on an interview Ruslan Akhmetzianov had with Eugene Sabirov, QA Chapter Lead at Tochka Bank. We discussed the challenges of a rapidly growing IT company and how they ensured software quality with Allure TestOps.
About Tochka
Tochka Bank is a business bank for individual entrepreneurs and — essentially — an IT company.
Tochka Bank has a large number of employees, including circa. 600 engineers and uses a microservice architecture. The biggest characteristic of Tochka Bank in terms of the organization of the development processes is that each dev team is cross-functional. Each dev team is responsible for its part of the single service or product and can use any technologies that it deems most suitable for its tasks. Consequently, at Tochka, you can find completely different technology stacks that use different development and testing processes.
On average, each team has ten IT specialists. Due to the team sizes and their freedom in decision-making, a dedicated testing department is not required as there are also testers in almost every team, and all the test management-related decisions are independent.
Difficulties of testing
The main objective for implementing a Test Management System (TMS) was to simplify the entry process into test case writing and documentation maintenance. Many responsible testers who were independently involved in compiling test cases and running the regression according to the test plan had no problems with documentation. They were using Google Sheets, Cucumber, or any other tool of their choice to work with test documentation. But over time, problems inevitably began to arise for less mature teams, for example, if an individual took annual leave and did not transfer access to their test cases to their colleagues.
To solve this problem, the bus factor needed to be increased. With the use of a centralized TMS, the occurrence of such situations should be minimized as all test cases would be maintained in one place and all the team members would have access to them.
It is important to note here that our main goal is not to centralize the testing process but to organize it and provide a tool that could be conveniently used out of the box to ensure that teams did not have to spend time and resources on organizing the process themselves.
Another important aspect of the test process relates to mobile applications testing. For the microservice, backend or frontend, everything is quite simple - each team releases the product or the feature when it is ready, but everything is different when it comes to mobile apps as these are released centrally for all teams.
There is also the problem of regression testing of mobile applications. Automated tests for these should be written by different teams, using common tools and standards, so that the release manager can run all tests with one button before the release, instead of collecting all the test result information from different teams.
It is these challenges that the use of a TMS would help to solve.
TMS selection
To solve these identified problems, it was decided to implement a centralized tool that would allow for standardized reporting, help maintain test documentation, organize the testing process, and ease the test cases transfer between teams. Initially, a different TMS was used but this only solved part of these problems, and after a year and a half of using this system it still wasn’t incorporated fully into the processes; the engineers simply did not use it.
Tochka Bank incorporates the ideas of holacracy. Each dev team is independent and allowed to use any tools they see fit, so even though an TMS system had been implemented the automation testers in the mobile dev team suggested trying Allure TestOps and this was taken on board.
The Allure TestOps trial period was approached as an experiment and by the end of it all participants concluded that the Allure TestOps solution was more comfortable to work with. For Tochka Bank is important that their employees feel comfortable working with the systems so it was decided to switch to Allure TestOps.
A key factor for choosing Allure TestOps, as indicated by the participants, was its reporting system. The team that requested the use of a new tool already used automated tests annotated with Allure Report, supported by TestOps out of the box.
Other important advantages of Allure TestOps included the out-of-the-box integrations for various test frameworks and the ability to store unit tests in it.
After considering all of these advantages, Tochka Bank purchased Allure TestOp licenses for six months and started implementing the system.
Migration
The company-wide migration process was rolled out without any major pitfalls.
The transition was a little stiff for some teams with a large test case base stored in other tools as they had to write scripts to transfer test cases from one system to another, but this was also a good reason to update the test cases. In reality, some projects didn’t need to be transferred at all, while others were so outdated that it was easier to rewrite them from the top and, as a result, eliminated legacy code that has benefited everyone.
It’s also important to note that there was no automated testing infrastructure tied to the previously used TMS. Therefore, no problems were encountered in this area during the migration.
The integration of existing automated tests was quite simple due to the fact that most of these had already been written with Allure Report annotations.
After six months of using the system, there are now 5,500 test cases, of which 3,700 are automated tests and 1,800 are manual.
What has improved
- Over 70% of testers now use Allure TestOps to write test cases and to manage test runs.
- Transparency of testing processes has increased for all teams.
- No need to spend time negotiating who tests what since Allure TestOps by design has projects that are broken down by functionality.
- Teams write more automated tests due to the convenient reporting system and the ability to combine auto- and manual testing in one launch.
- Transparency of the release process for mobile applications has increased for release managers. They can now see how the regression is going in one launch:
- whether automated tests have passed
- percentage of manual tests completed
- the success rate of all the tests
- etc.
Сonsequently, everybody wins:
- Testers: life has become easier
- Developers: independently look at the results of testing functionality
- Release Managers: receive up-to-date documentation and reports on the results of the runs.
