This post is the translated version of an _original blog post_ by Anton, Lead Test Engineer at Ozon, on the process of Allure Go integration development. We thank our dear colleagues for the opportunity to share it with our audience in this blog.
The Ozon team has been writing tests mostly in Python for quite a long time. Go is a relatively new language, and there are not many established testing tools yet. Python has pytest, Java has JUnit and TestNG, while Go lacks a standard testing tool.
One day, having once again rewritten a group of old Python tests, I decided that the time for changes had come. That thought eventually led me to create our Open Source library. It would support Allure Report (and TestOps) without UI overload, as well as have the infrastructure for storing tests both in the same repository as the services and in separate ones, reports in Slack, and other stuff.
This post covers two things: the reasons for creating a new solution, as well as QA migration to Go, and the process of integration with Allure tools.
Go in Ozon
Most of the services in Ozon are written in Go, and their number has been growing over time. Here's the profit from using Go in services:
- Infrastructure: relatively lightweight Docker, debugged jobs in GitLab CI to run integration and unit tests, lintering, and code reviews.
- A batch system on Git repositories: no need to worry about storing and importing auxiliary libraries; a convenient versioning system based on Git tags.
- A relatively low threshold of entry: you can hire people and train them up to the right level in Go - fairly easily and almost painlessly.
- All of this is true for E2E testing: the opportunity to easily deploy supporting services and build libraries and services using developer tools.
But we didn't use this potential to the fullest extent: the code was written in Go while the tests were in Python. This is kind of a typical scenario for E2E testing, given the lack of Go testing tools. But the Ozon team aimed to build better cooperation between QA and developers, involving the latter in the testing tools support and development. Switching the testing to Go automatically led the team to this goal.
First step: Allure Tools support for Go
Having that in mind, we started the research for a Go-based alternative to the Python backend testing solution. The first step was to formulate the requirements for an alternative toolkit:
Tests should be easy and fast to write (and thus be inexpensive to support).
The tools should be simple and easy to understand for people from different teams with different backgrounds.
One of the most important things was support for Allure, as Ozon actively uses it to build statistics and analytics on testing.
Having studied the experience of colleagues and conducted research, I found the following:
There was no official Allure solution for Go at the time;
Providers did exist (e.g. dailymotion/allure-go), but I strongly disliked their interface, or they hadn't been updated in a while, as if they were no longer supported (GabbyyLS/allure-go-common);
There weren't many testing frameworks, the most promising looked like Testify.
Summing that up, we decided to build our data provider that would be able to:
Fetch Allure reports;
Compose tests;
Run tests.
We hoped to implement test composition and run with Testify, but it turned out Testify doesn't support plugins. Because of that, we made our implementation from scratch. And so our Allure Go project was born. Let's take a look at how it works at Ozon.
How does the Allure-Go adaptor work?
First, just as Testify, the library allows organizing tests into test structure classes. It makes the test code clearer and helps to explicitly allocate test suites and use Before and After hooks. At the same time, it is still possible to write tests as functions (also using these hooks), which is a more Go-style approach.
Here is an example of a test structure:
type SuiteStruct struct {
suite.Suite
}
func (s *SuiteStruct) Test1() {
}
func (s *SuiteStruct) Test2() {
}
func TestRun(t *testing.T) {
runner.RunSuite(t, new(SuiteStruct))
}
And an example of a test as a function:
func TestSampleDemo(t *testing.T) {
r := runner.NewTestRunner(realT)
r.WithBeforeEach(func(t *provider.T) {})
r.WithAfterEach(func(t *provider.T) {})
r.Run("My test 1", func(t *provider.T) {})
r.Run("My test 2", func(t *provider.T) {})
r.Run("My test 3", func(t *provider.T) {})
}
In addition, Allure-Go provides various options for working with steps and Allure labels. We tried to make the interface as concise and user-friendly as possible, so we fetch it with provider.T and suite.Suite wrappers.
An example of a simple test with nested steps:
type StepTreeDemoSuite struct {
suite.Suite
}
func (s *StepTreeDemoSuite) TestInnerSteps() {
s.Epic("Demo")
s.Feature("Inner Steps")
s.Title("Simple Nesting")
s.Description(`
Step A is parent step for Step B and Step C
Call order will be saved in allure report
A -> (B, C)`)
s.Tags("Steps", "Nesting")
s.WithNewStep("Step A", func() {
s.NewStep("Step B")
s.NewStep("Step C")
})
}
And this is what Allure Report for such a test looks like.
The other thing we have been focusing on while developing the adaptor was keeping the syntax concise and easy to read and maintain. Take a look at the sample test written in dailymotion implementation:
func TestNewTest(t *testing.T) {
allure.Test(
t,
allure.Description("New Test Description"),
allure.Action(func() {
allure.Step(
allure.Description("Step description"),
allure.Action(func() {
}))
}))
}
and ours:
type SuiteStruct struct {
suite.Suite
}
func (s *SuiteStruct) TestNewTest() {
s.Description("New Test Description")
s.WithNewStep("Step description", func() {
})
}
Allure Go benefits so far
The main result, in my opinion, is this: we made another step toward a reality where Dev and QA teams speak the same language. Developers are helping with testing support and development, testers are getting a deeper dive into the codebase.
The library lived through three refactorings, changed two repositories, and keeps actively evolving. The reporting system provides graphs and statistics for more than 40 services.
Based on positive feedback, our library has been added to the convention for Go-tests as recommended for use. We had it in production for half a year, and so far it has zero problems with filling reports and a minimum entry threshold. As a consequence, colleagues became interested in the solution - to date, more than 30 teams have become involved and moved to Ozon.
Right now, Allure-Go does the following:
It integrates Allure into Go-tests (supports steps and attachments, offers a concise interface)
It builds tests into suites
It runs tests in Go
Here are our plans for future development:
release the library to the official Allure repository (Qameta Software);
add new features:
- asynchronous stacks via goroutine startup design
- wrapping asserts into steps (based on Testify asserts)
- swiping interfaces to simplify extensions
- …and others
