Allure-csharp out-of-the-box: a work in progress

A basic principle for Allure Report integrations is that they should work out of the box. You install it, and the reports generate without adding anything to the code of your tests. If Report is using some internal feature of the framework for which the integration is written, it should do so without any additional hints from you. Any additional calls to Report's API are a nice bonus that you can choose to have, not a precondition. This is how it usually works because the architecture of Report allows writing powerful and seamless integrations very quickly.

Unfortunately, for historical reasons the allure-csharp integrations (specifically, NUnit and xUnit) don't adhere to this principle. This is exactly what two brilliant fellows, Maksim Stepanov and Nikolai Laptev, are fixing right now. We're talking about a work-in-progress, so you'd have to wait a few months for the benefits; but it's a good opportunity to see how our team keeps up the quality of Report's code.

First problem. Attributes

Right now, both in xUnit and in NUnit, if you install the package and run your tests, the report you'd generate would be empty. In Nunit, for anything to happen you'd need to add a special attribute to all test classes - AllureNUnit.

In xUnit, it's even worse. You'd need to replace all Fact and Theory attributes with AllureXunit and AllureXunitTheory. And not for every class - for every function. This is something you'd have to do by hand, which of course is time-consuming and you'd probably miss a function or two.

What's more, these Allure attributes can't be combined with attributes from other libraries. If some library needs you to replace the Fact attribute with something else, you need to choose between that library and Allure Report, since the latter has its own attribute it needs you to use instead of Fact.

Second problem. Fixtures

A similar problem exists with fixtures in both frameworks. Right now, Allure Report doesn't "see" the fixtures in your code, unless you specify them with AllureBefore (for setting up) or AllureAfter (for tearing down). Again, this means manual work, which means spending a lot of time and quite possibly forgetting a fixture, or mixing up the setup and teardown fixtures.

There are other problems too, but they are mostly about the same thing - making the integrations work out of the box.

Solution

For historical reasons, the xUnit and NUnit integrations had to use an API that forced the original authors into creating all those custom attributes that cause problems right now. But today, we can change this by migrating to a different API that just looks at all tests regardless of their attributes.

Examples

Let's see how it all works right now. Those are some example tests for XUnit:

using System;

using Allure.XUnit.Attributes.Steps;
using Xunit;

namespace MyTests
{
    public class MyTestClass : IDisposable
    {
        [AllureBefore]
        public MyTestClass()
        {
            // Set up code
        }

        [AllureAfter]
        public void Dispose()
        {
            // Tear down code
        }

        [AllureXunit]
        public void MyTest()
        {
            // Test body
        }

        [AllureXunitTheory]
        [InlineData(1, "a")]
        [InlineData(2, "b")]
        public void MyTheory(int a, string b)
        {
            // Theory body
        }
    }
}

A setup, a teardown, and two unit tests. As you can see, all four functions have Allure annotations, which you've got to provide them manually. Without them, Report doesn't "see" the tests. Now, this is what the same code is going to look like once we're finished refactoring:

using System;
using Xunit;

namespace MyTests
{
    public class MyTestClass : IDisposable
    {
        public MyTestClass()
        {
            // Set up code
        }

        public void Dispose()
        {
            // Tear down code
        }

        [Fact]
        public void MyTest()
        {
            // Test body
        }

        [Theory]
        [InlineData(1, "a")]
        [InlineData(2, "b")]
        public void MyTheory(int a, string b)
        {
            // Theory body
        }
    }
}

No Allure annotations are necessary to run this stuff, you don't even import anything allure-specific - and it all works. You can still add anything from Allure Report - but because you want to, not because you need to!

Conclusion

All in all, the changes that Maksim is working on right now will make allure-csharp integrations work out of the box. That means less manual work, which also means fewer errors, and also more flexibility as to which libraries you can work with. All in all, we're hoping more people will use Allure Report!