TestRail and Pytest integration - where test cases have steps in TestRail and each test in Pytest handling one step

Question

In TestRail I have sections that each have different Test Cases. Each Test Case has steps. This is how it is look in TestRail

Automation is using Pytest framework where each file is a test-specific Test Case and each method in this file verifies one step.

import pytest

class TestTestCase1:

    def test_tc1_step_1(self):
        assert 1 == 1

    def test_tc1_step_2(self):
        assert 2 == 3

    @pytest.mark.parametrize("a, b", [("2+4", 7), ("3+5", 8)])
    def test_tc1_step_3(self, a, b):
        assert eval(a) == b

    def test_tc1_step_4(self):
        assert 4 == 4

    def test_tc1_step_5(self):
        assert 5 == 5

I am trying to populate automation execution results as per the steps in Test Rail like in the picture below Expected population

I tried TRCLI and pytest-testrail.

Pytest-testrail is good however I didn't find any way to populate the result as per step but only for the test case overall. In my case, it also not providing the correct result at the end since executing each method as a new run for a test plan and the last one is the one that set a final result.

With TRCLI each method creates its own run and it is very hard to map the results. Didn't succeed to map results to already created TestPlan.

Any help will be appreciated!

Thank you!

Answer 1

You might want to have a look at Railflow - the commercial tool that we are developing.
This tool is very flexible and provide extensive configuration on how to export results into TestRail.
Here is a link to the docs for PyTest integration: https://docs.railflow.io/docs/railflow-for-testrail/testing-frameworks/pytest

We are also open to enhancements if you need some extra features.

Disclaimer: I'm a developer of Railflow