XML JUnit report with logs

Cucumber JUnit XML report with logs

Posted on by guru

In the previous post, tutorial, I explained the solution of how to add logs for the cucumber JSON report. In the current post, I will try to make an easy explanation that will cover the JUnit cucumber XML report problem where no logs are displayed. At the same time, I will reuse the implementation from the previous post, so my recommendation is to read it carefully.

The problem

The initial problem is that the default cucumber Junit XML report doesn’t have the ability to add the logs in the generated report if the scenario has passed with success. So the initial implementation and generation of the XML JUnit report look something similar to this: 

<testcase classname="Validating comments emails for all posts written by specific user" name="Searching the user by name and validating all emails format for comments under his posts" time="6.197">
            <system-out>
                <![CDATA[Given system administrator is searching for user following "Delphine".......passed
    When system administrator is searching for all user posts...................passed
    Then system administrator validates all email from comments under posts written by user.passed
    ]]>
            </system-out>
</testcase>


As you can see no logs are displayed in the report only the step name is displayed in the system-out tag.

The solution

The solution is to rewrite the initial cucumber implementation in order to attach all the necessary logs in the generated XML report file.

  1. The first step is to copy and create a Custom JUnit Formatter base on the initial one, how you can see the cucumber Junit Formater is a final class so it can’t be extended.
    “public final class JUnitFormatter implements EventListener”
  2. So based on JUnitFormatter I will copy all the content and will create my CustomJUnitFormatter.
  3. From the initial implementation, you will need to do the following steps in order to get the logs populated in the XML report.
  4. Register the WriteEvent in the method setEventPublisher
@Override
public void setEventPublisher(EventPublisher publisher) {
    publisher.registerHandlerFor(TestRunStarted.class, this::handleTestRunStarted);
    publisher.registerHandlerFor(TestSourceParsed.class, this::handleTestSourceParsed);
    publisher.registerHandlerFor(TestCaseStarted.class, this::handleTestCaseStarted);
    publisher.registerHandlerFor(WriteEvent.class, this::handleWrite);
    publisher.registerHandlerFor(TestCaseFinished.class, this::handleTestCaseFinished);
    publisher.registerHandlerFor(TestStepFinished.class, this::handleTestStepFinished);
    publisher.registerHandlerFor(TestRunFinished.class, this::handleTestRunFinished);
}

5. Add a content log holder per test case, this will be a Map that will contain all logs that will be received in the WriteEvent. I’ve created the currentTestCaseLogs map and initialized it at the moment of Formater creation.

private Map<String, Object> currentTestCaseLogs;

public CustomJUnitFormatter(OutputStream out) {
    this.writer = new UTF8OutputStreamWriter(out);
    currentTestCaseLogs = new HashMap<>();

    try {
        document = DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument();
        rootElement = document.createElement("testsuite");
        document.appendChild(rootElement);
    } catch (ParserConfigurationException e) {
        throw new CucumberException("Error while processing unit report", e);
    }
}

6. It’s necessary to clean the map values after each test case execution or at the start of each test case starter that’s why in the event TestCaseStarted I’ve added the following code to clean the holder logs map on each test case start event. 



private void handleTestCaseStarted(TestCaseStarted event) {
    if (currentFeatureFile == null || !currentFeatureFile.equals(event.getTestCase().getUri())) {
        currentFeatureFile = event.getTestCase().getUri();
        previousTestCaseName = "";
        exampleNumber = 1;
    }
    testCase = new TestCase(event.getTestCase());
    root = testCase.createElement(document);
    testCase.writeElement(root);
    rootElement.appendChild(root);

    increaseTestCount(rootElement);
    currentTestCaseLogs.clear();
}

7. Adding the collected logs to StringBuffer in order to be saved in XM. I’ve created the below method in order to add it in the moment of TestCaseFinished in the method addTestCaseElement right after addStepAndResultListing(sb);

private void addLogs(StringBuilder stringBuilder) {
    if (currentTestCaseLogs.containsKey("output")) {
        stringBuilder.append("[Start logs]:").append(System.lineSeparator());
        List<String> logOutput = (List<String>) currentTestCaseLogs.get("output");
        logOutput.forEach(stringBuilder::append);
        stringBuilder.append("[End logs]");
    }
}

8. Adding the Custom created formatter as a plugin in configuration. I’m using Cucumber 7 with JUnit5 that why the plugin configuration is in the property file for me. In the file junit-platform.properties under property cucumber.plugin=com.steti.atf.restassured.formatter.CustomJUnitFormatter:target/custom.xml

9. The final result is the following, after passing the test steps we will see all the logs that had been collected during scenario execution.

    <testcase classname="Validating comments emails for all posts written by specific user" name="Searching the user by name and validating all emails format for comments under his posts" time="5.729">
        <system-out>
            <![CDATA[Given system administrator is searching for user following "Delphine".......passed
When system administrator is searching for all user posts...................passed
Then system administrator validates all email from comments under posts written by user.passed
[Start logs]:
15:27:51.567 [main] INFO  c.s.a.r.step.SearchUserStep - Retrieving data for following user Delphine
15:27:53.714 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Searching all posts for user with id: 9
15:27:53.844 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 81
15:27:54.024 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Vella.Mayer@colten.net
15:27:54.026 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Caleb_Dach@kathleen.us
15:27:54.027 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Patience_Bahringer@dameon.biz
15:27:54.028 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Destinee.Simonis@jose.tv
15:27:54.030 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Keshaun@brown.biz
15:27:54.037 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 82
15:27:54.258 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Merle.Schultz@marcel.org
15:27:54.260 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Malvina_Fay@louie.name
15:27:54.260 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Domenick_Douglas@gabe.us
15:27:54.262 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Isaac@allene.net
15:27:54.263 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Marianna.Pacocha@george.net
15:27:54.263 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 83
15:27:54.407 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Sister_Barton@lela.com
15:27:54.409 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Autumn.Lebsack@kasandra.ca
15:27:54.409 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Irma.OKon@arden.me
15:27:54.409 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Alaina_Hammes@carter.info
15:27:54.409 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Alia@addison.org
15:27:54.410 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 84
15:27:54.557 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Aurelie_McKenzie@providenci.biz
15:27:54.560 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: May_Steuber@virgil.net
15:27:54.561 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Tessie@emilie.co.uk
15:27:54.562 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Priscilla@colten.org
15:27:54.562 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Aylin@abigale.me
15:27:54.562 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 85
15:27:54.680 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Holden@kenny.io
15:27:54.682 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Guillermo_Dare@dorothea.tv
15:27:54.683 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Oscar@pearline.com
15:27:54.683 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Jonathon_Feest@maxime.io
15:27:54.684 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Micah_Wolf@lennie.co.uk
15:27:54.686 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 86
15:27:54.823 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Shany@daisha.biz
15:27:54.824 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Drew_Lemke@alexis.net
15:27:54.824 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Karina.Donnelly@liam.com
15:27:54.824 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Ahmed_Runolfsson@claire.name
15:27:54.824 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Marilou_Halvorson@kane.io
15:27:54.825 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 87
15:27:54.950 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Bernie.Schoen@seamus.co.uk
15:27:54.953 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Joesph@darryl.net
15:27:54.953 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Timmothy.Corwin@augustus.co.uk
15:27:54.954 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Julien_OHara@vance.io
15:27:54.954 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Susan.Bartell@euna.org
15:27:54.954 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 88
15:27:55.067 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Selena.Quigley@johan.co.uk
15:27:55.068 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Clifton_Boehm@jacynthe.io
15:27:55.069 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Lizzie_Bartell@diamond.net
15:27:55.069 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Yasmeen@golda.ca
15:27:55.070 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Adolf.Russel@clark.ca
15:27:55.070 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 89
15:27:55.179 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Elian@matilda.me
15:27:55.181 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Salma@francis.net
15:27:55.181 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Orlando_Dickinson@vern.org
15:27:55.181 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Elda@orval.name
15:27:55.182 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Dennis@karley.net
15:27:55.182 [main] INFO  c.s.a.r.a.impl.UserPostActionImpl - Evaluating all comment from post with id: 90
15:27:55.273 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Jedediah@mason.io
15:27:55.274 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Maryam@jack.name
15:27:55.274 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Rick@carlos.tv
15:27:55.274 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Vallie@jerrod.net
15:27:55.274 [main] INFO  c.s.a.r.v.impl.FieldsValidatorImpl - Validating following email: Adolph.Hayes@isobel.biz
[End logs]]]>
        </system-out>

Hope this will help you and have a happy testing!