Introduction

Extended Cucumber Runner is the extension of standard Cucumber JUnit runner which additionally supports:

Before- and After- suite methods
Failed Tests Re-run
Advanced Reporting after tests completion

This extension is done in a form of JUnit runner. Since there are some additional options there is dedicated @ExtendedCucumberOptions annotation for that.

Usage

The use of Extended Cucumber runner is similar to standard runner. Actually it is wrapper above the standard Cucumber runner object.

JUnit

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
package com.github.mkolisnyk.cucumber.reporting;

import org.junit.runner.RunWith;

import com.github.mkolisnyk.cucumber.runner.ExtendedCucumber;
import com.github.mkolisnyk.cucumber.runner.ExtendedCucumberOptions;

import cucumber.api.CucumberOptions;

@RunWith(ExtendedCucumber.class)
@ExtendedCucumberOptions(jsonReport = "target/cucumber.json",
        retryCount = 3,
        detailedReport = true,
        detailedAggregatedReport = true,
        overviewReport = true,
        //coverageReport = true,
        jsonUsageReport = "target/cucumber-usage.json",
        usageReport = true,
        toPDF = true,
        excludeCoverageTags = {"@flaky" },
        includeCoverageTags = {"@passed" },
        outputFolder = "target")
@CucumberOptions(plugin = { "html:target/cucumber-html-report",
        "json:target/cucumber.json", "pretty:target/cucumber-pretty.txt",
        "usage:target/cucumber-usage.json", "junit:target/cucumber-results.xml" },
        features = { "./src/test/java/com/github/mkolisnyk/cucumber/features" },
        glue = { "com/github/mkolisnyk/cucumber/steps" },
        tags = {"@consistent"})
public class SampleCucumberTest {
}

TestNG

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
package com.github.mkolisnyk.cucumber.reporting;

import com.github.mkolisnyk.cucumber.runner.ExtendedCucumber;
import com.github.mkolisnyk.cucumber.runner.ExtendedCucumberOptions;
import com.github.mkolisnyk.cucumber.runner.ExtendedTestNGRunner;

import cucumber.api.CucumberOptions;

@ExtendedCucumberOptions(jsonReport = "target/cucumber.json",
        retryCount = 3,
        detailedReport = true,
        detailedAggregatedReport = true,
        overviewReport = true,
        //coverageReport = true,
        jsonUsageReport = "target/cucumber-usage.json",
        usageReport = true,
        toPDF = true,
        excludeCoverageTags = {"@flaky" },
        includeCoverageTags = {"@passed" },
        outputFolder = "target")
@CucumberOptions(plugin = { "html:target/cucumber-html-report",
        "json:target/cucumber.json", "pretty:target/cucumber-pretty.txt",
        "usage:target/cucumber-usage.json", "junit:target/cucumber-results.xml" },
        features = { "./src/test/java/com/github/mkolisnyk/cucumber/features" },
        glue = { "com/github/mkolisnyk/cucumber/steps" },
        tags = {"@consistent"})
public class SampleCucumberTest extends ExtendedTestNGRunner {
}

Major options to set

Option	Type	Description	Default Value
jsonReport	String	Contains path to standard Cucumber JSON results output
jsonReports	String[]	Contains paths to multiple Cucumber JSON result output files. This way multiple results can be merged in one report. If jsonReport field is used, it’s value will be added to final array of paths	{}
jsonUsageReport	String	Contains path to standard Cucumber usage report	“cucumber-usage.json”
jsonUsageReports	String[]	Contains paths to multiple usage reports	{}
outputFolder	String	Contains path to output folder where all reports will be written to	”.”
reportPrefix	String	Common file prefix which will be used for all generated output files	“cucumber-results”
usageReport	boolean	Enables/disables usage report	false
overviewReport	boolean	Enables/disables overview report	false
featureOverviewChart	boolean	Enables/disables feature overview chart	false
overviewChartsReport	boolean	Enables/disables overview charts report	false
coverageReport	boolean	Enables/disables coverage report	false
includeCoverageTags	String[]	Used to define which tags are to be included in coverage report.	{ }
excludeCoverageTags	String[]	Used to define which tags are to be excluded from coverage report.	{ }
detailedReport	boolean	Enables/disables detailed report	false
detailedAggregatedReport	boolean	Enables/disables detailed results aggregation. Important if failed tests re-run is enabled	false
breakdownReport	boolean	Enables/disables breakdown report	false
breakdownConfig	String	Path to the configuration file for breakdown report. Mandatory if this report is enabled.	””
featureMapReport	boolean	Enables/disables feature map report	false
featureMapConfig	String	Path to the configuration file for feature map report. Mandatory if this report is enabled.	””
knownErrorsReport	boolean	Enables/disables known errors report	false
knownErrorsConfig	String	Path to the configuration file for known errors report. Mandatory if this report is enabled.	””
consolidatedReport	boolean	Enables/disables consolidated report	false
consolidatedReportConfig	String	Path to the configuration file for consolidated report. Mandatory if this report is enabled.	””
systemInfoReport	boolean	Enables/disables system information report	false
screenShotSize	String	Defines the size of screenshots in the report. It should be a string similar to the one which is used in width attribute of the img HTML tag. Examples: 300px, 50%	””
screenShotLocation	String	Not used at the moment	””
formats	String[]	Defines the list of export formats	{ }
toPDF	boolean	Enables/disables PDF export	false
pdfPageSize	String	Defines the page size of exported PDF	“auto”
retryCount	int	Defines the number of failed tests re-run	0
threadsCount	int	Defines the number of threads to run in parallel	1
threadsCountValue	String	Defines the system property name containing the number of threads to run in parallel	””
benchmarkReport	boolean	Flag which enables/disables generation of the Benchmark Report	false
benchmarkReportConfig	String	The configuration which defines the way the Benchmark Report is to be generated	””
customTemplatesPath	String	Defines the path for Custom report templates	””
customReport	boolean	Flag which enables/disables generation of the Custom Report	false
customReportTemplateNames	String[]	The list of template names to be used for custom report	{}

Parameterizing Values

In some cases we need to define dynamic values for the output. Normally it may be needed if we would like to drop reports to some folder which name corresponds to current date and time or we need to paste value from either system property or environment variable.

For this purpose the outputFolder and reportPrefix fields may contain specific expressions which are later calculated. The following expressions are supported:

Format	Description
DATE(<format>)	Inserts current date/time. The format should fit the Joda time date format specification
${<variable>}	Inserts system property or environment variable.

The following sample demonstrates the use of parameterizing values:

1
2
3
4
5
@ExtendedCucumberOptions(
        jsonReport = "target/cucumber.json",
        overviewReport = true,
        outputFolder = "${user.dir}/DATE(dd-MM-yyyy)"
        reportPrefix = "results-${user}")

In the above example entries ${user.dir} and ${user} will be replaced with system properties named user.dir and user. Otherwise, engine will try to replace values with environment variables of the same name.

The DATE(dd-MM-yyyy) statement will be replaced with actual current date in the specified format. E.g. 28-09-2016.

Overriding properties from external values

Sometimes we would like to define values externally depending on various parameters calculated in build scripts or so. For this purpose library reserves an ability to override fields in ExtendedCucumberOptions annotation with system properties. Such system properties should fit the following format:

cucumber.reports.<field name>

where field name is the name of the ExtendedCucumberOptions annotation field name. E.g. if we want to override retryCount field we should define cucumber.reports.retryCount property.

Note that currently we can override only booleans, integers and strings. Array values aren’t supported.

Cucumber Reports

Extended Cucumber Runner