Run Options

Command Line

Maven Plugin

Since 0.0.2 version the Aerial is provided not only as the library or command line utility but also as the Maven plugin which is bound to generate-sources phase. Main goal implementing generation is aerial:generate. More information can be found on project documentation pages.

Library

Maven Archetypes

Since 0.0.5 version Aerial was updated with the set of archetypes which generate sample test project with consistent references to Aerial and other test engines. Archetypes catalog can be references from here.

Currently available archetypes are:

Name	Description	Latest Version
aerial-cucumber-junit-archetype	Generates sample Aerial project integrated with Cucumber + JUnit
aerial-cucumber-testng-archetype	Generates sample Aerial project integrated with Cucumber + TestNG
aerial-maven-cucumber-junit-archetype	Generates sample Aerial project integrated with Maven + Cucumber + JUnit
aerial-maven-cucumber-testng-archetype	Generates sample Aerial project integrated with Maven + Cucumber + TestNG
aerial-maven-jbehave-junit-archetype	Generates sample Aerial project integrated with Maven + JBehave + JUnit
aerial-maven-jbehave-testng-archetype	Generates sample Aerial project integrated with Maven + JBehave + TestNG

Test Engines Supported

JUnit

Initially Aerial was designed as the wrapper on Cucumber and JUnit. So here is the typical configuration for test class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

package com.github.mkolisnyk.aerial;

import org.junit.runner.RunWith;
import com.github.mkolisnyk.aerial.annotations.Aerial;
import com.github.mkolisnyk.aerial.core.AerialRunner;
import com.github.mkolisnyk.aerial.core.params.AerialSourceType;
import cucumber.api.CucumberOptions;

@CucumberOptions(
        format = {"html:target/cucumber-html-report",
                  "json:target/cucumber.json",
                  "pretty:target/cucumber-pretty.txt",
                  "usage:target/cucumber-usage.json"
                 },
        features = {"output/" },
        glue = {"com/github/mkolisnyk/aerial" },
        tags = { }
)
@Aerial(
    inputType = AerialSourceType.FILE,
    source = "src/test/resources",
    additionalParams = { "" },
    destination = "output/")
@RunWith(AerialRunner.class)
public class AerialRunnerTest {
    @AerialBeforeSuite
    public static void setUp() {
        System.out.println("setUp");
    }
    @AerialAfterSuite
    public static void tearDown() {
        System.out.println("tearDown");
    }
}

TestNG

Since version 0.0.4

In some projects people use TestNG instead of JUnit as the engine which may provide some additional features (however in most cases they are pretty similar). For this purpose there was additional functionality added which wraps TestNG tests into runnable structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

package com.github.mkolisnyk.aerial;

import org.junit.runner.RunWith;
import com.github.mkolisnyk.aerial.annotations.Aerial;
import com.github.mkolisnyk.aerial.core.AerialTestNGRunner;
import com.github.mkolisnyk.aerial.core.params.AerialSourceType;
import cucumber.api.CucumberOptions;

@CucumberOptions(
        format = {"html:target/cucumber-html-report",
                  "json:target/cucumber.json",
                  "pretty:target/cucumber-pretty.txt",
                  "usage:target/cucumber-usage.json"
                 },
        features = {"output/" },
        glue = {"com/github/mkolisnyk/aerial" },
        tags = { }
)
@Aerial(
    inputType = AerialSourceType.FILE,
    source = "src/test/resources",
    additionalParams = { "" },
    destination = "output/")
public class AerialRunnerTest extends AerialTestNGRunner {
    @AerialBeforeSuite
    public static void setUp() {
        System.out.println("setUp");
    }
    @AerialAfterSuite
    public static void tearDown() {
        System.out.println("tearDown");
    }
}

All we have to do here is to extend our test class from AerialTestNGRunner class.

Input Sources Supported

String

This input source is rather done for testing purposes. However, it can be used to invoke test scenarios generation programmatically.

File

JIRA

Since version 0.0.2

In some cases initial requirements may be stored somewhere in tracking systems like Jira. E.g. we can use some specific issue type and all specifications are placed in the description field. In this case we should be able to retrieve them the same way as from multiple files. For this purpose we can use JIRA as an input source.

Here is the Maven example on how to set Jira as an input source:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>com.mkolisnyk.github</groupId>
        <artifactId>aerial-maven-plugin</artifactId>
        <version>0.0.2</version>
        <configuration>
            <inputType>JIRA</inputType>
            <source>http://localhost:8080/jira</source>
            <outputType>FILE</outputType>
            <destination>output/</destination>
            <namedParams>
                <user>some_user</user>
                <password>some_password</password>
                <field>description</field>
            </namedParams>
            <valueParams>
                <param>project=Wallboards AND status=Open</param>
            </valueParams>
        </configuration>
      </plugin>
      ...
    </plugins>
  </build>
  ...
</project>

More detailed description of the above example can be found here.

GitHub

Since version 0.0.4

GitHub itself contains built-in tracker which also can be the source for requirement documents. Thus we can query issues based on some search criteria.

Mainly, in order to operate with GitHub input source we need to set the following parameters:

• Input Type - GITHUB the pre-defined AerialSourceType enum value

• Source - that should be base URL for GitHub API, e.g. for official GitHub it is https://api.github.com

• Value Parameters - the list of search queries is to be defined as the set of value parameters. The list of query parameters and some sample queries can be found here.

In the code itself we can target input to GitHub via Aerial annotation:

1
2
3
4
5

@Aerial(
    inputType = AerialSourceType.GITHUB,
    source = "https://api.github.com",
    additionalParams = { "repo:mkolisnyk/aerial state:open" },
    destination = "output/")

The same input can be configured via Maven with the following configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>com.mkolisnyk.github</groupId>
        <artifactId>aerial-maven-plugin</artifactId>
        <version>0.0.4</version>
        <configuration>
            <inputType>GITHUB</inputType>
            <source>https://api.github.com</source>
            <outputType>FILE</outputType>
            <destination>output/</destination>
            <valueParams>
                <param>repo:mkolisnyk/aerial state:open</param>
            </valueParams>
        </configuration>
      </plugin>
      ...
    </plugins>
  </build>
  ...
</project>

Custom

Since version 0.0.4

Additionally Aerial has functionality to load and process custom classes defined outside of the box. For this purpose there is reserved CUSTOM keyword for input type. This is specific feature for better extensibility and customization without changing the core engine. More details on configuration and extensions development see the Extensions Development Guide: Custom Input Readers chapter.

Scenarios Generated

Aerial uses document in specific format to generate scenarios. Each document contains specific sections and generally can be represented with the following structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

< Entire document description >

Feature: < Feature Name >
    < Feature description >

    Case: < Case Name >
        < Case Description >

        Action:
            < Action text >
        Input:
            < Input Data Table >
        On Success:
            < Actions in case of success >
        On Failure:
            < Actions in case of error >
        Pre-requisites:
            < Pre-requisites steps >
    Additional Scenarios:
        < Explicitly defined custom Cucumber scenarios >

Mainly it generates several sets of scenarios based on input data given. They are:

• Positive Scenarios - scenarios operating with valid data and expecting valid output

• Negative Scenarios - scenarios operating with incorrect data expecting error output

• Unique Value Scenarios - scenarios which are targeted to make sure that some value cannot be used more than once within the same scenario

Depending on scenario type different sections are used in different combinations. The below paragraphs contain more detailed description of each scenario generation.

Positive Scenarios

Positive scenarios operate with positive values and they are targeted to check expected behaviour. Mainly positive scenarios are generated from document description using the following structure:

1
2
3
4
5
6

Scenario: < Case Name > positive test
        Given < Pre-requisites steps >
        When < Action text >
        Then < Actions in case of success >
    Examples:
        <The positive test data table>

Combinatorial tests optimization

Since version 0.0.3

In order to decrease the number of test scenarios to be performed positive test scenarios are additionally optimized using combinatorial methods. It means that is we have some set of valid input values the entire test data would contain not just all possible combinations but it should cover all possible combinations of N parameters (by default N equals 2 which leads to pair-wise testing). It is done to decrease the number of tests to be performed. In worst case the number of tests is the same as for all possible combinations. But normally, if N is less than number of fields the number of tests is smaller.

Main idea behind that approach is that potentially error is caused when we process some combination of N parameters. The smaller N is the bigger optimization we have. E.g. If we have total number of fields equal 10 and we look for any records where each value is present at least once (N = 1) the number of tests to perform is the biggest number of values among all 10 fields.

Negative Scenarios

Negative scenarios are built the same way as positive scenarios except they operate with negative test data where at least one item doesn’t fit acceptable format. Also, since this scenario uses invalid input it expects actions on errors to be expected results. So, mainly negative test scenario is build using the following template:

1
2
3
4
5
6

Scenario: < Case Name > negative test
        Given < Pre-requisites steps >
        When < Action text >
        Then < Actions in case of error >
    Examples:
        <The negative test data table>

Unique Value Scenarios

Unique value scenario generation is triggered as soon as at least one field has Unique column value set to true in the input data table. In this context the Unique term isn’t restricted just with the case when we cannot create 2 records with the same value of some field. In this case uniqueness means that we cannot perform some action twice having the same value for some field.

Getting to the technical side of the scenario generation we should get the scenario when we run action successfully at first turn but on the second turn we get the error if we use the same value in some field. Thus, the unique value scenario can be described with the following template:

1
2
3
4
5
6
7
8

Scenario: < Case Name > unique value test
        Given < Pre-requisites steps >
        When < Action text >
        Then < Actions in case of success >
        When < Action text >
        Then < Actions in case of error >
    Examples:
        <Unique scenario data>

Mandatory Value Scenarios

Since version 0.0.5

Mandatory value scenario generation is triggered as soon as at least one field has Mandatory column value set to true in the input data table. For Aerial the term “Mandatory” means that value cannot be empty. So, the generation for such scenario type is based on the following algorithm:

1. Take any valid data row
2. Replace mandatory field with empty value
3. Add the row to the list of test data
4. Repeat previous steps for all mandatory fields

Scenario itself is generated based on the following structure:

1
2
3
4
5
6

Scenario: < Case Name > mandatory value test
        Given < Pre-requisites steps >
        When < Action text >
        Then < Actions in case of error >
    Examples:
        <Mandatory scenario data>

Tags Generation

Since version 0.0.4

In order to provide functionality to run just some sub-set of tests the Aerial generates tags. Mainly they are tags for Cucumber or JBehave (depending on which generation template is used). Tags are generated both on feature and scenario levels.

There are some common tags as well as there are some tags which are defined only within some specific feature. Normally they contain some common part which uniquely identifies each specific feature. Further this tag common part will be named as <tag base> or tag base. This tag base is taken based on different information depending on source type. The table below describes what is the input for tag base depending on source type:

Feature mainly contain the following tags:

Scenario tags are applied to individual scenarios. They are:

Data Types Supported

Int

Date

String

Enums

Since version 0.0.3

In some cases we have pre-defined set of input values and this list is fixed. Thus, the potential list of input parameters is limited with the list of specific values. In order to handle this the Enum data type was introduced.

Here is an example of Enum type definition in the Input section:

1
2
3

Input:
    | Name   | Type | Value                       |
    | Season | enum | Winter;Spring;Summer;Autumn |

All constant values are split with ; (semi-colon) delimiter. If for any reason we need to include it into some enumerated string we can escape it with back-slash. E.g.:

1
2
3

Input:
    | Name   | Type | Value                       |
    | Season | enum | Win\;ter;Spring;Summer;Autumn |

This way the first enum value would be Win;ter.

When test data is generated based on this type it generates 2 set of values:

• Positive - contains every single value which belongs to the enum

• Negative - contains value which doesn’t belong to enum

Custom

Since version 0.0.5

News