Cucumber Parallel Run

篇幅比较长，列个提要吧：

背景
核心思想
方案一
方案二
方案三
方案四
执行策略
执行结果
POM 详细配置

背景：

日积月累 Smoke + Regression Test Cases 总数达 1 万+，运行 4 个半小时之久。正常情况，晚上跑完，第二天上班便可 triage，也不耽误。但总会遇到意外情况，如 VMs disconnection issue, 或是其他国外团队深夜暗戳戳 deploy 新版本，再或 504 gateway timeout。。。等等不确定因素，要是下午时间 trigger一把，作息 965 的我们就不能保准能顺利出 Triage Report, 基本就放弃了，哈哈！遇到临近上线关键时期，特别是有重大 Bug 修复，Leader 没有办法也只能人工吭哧吭哧了，个人觉得坚信 Automation 比人工更靠谱。到了真正为团队做贡献的时刻到了，哈哈，缩短运行时间不就能完美解决这个问题，而且是一劳永逸，以上废话比较多。

说干就干，趁 Sprint 开始没啥紧急 Task，花了 2 天时间研究一下 Parallel Run 方案，深入研究才发现有多种方案可以实现，但能只有一种满足我的需求，下面我会罗列 4 种方案，分析其中优劣。

环境：Maven + Cucumber + Java

核心思想：

首先要明白 Maven 是怎样执行 Cucumber cases
就要了解 Surefire Plugin，说白了它就是一个 Test Runner，测试运行器。

The Surefire Plugin is used during the test phase of the build lifecycle to execute the unit tests of an application.

那么 Surefire 怎么运行 Cucumber cases 呢，请参考官网，通过执行 Cucumber JUnit Runner，依赖 Junit 和 cucumber-junit puglins。Junit 就是充当一个桥梁，解析 Cucumber feature files，执行 steps implement，并汇总执行结果，生成 Report。

JUnit is an open source unit testing framework for the Java programming language.

下面就是一个 Cucumber Junit Runner Class，@CucumberOptions annotation 就是用来进行相关配置的，有关细节可以参考 Cucumber官网。

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;@RunWith(Cucumber.class)
@CucumberOptions(features = "src/test/resources/features",glue = {"com.company.infra.services.stepdefs"},format = {"pretty", "html:target/site/cucumber-pretty", "json:target/cucumber.json"},tags = {"not (@archive or @ignore or @manual)"}
)
public class AllTest {// This class is empty because the options above fulfill all needs.
}

SureFire 怎么识别到 Runner Class Files
Filtering by Test Class Names for Maven Surefire

The Maven Surefire Plugin will scan for test classes whose fully qualified names match the following patterns.
**/Test*.java
**/*Test.java
**/*Tests.java
**/*TestCase.java
Moreover, it will exclude all nested classes (including static member classes) by default. Note, however, that you can override this default behavior by configuring explicit include and exclude rules in your pom.xml file. For example, to keep Maven Surefire from excluding static member classes, you can override its exclude rules.

默认情况下，maven-surefire-plugin 的 test 目标会自动执行测试源码路径（src/test/java/）下所有符合上面命名模式的测试类，当然也可以在 POM File 中配置。

通常创建一个 Default Cucumber Junit Runner Class，那就是串行跑所有的 cases。那么要实现 Parallel Run，就等同于创建多个 Runner Classes，由 Surefire Plugin 并行执行这些 Rnnner Classes。

方案一

Cucumber 4.0.0 及以后版本支持 Parallel Run，详情请参考 Parallel Execution

Cucumber can be executed in parallel using JUnit and Maven test execution plugins. In JUnit the feature files are run in parallel rather than scenarios, which means all the scenarios in a feature file will be executed by the same thread. You can use either Maven Surefire or Failsafe plugin to execute the runners.

注意是 Feature level 的 Parallel Run，不是 Scenarios Level 的。

这个方案看上去不错，然后并不适合我们，我们用的 Cucumber JVM 版本是 2.4，我也尝试过升级到 4.0.0，发现有些 steps 并不兼容，可能有其它 dependencies，且这些出问题的 features 并不是我们 Owned，国外其它团队，出于谨慎，我决定另行出路。

方案二

苦力活，为每一个 feature 创建一个 Cucumber Junit Runner Class，这样做失优雅，哈哈！我们有大概 200 多个 feature files，大量的 Copy&Paste 工作不说，还得 Push 到 git上，这一串串无它用的 Class Files，真难看，放弃，还是尝试找个能自动生成 Runner Class的开源插件吧。

方案三

cucable-plugin 终于找到一款能自动 generate runner。

How it works

Cucable will cut up feature file into the smallest possible runnable scenarios
Each generated feature file includes a single scenario
After this, the runner classes for those generated features are generated based on a provided template file, either
--->one runner per generated "single scenario" feature file
or
--->one runner per group of "single scenario" feature files

有两种 Generate Runner 模式：
One runner per generated scenario
为每一个 Scenario 生成一个 feature file 并 generate一个 Runner Class file，如下图。

One runner per group of generated scenarios
还是为每一个 Scenario 生成一个 feature file，创建一个 Runer Class，运行多个 Features。可以通过配置 desiredNumberOfRunners or desiredNumberOfFeaturesPerRunner option

首先需要准备一个 Template Cucumber Junit Runner Class file. Cucable Plugin 就是根据这个模板 Runner 为 generate 的 feature files 生成多个 Runner Files。

import cucumber.api.junit.Cucumber;
import cucumber.api.CucumberOptions;
import org.junit.runner.RunWith;@RunWith(Cucumber.class)
@CucumberOptions(features = {"target/parallel/features/[CUCABLE:FEATURE].feature"},plugin = {"json:target/cucumber-report/[CUCABLE:RUNNER].json"}
)public class CucableJavaTemplate {}

了解一下 cucable plugin 相关配置

<plugin><groupId>com.trivago.rta</groupId><artifactId>cucable-plugin</artifactId><version>${cucable-plugin.version}</version><executions><execution><id>generate-test-resources</id><phase>generate-test-resources</phase><goals><goal>parallel</goal></goals></execution></executions><configuration><!-- Required properties --><sourceRunnerTemplateFile>src/test/resources/parallel/cucable.template</sourceRunnerTemplateFile><sourceFeatures>src/test/resources/features</sourceFeatures><generatedFeatureDirectory>src/test/resources/parallel/features</generatedFeatureDirectory><generatedRunnerDirectory>src/test/java/parallel/runners</generatedRunnerDirectory><!-- Optional properties --><numberOfTestRuns>1</numberOfTestRuns><includeScenarioTags>@includeMe and @includeMeAsWell</includeScenarioTags>                                <logLevel>compact</logLevel><desiredNumberOfRunners>2</desiredNumberOfRunners>                                <!-- or <desiredNumberOfFeaturesPerRunner>5</desiredNumberOfRunners> --></configuration>
</plugin>

必要参数，从名字就能看出，就不解释了，来了解一下比较重要的可选参数。
numberOfTestRuns
默认值为1, 如果值设置 n 次，那么这个 Scenario 将反复运行 n 次，也就为此生成多个 feature Files，其实只是一同一个 Scenario 而已，建议慎用。

This can be used if specific scenarios should be run multiple times. If this options is not set, its default value is 1.

For each test run, the whole set of features and runners is generated like this:
MyFeature_scenario001_run001_IT.feature
MyFeature_scenario001_run002_IT.feature
MyFeature_scenario001_run003_IT.feature
etc.

parallelizationMode
cucable 是支持 Scenario Level Mode 和 Feature Level Mode 的，但是如果用 Feature Level Mode, 那么 tag filter 将失效，这是个重要信息。

By default, Cucable uses the parallelizationMode = scenarios meaning that feature files are split into individual scenarios that each have a dedicated runner.

Sometimes it may be desirable, to parallelize complete features. When setting the parallelizationMode = features, only complete features containing all of their source scenarios are generated so each runner runs a complete feature.

Note: For this mode to work, <sourceFeatures> must specify a directory. Also, includeScenarioTags cannot be used.

desiredNumberOfRunners
设置固定 Runners 个数，这样每个 Runner 会运行多个 generated feature 的 cases，也就是每个 Runner 执行的 feature file 个数是变化的。

If you set this options, all generated features will be distributed to a fixed set of runner classes. This means that one runner can potentially run multiple features in sequence.

If this option is not set, its default value is 0 which basically
means “Generate a dedicated runner for every generated feature”.

Note: This cannot be used together with desiredNumberOfFeaturesPerRunner!

desiredNumberOfFeaturesPerRunner
设置每个 Runner 跑多少个 Feature files，根据 feature 的多少，Runner 个数会变动

If you set this option, all generated features will be distributed to a dynamic set of runner classes so that every runner contains a fixed number of generated features. This means that one runner can potentially run multiple features in sequence.

If this option is not set, its default value is 0 which basically
means “Generate a dedicated runner for every generated feature”.

Note: This cannot be used together with desiredNumberOfRunners!

完成了 Cucable 配置，接下来配置并行策略了。相关参数将在后面详细介绍。

<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-failsafe-plugin</artifactId><executions><execution><id>Run parallel tests</id><phase>integration-test</phase><goals><goal>integration-test</goal></goals></execution></executions><configuration><testFailureIgnore>true</testFailureIgnore><forkCount>${maven.fork.count}</forkCount><reuseForks>false</reuseForks><argLine>-Dfile.encoding=UTF-8</argLine><disableXmlReport>true</disableXmlReport></configuration></plugin>

缺陷
综上就是有关 Cucable Plugin的介绍，看着挺完美的，但存在不少缺点。

Scenario Level 缺陷
我尝试了，如果不设置 desiredNumberOfRunners 或 desiredNumberOfFeaturesPerRunner 的值，就一个 Scenario 一个 Runner，就算把 forkCount 设置为4，reuseForks 设置为 ture，也能将 Memory 和 CPU 撑爆，跟串行几乎没啥基本，主要是 Runner 太多，4个 Java™ Platform SE binary 进程每跑完一个 Scenario Runner 就得切换执行其它 Scenario Runner，太频繁了，开销太大。

有兴趣的可以尝试设置 desiredNumberOfRunners 或 desiredNumberOfFeaturesPerRunner 的值，我尝试了一下，forkcount 为 10， reusefork 为 true，只看到 4 个 Java 进程，然而还没啥效果呢。

Feature Level缺陷
一个 feature file 生成一个 Runner，这样可以克服上面 Scenario Level 的缺陷，可以减少Java™ Platform SE binary进程切换频率，减少开销。但是这种模式也有缺陷，就是不支持 Tag filter。由于产品历史原因，我们大量 feature file 有包含其它国外团队的Scenaios，而我们只跑 APAC 范围的 Scenarios，所以哇凉哇凉的，但是执着的我怎么可以放弃的，哈哈！

方案四
cucumber-jvm-parallel-plugin
停止维护了，但它是我的最佳方案，哈哈！！

As of cucumber-jvm:4.0.0 parallel execution is supported natively by cucumber. As such, upgrading to Cucumber 4.0.0 is recommended and this plugin is no longer maintained.

支持 Scenario Level 和 Feature Level

This plugin automatically generates a Cucumber JUnit or TestNG runner for each scenario/feature file found in your project.

配置如下，我是采用 Feature Level 的模式，ParallelScheme->FEATURE, 支持 tag filter，克服了方案三的 Feature Level 的缺陷

<plugin><groupId>com.github.temyers</groupId><artifactId>cucumber-jvm-parallel-plugin</artifactId><version>5.0.0</version><executions><execution><id>generateRunners</id><phase>generate-test-sources</phase><goals><goal>generateRunners</goal></goals><configuration><!-- Mandatory --><!-- List of package names to scan for glue code. --><glue><package>com.company.infra.services.stepdefs</package></glue><featuresDirectory>src/test/resources/features</featuresDirectory><!-- These are optional, with the default values --><!-- The naming scheme to use for the generated test classes.  One of ['simple', 'feature-title', 'pattern'] --><namingScheme>feature-title</namingScheme><!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario.  FEATURE generates a runner per feature. --><parallelScheme>FEATURE</parallelScheme><!-- List of cucumber plugins. When none are provided the json formatter is used. For more advanced usage see section about configuring cucumber plugins --><plugins><plugin><name>json</name><extension>json</extension><outputDirectory>${project.build.directory}/cucumber-parallel/json</outputDirectory></plugin><plugin><name>html</name><extension>html</extension><outputDirectory>${project.build.directory}/cucumber-parallel/html</outputDirectory></plugin><plugin><name>rerun</name><extension>rerun</extension><outputDirectory>${project.build.directory}/rerun</outputDirectory></plugin></plugins></configuration></execution></executions></plugin>

featuresDirectory parameter
feature 文件的根目录

The plugin will search featuresDirectory for *.feature files and generate a JUnit test for each one.
WARNING: featuresDirectory must denote a directory within the root of the classpath.
Example:
Resources in src/test/resources are added to the classpath by default.
src/test/resources/features is in the root of the classpath, so would be valid for featuresDirectory
src/test/resources/features/sub_folder is not in the root of the classpath, so would not be valid to put in featuresDirectory

Naming Scheme parameter
配置自动生成文件的命名规则

The naming scheme used for the generated files is controlled by the namingScheme property. The following values are supported:

Property	Generated Name
simple	ParallelXXIT.java, where XX is a one up counter.
feature-title	The name is generated based on the feature title with a set of rules to ensure it is a valid classname. The reules are detailed in the next subsection below.
pattern	Generate the filename based on the namingPattern property.

By default, generated test files use the simple naming strategy.

执行策略

确定了方案，就得采用最优策略执行。
Fork Options and Parallel Test Execution

两种方式：

1. parallel parameter
控制并发执行的对象，注意这种方式是在一个进程中执行多个线程。

For JUnit 4.7 and onwards, this may be methods, classes, both, suites, suitesAndClasses, suitesAndMethods, classesAndMethods or all.

The important thing to remember with the parallel option is: the concurrency happens within the same JVM process. That is efficient in terms of memory and execution time

线程数配置：
useUnlimitedThreads 为 true，不限制线程数。useUnlimitedThreads 为 false 时可以使用 threadCount 和 perCoreThreadCount 参数。还可以通过 threadCountSuites，threadCountClasses，threadCountMethods 在不同粒度限制线程。parallelTestsTimeoutInSeconds 和 parallelTestsTimeoutForcedInSeconds 参数设置线程的超时时间。

The parameter useUnlimitedThreads allows for an unlimited number of threads. Unless useUnlimitedThreads=true, the parameter threadCount can be used with the optional parameter perCoreThreadCount=true (true by default). The parameters useUnlimitedThreads and threadCount are to be interpreted in the context of the value specified for the parallel parameter.

2. Forked Test Execution
创建多个测试进程，如果 forkCount 参数值后加 C，表示乘以 CPU 核数

The parameter forkCount defines the maximum number of JVM processes that maven-surefire-plugin will spawn concurrently to execute the tests. It supports the same syntax as -T in maven-core:
if you terminate the value with a ‘C’, that value will be multiplied with the number of available CPU cores in your system.
For example
forkCount=2.5C on a Quad-Core system will result in forking up to ten concurrent JVM processes that execute tests.

reuseForks 表示一个测试进程执行完了之后是杀掉还是重用来继续执行后续的测试。

The parameter reuseForks is used to define whether to terminate the spawned process after one test class and to create a new process for the next test in line (reuseForks=false), or whether to reuse the processes to execute the next tests (reuseForks=true).

Combining forkCount and parallel
多种并行方式组合

forkCount=0, 或 forkCount=1/reuseForks=true，可以和 parallel 自由组合。

The modes forkCount=0 and forkCount=1/reuseForks=true can be combined freely with the available settings for parallel.

forkCount 的测试进程是按类为单位执行的，测试类整个整个的传到测试进程中执行。reuseForks=false 或 forkCount>1 时，就会使用独立的测试进程，所以 parallel=classes就失效了。但是还是可以组合 parallel=methods/threadCount=n 指定每个测试进程里的并发线程数。

As reuseForks=false creates a new JVM process for each test class, using parallel=classes would have no effect. You can still use parallel=methods, though.

When using reuseForks=true and a forkCount value larger than one, test classes are handed over to the forked process one-by-one. Thus, parallel=classes would not change anything. However, you can use parallel=methods: classes are executed in forkCount concurrent processes, each of the processes can then use threadCount threads to execute the methods of one class in parallel.

parellel 限制了类不能并发，但在 forkCount 并发进程中可以执行类，然后每个进程可以使用threadCount线程并行执行一个类的方法。这句话说明了 parallel 只能限制在 jvm 内部是并发执行方法或者类，说白是限制线程，而对进程没有约束。parallel 和 forkcount 分别是线程和进程级别。

本地Jenkins执行结果

Smoke Test Cases：
feature Files: 84
Scenarios: 596

方式	执行时间	说明
普通串行	20min
Cucable-plugin	30min	Scenario level paralle run，forkcout=10，reusefork=true, desiredNumberOfRunners=84
Cucable-plugin	37min	Scenario level paralle run，forkcout=10，reusefork=true, desiredNumberOfRunners=0
Cucable-plugin	NA	Feature level paralle run，因为不支持 tag filter 所以没必要验证
cucumber-jvm-parallel-plugin	5-6min	Feature level praralle run，forkcout=10，reusefork=true 最佳方案`建议将 reusefork 设为 false，请看下面的补充更新`
cucumber-jvm-parallel-plugin	5-6min	Feature level praralle run，forkcout=2.5C，reusefork=true 本机 4 核，但是居然起了 20 个 java 进程
cucumber-jvm-parallel-plugin	大量 case 失败，多线程不安全	Feature level praralle run，parallel=classes，useUnlimitedThreads=true
cucumber-jvm-parallel-plugin	大量 case 失败，多线程不安全	Feature level praralle run，parallel=classesAndMethods，useUnlimitedThreads=true
cucumber-jvm-parallel-plugin	52min 震惊，尽管没有因为多线程失败	Feature level praralle run，parallel=methods，useUnlimitedThreads=true
cucumber-jvm-parallel-plugin	7min	Feature level praralle run，Combining forkcout=10，reusefork=true，parallel=methods，useUnlimitedThreads=true,

Regresion Test Cases：
feature Files: 84
Scenarios: 9852

方式	执行时间	说明
普通串行	4h
cucumber-jvm-parallel-plugin	1h20min	Feature level praralle run，forkcout=10，reusefork=true

补充更新 - 2020-10-24

经过一周在 remote jenkins 上的运行情况观察，这种方案 forkcout=10， reusefork=true 经常出现下列异常，Runner 进程异常退出，导致一个 feature cases 执行结束后没法正常将执行结果写入 cucumber report 的 json文件，也就不能正常生成 report 了

org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?

导致上面异常的原因，fork 的 java 进程由于是 reuse 模式，随着不断执行太多 feature cases，内存一直累积攀升，没能及时释放无效内存，同时 10 个 java 进程都如此，VM 性能可想而知了。设置reusefork=false，观察 java 进程的内存占用一直维持在一个稳定的数字，没有累积变高的现象，就不会出现这种异常了。

详细配置：

Cucumber-JVM-Parallel-Plugin：
mvn clean test -P p-parallel cluecumber-report:reporting -Dcucumber.options=“–tags @smoke”

<profile><id>p-parallel</id><activation><activeByDefault>false</activeByDefault></activation><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.7.0</version><configuration><source>1.8</source><target>1.8</target></configuration></plugin><plugin><groupId>com.github.temyers</groupId><artifactId>cucumber-jvm-parallel-plugin</artifactId><version>5.0.0</version><executions><execution><id>generateRunners</id><phase>generate-test-sources</phase><goals><goal>generateRunners</goal></goals><configuration><!-- Mandatory --><!-- List of package names to scan for glue code. --><glue><package>com.company.infra.services.stepdefs</package></glue><featuresDirectory>src/test/resources/features</featuresDirectory><!-- These are optional, with the default values --><!-- The naming scheme to use for the generated test classes.  One of ['simple', 'feature-title', 'pattern'] --><namingScheme>feature-title</namingScheme><!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario.  FEATURE generates a runner per feature. --><parallelScheme>FEATURE</parallelScheme><!-- List of cucumber plugins. When none are provided the json formatter is used. For more advanced usage see section about configuring cucumber plugins --><plugins><plugin><name>json</name><extension>json</extension><outputDirectory>${project.build.directory}/cucumber-parallel/json</outputDirectory></plugin><plugin><name>html</name><extension>html</extension><outputDirectory>${project.build.directory}/cucumber-parallel/html</outputDirectory></plugin><plugin><name>rerun</name><extension>rerun</extension><outputDirectory>${project.build.directory}/rerun</outputDirectory></plugin></plugins></configuration></execution></executions></plugin><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><version>3.0.0-M3</version><configuration><argLine>-Dfile.encoding=UTF-8</argLine><testFailureIgnore>true</testFailureIgnore><!--<forkCount>2.5C</forkCount>--><!--<reuseForks>true</reuseForks>--><parallel>classesAndMethods</parallel ><useUnlimitedThreads>true</useUnlimitedThreads><includes><include>**/*IT.class</include></includes></configuration></plugin><plugin><groupId>com.trivago.rta</groupId><artifactId>cluecumber-report-plugin</artifactId><version>2.5.0</version><executions><execution><id>report</id><phase>post-integration-test</phase><goals><goal>reporting</goal></goals></execution></executions><configuration><startPage>ALL_FEATURES</startPage><expandBeforeAfterHooks>false</expandBeforeAfterHooks><expandStepHooks>false</expandStepHooks><expandDocStrings>true</expandDocStrings><customParameters><Cucuzzi>Infra Shared Service Tests Report</Cucuzzi></customParameters><sourceJsonReportDirectory>${project.build.directory}/cucumber-parallel/json</sourceJsonReportDirectory><generatedHtmlReportDirectory>${project.build.directory}/cluecumber-report</generatedHtmlReportDirectory></configuration></plugin></plugins></build></profile>

cucable plugin：
mvn clean verify failsafe:verify -Pparallel -Dist=“@smoke”

<profile><id>parallel</id><activation><activeByDefault>false</activeByDefault></activation><properties><ist>@ist</ist></properties><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.7.0</version><configuration><source>1.8</source><target>1.8</target></configuration></plugin><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-surefire-plugin</artifactId><version>3.0.0-M3</version><configuration><argLine>-Dfile.encoding=UTF-8</argLine><testFailureIgnore>false</testFailureIgnore><skipTests>true</skipTests></configuration></plugin><plugin><groupId>com.trivago.rta</groupId><artifactId>cucable-plugin</artifactId><version>1.5.1</version><executions><execution><id>generate-test-resources</id><phase>generate-test-resources</phase><goals><goal>parallel</goal></goals></execution></executions><configuration><!-- Required properties --><sourceRunnerTemplateFile>src/test/java/parallel/CucableJavaTemplate.java</sourceRunnerTemplateFile><sourceFeatures>src/test/resources/features</sourceFeatures><generatedFeatureDirectory>${project.build.directory}/parallel/features</generatedFeatureDirectory><generatedRunnerDirectory>${project.build.directory}/parallel/runners</generatedRunnerDirectory><!-- Optional properties --><numberOfTestRuns>1</numberOfTestRuns><includeScenarioTags>${ist}</includeScenarioTags><logLevel>default</logLevel><desiredNumberOfRunners/></configuration></plugin><plugin><groupId>com.trivago.rta</groupId><artifactId>cluecumber-report-plugin</artifactId><version>1.10.2</version><executions><execution><id>report</id><phase>post-integration-test</phase><goals><goal>reporting</goal></goals></execution></executions><configuration><expandBeforeAfterHooks>false</expandBeforeAfterHooks><expandStepHooks>false</expandStepHooks><expandDocStrings>true</expandDocStrings><customParameters><Cucuzzi>Infra Shared Service Tests Report</Cucuzzi></customParameters><sourceJsonReportDirectory>${project.build.directory}/cucumber-report</sourceJsonReportDirectory><generatedHtmlReportDirectory>${project.build.directory}/test-report</generatedHtmlReportDirectory></configuration></plugin><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-failsafe-plugin</artifactId><version>2.18</version><executions><execution><id>Run parallel tests</id><phase>integration-test</phase><goals><goal>integration-test</goal></goals></execution></executions><configuration><testFailureIgnore>false</testFailureIgnore><forkCount>15</forkCount><reuseForks>false</reuseForks><argLine>-Dfile.encoding=UTF-8</argLine><disableXmlReport>false</disableXmlReport></configuration></plugin><plugin><groupId>org.codehaus.mojo</groupId><artifactId>build-helper-maven-plugin</artifactId><version>1.12</version><executions><execution><id>add-test-source</id><phase>generate-test-sources</phase><goals><goal>add-test-source</goal></goals><configuration><sources><source>${project.build.directory}/parallel/runners</source></sources></configuration></execution></executions></plugin></plugins></build>
</profile>