Currently SBT is unable to run tests in parallel while providing a meaningful trace:
test outputs are interleaved. This is especially annoying when a test pass on
your machine but breaks on a continuous integration server.
This commit allows running forked tests in multiple VM, enabling test output segregation.
In order to report test outputs a refactoring of `TestReportListener` was necessary,
breaking compatibility with 0.13.
In addition SBT provides now a reference implementation of the new `TestReportListener`,
formatting test reports in JUnit XML.
Those features are disabled by default and are activated with the following settings:
* `testNumberForkedJvm` : controls how many VM are used to run the tests
Note: this is reduced if there are fewer tests than vm
Note: forking must be enabled (`fork in test := true`)
* `testHideSuccessfulOutput` : when set to true, do not print tests output on the console
Note: this does not work in non-forked mode
* `testReportJUnitXml` : when set to true produce reports in the JUnit XML format
Note: in non-forked mode test outputs are not captured
Meaningful (debug level) logs are produced for each of these features.
This feature is not activated by default. To enable it set `testForkedParallel` to `true`.
The test-agent then executes the tests in a thread pool.
For now it has a fixed size set to the number of available processors.
The concurrent restrictions configuration should be used.
This protects against deadlocks between the writing and reading end,
since the ObjectOutputStream constructor writes a header, but does not
flush, and the ObjectInputStream constructor reads the header, and
blocks until it's read.
This drops older code that picked a single fingerprint within a framework.
This commit ensures only one name is executed across all frameworks.
The precedence is the order that frameworks are declared in Defaults.scala.
JUnit should be last because it is common for specs tests to be annotated
with @RunWith for Eclipse support. If a distinct fingerprint is not
picked, the test will be run by both specs and JUnit.
In addition to this main reason, result processing code assumes one result
per test class name and would have to be fixed if multiple fingerprints
were allowed.
-Changed usages and implementations of interfaces in org.scalatools.testing._ to use/implement interfaces/classes in sbt.testing._ instead.
-Added sbt.testing to interfaceFilter in TestFramework.createTestLoader method to enable loading of classes in sbt.testing package.
-Added FrameworkWrapper.java to wrap old framework implementations.
-Added code in ForkMain.java to serialize Selectors.
It is now used for consoleProject, run, and test. This loader verifies
that all classes loaded through it came from a particular classpath or
from the "root" loader. Root loader here is the launcher loader so that
those classes with native bindings come from that shared loader.
- Requires selecting one fingerprint if there are more than one
- The selected fingerprint is by reverse ordering of Framework.tests
- Reverse ordering chosen to work with junit-interface without changes
Bruno Bieth
Mark Harrah
James Roper
cheeseng
Paolo G. Giarrusso
Grzegorz Kossakowski
Eugene Vigdorchik
Eugene Vigdorchik