It is possible with the new layering strategies that tests may fail if a
java package private class is accessed across classloader layers. This
will result in an IllegalAccessError that is hard to debug. With this
commit, I add an error message that will be displayed if run throws an
IllegalAccessError that suggests that the user try the
ScalaInstance layering strategy or the flat layering strategy.
In code review, @eed3si9n suggested that I switch to a more verbose and
descriptive naming scheme. In addition to trying to make layers more
descriptive, I also made the various layer case objects extend the
relevant layers so it's more clear what the layer should look like.
Using the data structures that I added in the previous commits, it is
now possible to rework the run and test task to use (configurable)
layered class loaders. The layering strategy is globally set to
LayeringStrategy.Default. The default strategy leads to what is
effectively a three layered ClassLoader for the both the test and run
tasks. The first layer contains the scala instance (and test framework
loader in the test task). The second layer contains all of the
dependencies for the configuration while the third layer contains the
project artifacts.
The layering strategy is very easily changed both at the Global or
Configuration level, e.g. adding
Test / layeringStrategy := LayeringStrategy.Flat
to the project build.sbt will make the test task not even use the scala
instance and instead a create a single layer containing the full
classpath of the test task.
I also tried to ensure that all of the ClassLoaders have good toString
overrides so that it's easy to see how the ClassLoader is constructed
with, e.g. `show testLoader`, in the sbt console.
In this commit, the ClassLoaderCache instances are settings. In the next
commit, I make them tasks so that we can easily clear out the caches
with a command.
It is possible for the test task to fail because of issues with the
layered ClassLoaders. When this occurs and is detectable, I try to
provide a useful error message that will help the user fix the issue.
I had to turn off -Xfatal-warnings in commandProj because after updating
io, commandProj depends on the deprecated EventMonitor class. In #4335,
I stop using EventMonitor, but deprecate the Watched class which is both
defined and used (as an unused attribute key) in commandProj. I think we
can probably get rid of Watched in 1.4.x and certainly in a hypothetical
2.x, so hopefully we can restore -Xfatal-warnings sooner than later.
I also had to replace uses of IO.classLocationFile with
IO.classLocationPath to avoid compilation failures due to
-Xfatal-warnings.
I spent a lot of time debugging why it took 5 seconds to run tests each
time. It turns out that if the hostname is not set explicitly on os x,
then getaddrinfo takes 5 seconds to try (and fail) to resolve the dns
entry for the localhostname. This is easily fixed by setting the
hostname, but it is not at all easy to figure out that a slow hostname
lookup is the reason why tests are slow to start.
I don't know if this is a common issue on other platforms, so only issue
the warning on OS X.
ArrayList::add is not thread safe. I ran into cases where async tests
using utests would fail even when all of the individual tests passed.
This was because multiple threads called back into the handle method of
the handler instance variable, which just delegated to eventList::add.
When this happened, one of the events would get added to the list as a
null reference, which would manifest as an NPE upstream on the master
process. After this change, my tests stopped failing.
java.lang.Class#newInstance deprecated since Java 9
http://download.java.net/java/jdk9/docs/api/java/lang/Class.html#newInstance--
```
Deprecated. This method propagates any exception thrown by the nullary constructor, including a checked exception. Use of this method effectively bypasses the compile-time exception checking that would otherwise be performed by the compiler. The Constructor.newInstance method avoids this problem by wrapping any exception thrown by the constructor in a (checked) InvocationTargetException.
The call
clazz.newInstance()
can be replaced by
clazz.getDeclaredConstructor().newInstance()
The latter sequence of calls is inferred to be able to throw the additional exception types InvocationTargetException and NoSuchMethodException. Both of these exception types are subclasses of ReflectiveOperationException.
Creates a new instance of the class represented by this Class object. The class is instantiated as if by a new expression with an empty argument list. The class is initialized if it has not already been initialized.
```
Fixes#2198 Ref #2854
Generated Junit-style XML reports now include a count of ignored,
skipped and pending tests; and individual tests are correctly flagged
with the <skipped/> element.
The issue is that when you manually set a ScalaInstance, i.e. not one from Ivy, the
classpath which is returned for any given configuration ONLY uses Ivy. This means that
the legitimate Scala JAR files that need to be on the classpath are missing from the list.
For some reason, the way we instantiate tests uses an unfiltered classloader against the
ScalaInstance, *BUT* the thread-context-classloader DOES use a filtered instance by
classpath. This add the hook into the TestFramework runner creation so that
the classpath accurately reflects the jars needed.
cc @rkuhn
Taken from https://github.com/chenkelmann/junit_xml_listener and slightly improved.
Activating is now just a matter of setting `testReportJUnitXml` to true.
This not only brings a very handy feature (hudson/jenkins support that syntax),
but gives more options for testing the SBT test infrastructure.
For instance checking that the test output is well segregated during a parallel
test run is just a matter of checking the report output.
Conflicts:
main/src/main/scala/sbt/Keys.scala
This feature is not activated by default. To enable it set `testForkedParallel` to `true`.
The test-agent then executes the tests in a thread pool.
For now it has a fixed size set to the number of available processors.
The concurrent restrictions configuration should be used.
This protects against deadlocks between the writing and reading end,
since the ObjectOutputStream constructor writes a header, but does not
flush, and the ObjectInputStream constructor reads the header, and
blocks until it's read.
This drops older code that picked a single fingerprint within a framework.
This commit ensures only one name is executed across all frameworks.
The precedence is the order that frameworks are declared in Defaults.scala.
JUnit should be last because it is common for specs tests to be annotated
with @RunWith for Eclipse support. If a distinct fingerprint is not
picked, the test will be run by both specs and JUnit.
In addition to this main reason, result processing code assumes one result
per test class name and would have to be fixed if multiple fingerprints
were allowed.
Eugene Yokota
Ethan Atkins
Ignasi Marimon-Clos
xuwei-k
Daniel Riquelme
Tim Harper
Dale Wijnand
OlegYch
Jason Zaugg
Simon Schäfer
Răzvan Flavius Panda
James Koch
Ashley Mercer
Martin Duhem
Pierre DAL-PRA
Kamil Kloch
Allan Erskine
Jean-Rémi Desjardins
Josh Suereth
Peter Vlugter
cheeseng
Bruno Bieth
Mark Harrah
James Roper