This works around the name conflict between sbt.test package and sbt.Keys.test.
1. sbt.test package is renamed to sbt.scriptedtest. This allows 1.0 plugins and builds to use `test` to mean `Keys.test`.
2. To keep binary compatibility for sbt 0.13 scripted, I am adding `sbt.test.ScriptedRunner` and `sbt.test.ScriptedTests` in `scripted-plugin` artifact.
3. Another affected user is Giter8 plugin that uses ScriptedPlugin. Since the intereactions are limited to `sbt.ScriptedPlugin.*`, we should be fine here. - https://github.com/foundweekends/giter8/blob/v0.11.0-M2/plugin/src/main/scala-sbt-1.0/giter8/SBTCompat.scala
The change to enable batched and parallel execution for scripted was
done only for the scripted-sbt project. This pull request enables it for
scripted-plugin, so that all sbt plugins in 1.x. can benefit from it.
By default, it configures a number of parallel instances of 1 and batch
execution is disabled. Users can change the number of parallel sbt hosts
running scripted tests via the `scriptedParallelInstances` setting.
In some plugins scripted tests', batch execution can cause issues
because the first time `>` commands are executed they assume sbt starts
up. This error can be fixed by doing `reload` before running the `>`
command.
Note that the current scripted plugin does not allow parallel execution
in non-batched mode.
Some plugins remain commented out, for now.
sbt-doge is no longer needed because a variant of it has been folded
into sbt 1.
For some reason scripted requires src/doc jars of sbt, so switch back to
using `publishAll` rather than `publishLocalBinAll`. :(
Also, the sys.prop change in scripted is to force log4j2 to not use a
thread context classloader, and avoid the following:
ERROR StatusLogger Unable to create custom ContextSelector. Falling back to default.
java.lang.ClassCastException: Cannot cast org.apache.logging.log4j.core.async.AsyncLoggerContextSelector to org.apache.logging.log4j.core.selector.ContextSelector
at java.lang.Class.cast(Class.java:3369)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOf(LoaderUtil.java:201)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOfProperty(LoaderUtil.java:226)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.createContextSelector(Log4jContextFactory.java:97)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.<init>(Log4jContextFactory.java:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:94)
at org.apache.logging.log4j.spi.ThreadContextMapFactory.createThreadContextMap(ThreadContextMapFactory.java:73)
at org.apache.logging.log4j.ThreadContext.init(ThreadContext.java:223)
at org.apache.logging.log4j.ThreadContext.<clinit>(ThreadContext.java:202)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createDefaultInjector(ContextDataInjectorFactory.java:83)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createInjector(ContextDataInjectorFactory.java:67)
at org.apache.logging.log4j.core.lookup.ContextMapLookup.<init>(ContextMapLookup.java:34)
at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:117)
at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:125)
at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:46)
at org.apache.logging.log4j.core.layout.PatternLayout$Builder.build(PatternLayout.java:650)
at org.apache.logging.log4j.core.layout.PatternLayout.createDefaultLayout(PatternLayout.java:487)
at sbt.internal.util.ConsoleAppender.<init>(ConsoleAppender.scala:245)
at sbt.internal.util.ConsoleAppender$.apply(ConsoleAppender.scala:196)
at sbt.internal.util.ConsoleLogger.<init>(ConsoleAppender.scala:42)
at sbt.internal.util.ConsoleLogger$.apply(ConsoleAppender.scala:34)
at sbt.test.ScriptedRunner.run(ScriptedTests.scala:221)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at Scripted$.doScripted(Scripted.scala:125)
at $0483e89d182e9d240274$.$anonfun$scriptedTask$5(build.sbt:301)
I've tried to put together some scripted tests to remove the overhead of
compiling the whole sbt, which is around 3 minutes every time.
This new structure *should* make the scripted tests run faster.
Aside from this, we do some more tweaks:
* Increase memory.
* Fork processes to compile and run (to see if it makes a difference).
* Pass in the server flag to sbt.
This commit makes sure that between every run in batch mode the local
root project name is updated if no name is set.
This fixes project/generated-root-no-project that was assuming that the
root directory name was `generated-root-no-project`. This invariant does
not hold anymore with the batch-mode. Now one sbt dir is shared for lots
of scripted tests.
* Remove MaxPermSize from another scripted opts
* Reduce memory of sbt host to 1g instead of 2g
* Add Xms java options to scripted
* Enable parallelism with 512M for sbt tests
* Parallelism + batch mode
For that, we:
* Change the existing infrastructure to recycle as much code as
possible.
* Use `BatchScriptRunner` since `ScriptRunner` is too restrictive to
programmatically control the underlying sbt servers.
* Unify `TestRunner` to the more general way for both batch and
non-batch modes.
The following commit minimises the amount of exceptions that leak
through our internal signatures. It thus changes the return type
signature of `scriptedTest` to return an `Option` instead of `Unit`.
At the call-site of `scriptedTest`, we also remove the code to handle
the exceptions thrown by it.
Add binary-compatible methods to run scripted tests in parallel using
parallel collections. Parallel collections are not tuned and a default
of 4 cores is used for the `ForkJoinPool`. This can be a matter of
experimentation in the future but for now I prefer to stay with it.
The non-parallel methods are not affected by this change.
The changes to the scripted plugin have not been used, only the sbt
scripted plugin is using `runInParallel`. In the future, this change can
also happen so that plugin authors can benefit from parallel sbt
testing. This is not a priority for me now :)
We don't have to care about migrations in these removals because no one
should be depending on these methods directly. People consume scripted
via the sbt plugin.
This commit also removes the unused `CompatibilityLevel` which is
completely outdated, making reference to Scala 2.9 et al.
To pass File => Unit callback across the classloader boundary
I am encoding it as a java.util.List[File] by overriding
method.
This was needed since Java didn't allow me to cast
from one classloader to the other.
Dale Wijnand
Eugene Yokota
Allan Renucci
jvican
Simon Schäfer
Jonas Fonseca
Martin Duhem
Pierre DAL-PRA