Sometimes, for predictability and performance, we may be interested in
specifying the concrete resolver that a `ModuleID` should use.
This patch achieves this by adding a new field to `UpdateOptions` and
then getting this information from the `SbtChainResolver`, that will
select the concrete resolver for a given dependency descriptor.
Why is this useful? Well, two reasons:
* Predictable behaviour. We have the guarantee that an artifact only
comes from a concrete resolver.
* Resolution speedup. Around 1/3 or 1/2 times faster than normal
resolution in a moderate test case scenario. If there is a lot of
latency or network connection is poor, speedups will be higher.
LOGS:
```
NORMAL RESOLUTION TIME 1790
FASTER RESOLUTION TIME 1054
```
```
NORMAL RESOLUTION TIME 2078
FASTER RESOLUTION TIME 1055
```
Lots of projects can benefit from this option, as well as organizations
and companies. This will eventually integrate with the dependency lock
file, but can be used independently of it.
To ensure that SNAPSHOTs are always the latest, we go through all the
resolved modules and check their timestamps. Good.
However, if in the process of reparsing/redownloading the module
descriptor we fail (or it's not found in that resolver at all), then we
cannot refresh the resolved module that could have been internalized or
heavily cached in memory by ivy. We do this for correctness.
This patch does two things:
1. Adds more comments.
2. Warns only when there are parsing errors in ivy files.
3. Adds debug info in the rest of the cases.
This removes the pain of seeing `Unable to parse` that could be caused by
other reasons that are not related to parsing at all and which would not
affect the algorithm at hand. For instance, if we get a URLResource,
that's totally fine -- there is no way we could parse the ivy file. Ivy
uses URLResources in several cases where the artifact origin URL is
specified.
The frozen mode is used by the dependency lock file.
It makes sure that the resolution is always intransitive and that ivy
does not check for changed dependencies and stores that information in
the resolved ivy files. Following the ivy documentation, the last change
may bring a slight performance improvement:
http://ant.apache.org/ivy/history/latest-milestone/use/resolve.html
Some plugins remain commented out, for now.
sbt-doge is no longer needed because a variant of it has been folded
into sbt 1.
For some reason scripted requires src/doc jars of sbt, so switch back to
using `publishAll` rather than `publishLocalBinAll`. :(
Also, the sys.prop change in scripted is to force log4j2 to not use a
thread context classloader, and avoid the following:
ERROR StatusLogger Unable to create custom ContextSelector. Falling back to default.
java.lang.ClassCastException: Cannot cast org.apache.logging.log4j.core.async.AsyncLoggerContextSelector to org.apache.logging.log4j.core.selector.ContextSelector
at java.lang.Class.cast(Class.java:3369)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOf(LoaderUtil.java:201)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOfProperty(LoaderUtil.java:226)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.createContextSelector(Log4jContextFactory.java:97)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.<init>(Log4jContextFactory.java:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:94)
at org.apache.logging.log4j.spi.ThreadContextMapFactory.createThreadContextMap(ThreadContextMapFactory.java:73)
at org.apache.logging.log4j.ThreadContext.init(ThreadContext.java:223)
at org.apache.logging.log4j.ThreadContext.<clinit>(ThreadContext.java:202)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createDefaultInjector(ContextDataInjectorFactory.java:83)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createInjector(ContextDataInjectorFactory.java:67)
at org.apache.logging.log4j.core.lookup.ContextMapLookup.<init>(ContextMapLookup.java:34)
at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:117)
at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:125)
at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:46)
at org.apache.logging.log4j.core.layout.PatternLayout$Builder.build(PatternLayout.java:650)
at org.apache.logging.log4j.core.layout.PatternLayout.createDefaultLayout(PatternLayout.java:487)
at sbt.internal.util.ConsoleAppender.<init>(ConsoleAppender.scala:245)
at sbt.internal.util.ConsoleAppender$.apply(ConsoleAppender.scala:196)
at sbt.internal.util.ConsoleLogger.<init>(ConsoleAppender.scala:42)
at sbt.internal.util.ConsoleLogger$.apply(ConsoleAppender.scala:34)
at sbt.test.ScriptedRunner.run(ScriptedTests.scala:221)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at Scripted$.doScripted(Scripted.scala:125)
at $0483e89d182e9d240274$.$anonfun$scriptedTask$5(build.sbt:301)
I've tried to put together some scripted tests to remove the overhead of
compiling the whole sbt, which is around 3 minutes every time.
This new structure *should* make the scripted tests run faster.
Aside from this, we do some more tweaks:
* Increase memory.
* Fork processes to compile and run (to see if it makes a difference).
* Pass in the server flag to sbt.
Alexandre Archambault
Alexandre Archambault
xuwei-k
eugene yokota
jvican
Dale Wijnand