Fixessbt/sbt#2699
Before:
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
[warn] * com.google.code.findbugs:jsr305:2.0.1 -> 3.0.0
[warn] Run 'evicted' to see detailed eviction warnings
After:
[warn] Found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[warn]
[warn] * com.typesafe.akka:akka-actor_2.12:2.5.0 is selected over 2.4.17
[warn] +- de.heikoseeberger:akka-log4j_2.12:1.4.0 (depends on 2.5.0)
[warn] +- com.typesafe.akka:akka-parsing_2.12:10.0.6 (depends on 2.4.17)
[warn] +- com.typesafe.akka:akka-stream_2.12:2.4.17 () (depends on 2.4.17)
[warn]
[warn] Run 'evicted' to see detailed eviction warnings
Dotty is versioned as 0.*, but `CrossVersionUtil#binaryScalaVersion`
will return the full version instead of just `major.minor` for all
compiler versions < 2.10, add a special case for Dotty to avoid this.
`UpdateOptions` is only meant for user-provided options. Since this
option should not be exposed to users, it's moved to inline ivy
configuration, whose parameters require information about the build and
is usually defined project-wide instead of `Global`-wide.
Managed checksums tells ivy to forget about checking checksums for jar
files and just persist them in the cache.
The user that enables that option will take care of verifying they are
correct.
This is a big deal because:
1. Ivy takes *a lot of time* checking timestamps for big jars, and does
it sequentially. The user (sbt) can do better by running these checks
in parallel, speeding up the whole resolution process!
2. The fact that the sha files are not present in the cache means that
build tools cannot check if a checksum is correct without preprocessing
the jar.
Note that the user is responsible for keeping the cache consistent. If
the checksum is not correct, the user should report it and overwrite the
checksum file with the valid digest.
Sometimes, for predictability and performance, we may be interested in
specifying the concrete resolver that a `ModuleID` should use.
This patch achieves this by adding a new field to `UpdateOptions` and
then getting this information from the `SbtChainResolver`, that will
select the concrete resolver for a given dependency descriptor.
Why is this useful? Well, two reasons:
* Predictable behaviour. We have the guarantee that an artifact only
comes from a concrete resolver.
* Resolution speedup. Around 1/3 or 1/2 times faster than normal
resolution in a moderate test case scenario. If there is a lot of
latency or network connection is poor, speedups will be higher.
LOGS:
```
NORMAL RESOLUTION TIME 1790
FASTER RESOLUTION TIME 1054
```
```
NORMAL RESOLUTION TIME 2078
FASTER RESOLUTION TIME 1055
```
Lots of projects can benefit from this option, as well as organizations
and companies. This will eventually integrate with the dependency lock
file, but can be used independently of it.
The frozen mode is used by the dependency lock file.
It makes sure that the resolution is always intransitive and that ivy
does not check for changed dependencies and stores that information in
the resolved ivy files. Following the ivy documentation, the last change
may bring a slight performance improvement:
http://ant.apache.org/ivy/history/latest-milestone/use/resolve.html
The previous custom offline implementation was not working on 100% of
the cases and relied on the TTL of ivy. As the previous commit
enabled the native offline implementation provided by ivy as of 2.3.0,
this functionality is not useful anymore.
The current place to specify offline is `UpdateConfiguration`, and not
`InlineIvyConfiguration` that is required to instantiate sbt. With the
current approach, we can be online or offline without having to
instantiate ivy sbt twice.
I will provide a Scalafix rewrite for this change.
The following commit tries to address the well-known issue that sbt
cannot be used in offline mode. In order to enable that use case, this
commit adds support for a flag in update configuration called `offline`
that users can change as they wish (and that sbt will expose via
settings).
It adds tests to check that the resolution uses the caches instead of
trying to resolve from the Internet. Unfortunately, ivy does not expose
a way to know whether a resolution was made from the cache or the
Internet, so the test case invents a metric to check that resolution
indeed happens from cache.
In order to benefit from this 100%, we need to update to ivy 2.4.0 or
cherry-pick a commit because a major issue in `useCacheOnly` has been
fixed: https://issues.apache.org/jira/browse/IVY-1515.
In short, this is good for the dependency lock file too. Since we can
make sure that once we have downloaded and resolved all the dependencies
locally, we do resolve from the cache.
The following PR does two things:
* Removes the unnecessary `SbtExclusionRule` that was introduced to
exclude artifacts at the project level (and not the dependency level).
This change was done in an independent class to avoid breaking
bincompat in 0.13.x series.
* Handle exclusion rules correctly, propagating the cross version to the
exclusions of the dependencies.
To fixsbt/sbt#1518, this PR takes the avenue taken in
`SbtExclusionRule`, it accepts `GroupArtifactID` which should be the
preferred way to specify dependencies from now on. Unlike
`SbtExclusionRule`, it also supports `ModuleID` for those that want to
exclude a concrete dependency.
`InclExcl` did not have any tests. The following commit also adds a
testing suite for it, showing how the issue it's fixed and how you
should use `exclude` if you're calling directly `ExclusionRule` instead
of passing in `GroupArtifactID` and `ModuleID`.
I'm now going to use `CacheStore.apply` in `JsonUtil` (used by cached resolution). This gets rid of `fileToStore` parameter from a bunch of classes and simplifies the setup around librarymanagement.
Previously, when the dependency resolver (Ivy) encountered a Maven version range such as `[1.3.0,)`
it would go out to the Internet to find the latest version.
This would result to a surprising behavior where the eventual version keeps changing over time
*even when there's a version of the library that satisfies the range condition*.
This changes to some Maven version ranges would be replaced with its lower bound
so that when a satisfactory version is found in the dependency graph it will be used.
You can disable this behavior using the JVM flag `-Dsbt.modversionrange=false`.
Fixes#2954
Ref #2291 / #2953
Fixes #2002/#1500
Given a dependency graph such as:
libraryDependencies += "com.google.guava" % "guava-tests" % "18.0"
libraryDependencies += "com.google.guava" % "guava-tests" % "18.0"
% "test" classifier "tests"
previous releases of sbt would drop the Test configuration from the
classifier "tests" artifacts, and end up including the test JARs into
the Compile configuration instead of the Test configuration, which
would result in runtime error.
This fix configures the explicit artifacts into the configuration
during merge even when it says `"*"`.
Two instance of functions defined in the same way don't equal themselves, but
the same instance of a function does. So by using a val idStringFun for full and
binary, and making Binary and Full case classes there are much better chances
that ModuleId's defined in the same way will be equal.
This is a combination of 13 commits. I squashed these 13 commits to make
forward porting those changes easier, since some commit undo the changes
of other commits.
The PRs that include these changes can be found at
https://github.com/sbt/sbt/pull/2564 and
https://github.com/sbt/sbt/pull/2576.
Static launcher, get bridge sources from resources
This commit introduces a new "static" launcher that does not use Ivy to
gather all the artifacts that it requires, but rather expect them to be
immediately available.
To be able to use sbt without Internet access, we add a new
`ComponentCompiler` that is able to retrieve the bridge sources from the
resources on classpath and compile it.
Fix classpath issues in static launcher
The launcher defines a top classloader that willbe used by all
`ScalaInstance`s. Previously, this top classloader had a parent that
contained the scala library 2.10, which prevented the correct
compilation of the compiler bridge for scala 2.11.
Also, we no longer need the scala-reflect JAR.
Tests for FakeResolver
Add `scala-reflect.jar` to JARs of `StaticScalaProvider`
It turns out we need to have `scala-reflect.jar` on classpath to compile
the compiler bridge for the static scala instance of the launcher.
Comply to Ivy's specification in `FakeResolver`
Remove `CompilerBridgeProvider` and `ResourceBridgeProvider`
It turns out that we can leverage the`FakeResolver` that has been
implemented to use with the static launcher, and resolve a "fake
compiler bridge" using it, rather than copying it from the resources.
This also has the advantage of not requiring to change the build
definition.
Fix NPE in FakeResolver
Add compiler bridge sources to fake resolver
This allows sbt to resolve the compiler bridge sources when using the
static launcher
Don't hardcode sbt version in static launcher
Add scala compiler and library to fake resolver
This allows us to still resolve them if we have no other resolver
configured.
Add `RepositoriesParser`
This parser is used by the static launcher to parse the definition of
resolvers that override the build resolvers.
Support repositories override in static launcher
The static launcher will now parse user-defined repositories like the
usual launcher does.
Specifically, the static launcher now uses the following configuration:
- `sbt.boot.directory`: specifies the boot directory that sbt will use.
Defaults to `~/.sbt/boot`.
- `sbt.override.build.repos`: indicate whether we want to override the
build resolvers. Defaults to false.
- `sbt.repository.config`: specifies the path to the files that
contains repositories definition. Defaults to
`${sbt.boot.directory}/repositories`.
Notes for sbt/sbt#2564 & sbt/sbt#2576
(This is a port of sbt/sbt#2258)
sbt was reporting warning abouts inconsistent versions of dependencies
even if these dependencies didn't have the same configuration (as in
`provided` vs `compile`).
This commit fixes this problem by comparing the dependencies by
organization, artifact name and configuration.
Undeprecated `CustomPomParser` because it looks like nothing has
replaced it yet.
Removed usage of trait `scala.NotNull`, because it has been deprecated
as of Scala 2.11 and its support has never been implemented in scalac.
Eugene Yokota
Dale Wijnand
Guillaume Martres
jvican
Miles Sabin
Martin Duhem
Dan Sanduleac
Eugene Platonov