Sometimes, for predictability and performance, we may be interested in
specifying the concrete resolver that a `ModuleID` should use.
This patch achieves this by adding a new field to `UpdateOptions` and
then getting this information from the `SbtChainResolver`, that will
select the concrete resolver for a given dependency descriptor.
Why is this useful? Well, two reasons:
* Predictable behaviour. We have the guarantee that an artifact only
comes from a concrete resolver.
* Resolution speedup. Around 1/3 or 1/2 times faster than normal
resolution in a moderate test case scenario. If there is a lot of
latency or network connection is poor, speedups will be higher.
LOGS:
```
NORMAL RESOLUTION TIME 1790
FASTER RESOLUTION TIME 1054
```
```
NORMAL RESOLUTION TIME 2078
FASTER RESOLUTION TIME 1055
```
Lots of projects can benefit from this option, as well as organizations
and companies. This will eventually integrate with the dependency lock
file, but can be used independently of it.
To ensure that SNAPSHOTs are always the latest, we go through all the
resolved modules and check their timestamps. Good.
However, if in the process of reparsing/redownloading the module
descriptor we fail (or it's not found in that resolver at all), then we
cannot refresh the resolved module that could have been internalized or
heavily cached in memory by ivy. We do this for correctness.
This patch does two things:
1. Adds more comments.
2. Warns only when there are parsing errors in ivy files.
3. Adds debug info in the rest of the cases.
This removes the pain of seeing `Unable to parse` that could be caused by
other reasons that are not related to parsing at all and which would not
affect the algorithm at hand. For instance, if we get a URLResource,
that's totally fine -- there is no way we could parse the ivy file. Ivy
uses URLResources in several cases where the artifact origin URL is
specified.
The frozen mode is used by the dependency lock file.
It makes sure that the resolution is always intransitive and that ivy
does not check for changed dependencies and stores that information in
the resolved ivy files. Following the ivy documentation, the last change
may bring a slight performance improvement:
http://ant.apache.org/ivy/history/latest-milestone/use/resolve.html
This is the most important part of the sbt ivy management and it's
almost unreadable in the previous shape...
This attempts to make a rewrite without any semantic change.
The previous custom offline implementation was not working on 100% of
the cases and relied on the TTL of ivy. As the previous commit
enabled the native offline implementation provided by ivy as of 2.3.0,
this functionality is not useful anymore.
The current place to specify offline is `UpdateConfiguration`, and not
`InlineIvyConfiguration` that is required to instantiate sbt. With the
current approach, we can be online or offline without having to
instantiate ivy sbt twice.
I will provide a Scalafix rewrite for this change.
The following commit tries to address the well-known issue that sbt
cannot be used in offline mode. In order to enable that use case, this
commit adds support for a flag in update configuration called `offline`
that users can change as they wish (and that sbt will expose via
settings).
It adds tests to check that the resolution uses the caches instead of
trying to resolve from the Internet. Unfortunately, ivy does not expose
a way to know whether a resolution was made from the cache or the
Internet, so the test case invents a metric to check that resolution
indeed happens from cache.
In order to benefit from this 100%, we need to update to ivy 2.4.0 or
cherry-pick a commit because a major issue in `useCacheOnly` has been
fixed: https://issues.apache.org/jira/browse/IVY-1515.
In short, this is good for the dependency lock file too. Since we can
make sure that once we have downloaded and resolved all the dependencies
locally, we do resolve from the cache.
This commit reduces the code duplication in `updateEither` which was
duplicating a good deal of the resolution logic to deal with the
different resolution mechanisms: the simple one and the cached one.
It also unifies the signatures of the helpers that are invoked by
`updateEither`, removing the weirdness of the different return type
signatures and ad-hoc logic handling.
`cleanCachedResolutionCache` and `clean` were doing extra work because
`clean` does only need to clean the cache in the configuration (that is
a class parameter).
This is a port of https://github.com/sbt/sbt/pull/2992.
Original description of the feature:
```
Ivy downloads have traditionally been single-threaded.
Parallel downloads are a must for a modern build tool. This commit
builds upon the work done by Josh Suereth in the branch
sbt/ivy-parallel-download-artifact.
To avoid adding external dependencies, it uses the Scala parallel
collections. If maintainers consider that is worth it to use a more
modern and appropriate approach, like Task, I'm happy to reimplement
the features with it.
```
This commit does not preserve Josh's metadata in the commit since the
whole design of the repository has changed and I did not know how to
import a commit from sbt/sbt. However, it does apply several changes to
the original PR.
Co-authored-by: Josh Suereth <joshua.suereth@gmail.com>
The following PR does two things:
* Removes the unnecessary `SbtExclusionRule` that was introduced to
exclude artifacts at the project level (and not the dependency level).
This change was done in an independent class to avoid breaking
bincompat in 0.13.x series.
* Handle exclusion rules correctly, propagating the cross version to the
exclusions of the dependencies.
To fixsbt/sbt#1518, this PR takes the avenue taken in
`SbtExclusionRule`, it accepts `GroupArtifactID` which should be the
preferred way to specify dependencies from now on. Unlike
`SbtExclusionRule`, it also supports `ModuleID` for those that want to
exclude a concrete dependency.
`InclExcl` did not have any tests. The following commit also adds a
testing suite for it, showing how the issue it's fixed and how you
should use `exclude` if you're calling directly `ExclusionRule` instead
of passing in `GroupArtifactID` and `ModuleID`.
I'm now going to use `CacheStore.apply` in `JsonUtil` (used by cached resolution). This gets rid of `fileToStore` parameter from a bunch of classes and simplifies the setup around librarymanagement.
Previously, when the dependency resolver (Ivy) encountered a Maven version range such as `[1.3.0,)`
it would go out to the Internet to find the latest version.
This would result to a surprising behavior where the eventual version keeps changing over time
*even when there's a version of the library that satisfies the range condition*.
This changes to some Maven version ranges would be replaced with its lower bound
so that when a satisfactory version is found in the dependency graph it will be used.
You can disable this behavior using the JVM flag `-Dsbt.modversionrange=false`.
Fixes#2954
Ref #2291 / #2953
Dotty nightly builds are published to maven, so they end up in
configuration "default", not "compile". We still need to look into
"compile" when dotty is published locally.
You can test this using https://github.com/smarter/dotty-example-project
by updating the sbt version used in project/build.properties and by
replacing "0.1.1-SNAPSHOT" by a nightly version like
"0.1.1-20170109-be64643-NIGHTLY" in build.sbt
The old way to customize CrossVersion results was a `String => String`
function called 'remapVersion', removed in
301ec787f2.
That was removed because it's not possible to serialise Function1s, and
we want to serialise CrossVersion (and therefore ModuleID, etc) to be
able to transfer them in sbt server.
This commit reintroduces a less-powerful way to vary the results of
apply CrossVersion, but just providing the opportunity to define a
suffix. From looking at the users of CrossVersion (Scala.js'
sbt-scalajs-plugin & Scala Native's sbt-crossproject) this looks to be
sufficient.
Fixes#1466 Ref #2786
Even after fixing the mediator issue, we still have spurious binary
version conflict warning that does not account for sandbox
configurations.
This change follows the scalaVersionConfigs work.
Fixes#2786. Ref #2634.
sbt 0.13.12 added Ivy mediator that enforces scalaOrganization and
scalaVersion for Scala toolchain artifacts.
This turns out to be a bit too aggressive because Ivy configurations
can be used as an independent dependency graph that does not rely on
the scalaVersion used by Compile configuration. By enforcing
scalaVersion in those graph causes runtime failure.
This change checks if the configuration extends Default, Compile,
Provided, or Optional before enforcing scalaVersion.
Eugene Yokota
jvican
Dale Wijnand
Guillaume Martres