I was looking into sbt start up time and in profiling was able to
identify a number of classloading bottlenecks. To speed up
initialization, we can preload those classes in the background. I saw
average speedups of roughly .75 seconds after this change. Also, the `time`
command would consistently report cpu system time very close to 400% and
I have 4 cores on my laptop. With 1.3.0 it would be more like 350%.
There have been a number of complaints about the new classloader closing
behavior. It is too aggressive about closing classloaders after test and
run. This commit softens the behavior by allowing a classloader to be
resurrected after close by creating a new zombie classloader that has
the same urls as the original classloader. After this commit, we always
close the classloaders when we are done, but they can still leak
file descriptors if a zombie is created.
To configure the behavior, I add the allowZombieClassLoaders key. If it
is false (which is default), we will warn but still allow them. If it
is true, then we silence the warning. In a later version of sbt, we can
change the semantics to be strict.
I verified after this change that I could add a shutdown hook in `run`
and it would be evaluated so long as I set `bgCopyClasspath := false`.
Otherwise the needed jars were deleted before the hooks could run.
Bonus: delete unused ResourceLoaderImpl class
It takes about a second to load scala.reflect.runtime.universe. If we
lazy load here, we can load scala.relect.runtime.universe in the
background to speed up the sbt start up time. See
0ebb7a5662.
Fixes https://github.com/sbt/sbt/issues/5045
Currently we set `-XX:+UseParallelGC` for JDK greater than 9. This isn't a great default because while the peak throughput is somewhat better than the default g1 collector, the worst case performance is really bad with UseParallelGC, especially when the heap size is large.
Given the sudden diversification of JDK implementations, we should stay clear from `-XX` flags, and let the build users add them if they must.
Fixes#5070
This adds a new setting called `includePluginResolvers` (default `false`).
When set to `true`, it the project will include resolvers from the metabuild.
This allows the build user to declare a resolver in one place (`project/plugins.sbt`) that gets applied to both the metabuild as well as all the subprojects. The scenario comes up when someone distributes a software on their own repo. Ref #4103
We want to recursively monitor the project meta build, but we also want
to avoid listing directories that don't exist. To compromise, I rework
the buildSourceFileInputs to add the nested project directories if they
exist. Because the fileInputs are a setting, this means that adding a
new project directory and *.sbt or *.scala will not immediately trigger
a rebuild, but in most common cases, it will. I added a scripted test
for this.
In sbt 1.3.0, we only monitor build sources in the root project
directory and the root project meta build directory. This commit adds
these inputs for each project.
Fixes https://github.com/sbt/sbt/issues/5061.
This was an oversight that caused consoleQuick to not work with
supershell. We should probably try to figure out a way to allow custom
tasks to black list themselves from super shell reporting.
We have seen failures in scripted to create the output file for streams
and it has also been reported in https://github.com/sbt/sbt/issues/5067.
I believe this may caused by the same stream output being initialized by
multiple tasks. To fix this, I add locking on a per-file basis. There
was (and is) additional synchronization on the Streams _instance_, but
the per-file locks are stored in the Streams companion object so the
locking should be honored no matter which Streams instance calls make.
I also skip the call to IO.touch if the file already exists. I believe
that this is the common case and since IO.touch was being called with
setModified = false, it should be fine to skip the touch when the file
exists.
Prior to this change, I was able to induce the issue in #5067 in roughly
1/50 of calls to `scripted actions/cross-multiproject`. I wasn't able to
reproduce after this change.
In https://github.com/sbt/sbt/issues/5075 we realized that sbt 1.3.0
introduces a regression where it closes the classloader used to invoke
the main method for in process run before all of its non-daemon threads
have terminated. To fix this and still close the classloader, I add a
method, runInBackgroundWithLoader that provides the background job
services with an optional classloader that it can close after the job
completes.
This cleanly merges and works with 1.3.x as well.
https://github.com/sbt/sbt/issues/5075 reports that sbt 1.3.0 has a
regression for in process run. In sbt 1.3.0, we close the classloader
that invokes the main method before all of the daemon threads complete.
This adds a scripted test that fails due to this regression.
Fixes https://github.com/sbt/sbt/issues/5076. Arrays should be enclosed in
quotes, or otherwise elements with spaces will be broken, e.g. `-Dfoobar="foo
bar"` will become `-Dfoobar=foo` and `bar`.
Fixes https://github.com/sbt/sbt/issues/5047
When setting swoval.tmpdir via globalBase, changed to set globalBase as absolute path.
`com.swoval.runtime.NativeLoader.loadPackaged` uses `java.lang.System.load`.
It requires absolute path, so we should set `swoval.tmpdir` with absolute path.
Before 7992dc9, a custom, non-cached-resolution-aware update task was
used to generate the report that the tree is based on, effectively
ignoring the cached resolution flag at the project level.
Starting 7992dc9, this plugin, when run with sbt 0.13.8 or sbt 1.2.5+,
relies on cached-resolution-backed reports for projects that have
the engine enabled via `updateOptions`. Other 1.x releases are not
directly impacted as sbt had a buggy implementation of the feature
anyway, see https://github.com/sbt/sbt/issues/3761.
Cached resolution has the side effect of generating an ivy report
with artificial module descriptors which makes it hard to reconstruct
the tree without inlining sbt internals (see below), so this
effectively ignores it *for the purpose of the tree generation*, even
if the project enabled it for the regular report.
ModuleId(
org.scala-sbt.temp,
temp-resolve-e2a956132f02c038285b41b374c02f5838076f37,
1.0
)
https://github.com/sbt/librarymanagement/blob/984de6f/ivy/src/main/scala/sbt/internal/librarymanagement/ivyint/CachedResolutionResolveEngine.scala#L137
During refactoring, these warnings got out of date. I also added
scaladoc to the watchTriggeredMessage key.
Ref: https://github.com/sbt/sbt/issues/5051.
eugene yokota
Ethan Atkins
Anil Kumar Myla
Tuomas Lappeteläinen
Yusuke Yamada
Dmitrii Naumenko
Brice Jaglin
Josh Soref