It turns out that it can take roughly one second to instantiate a
scala.nsc.tools.Global instance for the first time. When sbt is starting
up, it also takes nearly 2 seconds to initialize logging. We can speed
up the boot time by doing these two things concurrently. On my machine,
I saw on average a 500ms decrease in startup time after this change.
I finally realized that the trick is that for non cygwin windows, the
available method on the jline wrapped input stream always returns zero.
Unlike on posix, however, the read method is interruptible which means
that we can just spin up a background thread that polls from the input
stream and writes it into a buffer.
I verified that it was no longer necessary to hit <enter> after 'r' to
rerun the continuous command on my windows vm after this change.
Given that there are io differences between windows and posix systems,
we should aim to run the tests that do a lot of io on windows. There are
a few tests that don't work because of some platform specific issues so
I added a filter that excludes these tests on windows in ScriptedTests.
This commit finally fixes#241 by adding support for sbt to either
print a warning or automatically reload the project if the metabuild
sources have changed. To facilitate this, I introduce a new key,
metaBuildSourceOption which has three options:
1) IgnoreSourceChanges
2) WarnOnSourceChanges
3) ReloadOnSourceChanges
When the former is set, sbt will not check if the meta build sources
have changed. Otherwise, sbt will use the buildStructure / fileInputs to
get the ChangedFiles for the metabuild. If there are any changes, it
will either warn or reload the build depending on the value of
metaBuildSourceOption.
The mechanism for diffing the files is that I add a step to EvaluateTask
where, if the project has been loaded and
metaBuildSourceOption != IgnoreSourceChanges, we evaluate the needReload
task. If we need a reload, we return an error that indicates that a
Reload is necessary. When that error is detected, the MainLoop will
prepend "reload" to the pending commands for the state. Otherwise we
just print a warning and continue.
I benchmarked the overhead of this and it wasn't too bad. I generally
saw it taking 5-20ms to perform the check. Since this is only done once
per task evaluation run, I don't think it's a big deal. When
IgnoreSourceChanges is set, there is O(10us) overhead. If performance
does become a problem, we could add a global watch service and skip the
needReload evaluation if no files have been modified.
I removed the watchTrackMetaBuild key and made it so that the continuous
builds only track the meta build when
metaBuildSourceOption == ReloadOnSourceChanges
The persistentFileStampCache does seem to work pretty well but in case
users encounter issues, I add a boolean flag that allows the user to
turn this behavior off and always re-stamp every source file in every
task evaluation run.
Previously the persistent attribute map was only reset when the file
event monitor detected a change. This made it possible for the cache to
be inconsistent with the state of the file system*. To fix this, I add an
observer on the file tree repository used by the continuous build that
invalidates the cache entry for any path for which it detects a change.
Invalidating the cache does not stamp the file. That only happens either
when a task asks for the stamp for that file or when the file event
monitor reports an event and we must check if the file was updated or
not.
After this change, touching a source file will not trigger a build
unless the contents of the file actually changes.
I added a test that touches one source file in a project and updates the
content of the other. If the source file that is only ever touched ever
triggers a build, then the test fails.
* This could lead to under-compilation because ExternalHooks would not
detect that the file had been updated.
I had tried to be cute and only inject certain tasks if they're actually
used, but that made it so that dynamic tasks may not have be able to use
them.
The new io verion removes the PathFinder <-> Glob implicit translations.
It also has a number of small bug fixes related to directory listing via
FileTreeView.
The main project emits a number of deprecation warnings. I've isolated
the deprecation warnings related to Watch to the DeprecatedContinuous
file. I fixed the deprecation warnings where it was straightforward to
do so. After this change, there are three non-watch related changes
emitted:
1) Defaults.scala:3760 uses the deprecated InputTask.apply. This seems
fixable but I'm not in a hurry
2) oldLoadFailed and oldLastGrep are used by Main. I think this could
just be fixed by removing the deprecation warnings and setting them
private[sbt] since they will still be available in the shell.
I previously tried to fix https://github.com/sbt/sbt/issues/4608 in
fc715cab44 by finding the instance of
xsbt.boot.BootFilteredLoader in the classloader heirarchy. This was a
risky approach since it made a lot of assumptions about the classloaders
used to invoke xMain.run. Since the point is to filter out the scala
standard library jar, I reworked things to just find all the parents of
the scala provider loader and then walk the graph from the root
classloader until it finds the classloader that contains the scala
library. If no such classloader exists, it ends up returning the parent
of the scala provider library.
I also renamed the libraryLoader parameter to scalaProviderLoader since
that is what is actually passedin. It is actually the libraryLoader that
we want to exclude.
Ref https://github.com/sbt/sbt/issues/4661
local-preloaded-ivy contains dangling ivy.xml without JAR files.
We might include local-preloaded again once we have a preloaded in Maven layout.
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler classpath.
Previously the zincLmIntegrationProj would compile if the meta build classpath
leaked into the compilation classpath. I found that the project would
not compile a clean build running an sbt built off of origin/develop.
I discovered there were a number of places where closing a ClassLoader
didn't work correctly because I was assuming it was a URLClassLoader
when it was actually a ClasspathFilter. I also incorrectly imported the
wrong kind of URLClassLoader in Run.scala. Finally, I close the
SbtMetaBuildClassLoader when xMain exits now.
I noticed that when I ran scripted locally (with no arguments) it was
hard to tell if they succeeded or not because the last test it ran was
metadata-only-resolver which had no valid test files and dumped a bunch
of lines to stderr. I noticed there were a number of tests that had
files named disabled but no other tests with a test named disable so I
renamed metadata-only-resolver to disabled..
Ethan Atkins
eugene yokota
xuwei-k
Dale Wijnand