Supershell was not reliably working and I tracked it down to
TaskProgress not actually publishing updates during task execution. This
seemed to happen because the background task was only run once when the
task started up. Once that task exited, no further task reports would be
published. The fix is to start a new thread every time we enter
EvaluateTask. I verified manually that it did not seem to leak threads
because EvaluateTask always calls shutdown, which calls
afterAllCompleted, which stops the progress thread.
I also decreased the default report period to 100ms. I can't imagine
that this will have a big effect on performance. It can be tuned with
the sbt.supershell.sleep parameter.
There are issues when using jdk > 8 where the rt.jar file can be
invalidated by ExternalHooks. This causes spurious rebuilds. I think
it's fair to assume that rt.jar never changes. If a dependency is named
rt.jar, then invalidation may not work correctly but I think that this
is the more important case to handle.
I verified that before this change, it was impossible to run
akka-actor/compile twice in a row using adopt jdk 11 and, after this
change, re-compilation worked as expected.
I discovered that some registered shutdown hooks would crash due to
67df72ab01 because they would try to load
classes from the closed classloader. To fix this, I add a internal
shutdown hooks mechanism that can be managed by sbt. Any unevaluated
shutdown hooks will be run when the sbt main method exits. This means
that they will be run when the user calls reboot. I think that is
reasonable.
I realized that all of the data structures that I needed to isolate the
classpath are contained in the AppProvider interface so there was no
need to use structural reflection on the top class loader.
It turns out that it can take roughly one second to instantiate a
scala.nsc.tools.Global instance for the first time. When sbt is starting
up, it also takes nearly 2 seconds to initialize logging. We can speed
up the boot time by doing these two things concurrently. On my machine,
I saw on average a 500ms decrease in startup time after this change.
I finally realized that the trick is that for non cygwin windows, the
available method on the jline wrapped input stream always returns zero.
Unlike on posix, however, the read method is interruptible which means
that we can just spin up a background thread that polls from the input
stream and writes it into a buffer.
I verified that it was no longer necessary to hit <enter> after 'r' to
rerun the continuous command on my windows vm after this change.
Given that there are io differences between windows and posix systems,
we should aim to run the tests that do a lot of io on windows. There are
a few tests that don't work because of some platform specific issues so
I added a filter that excludes these tests on windows in ScriptedTests.
This commit finally fixes#241 by adding support for sbt to either
print a warning or automatically reload the project if the metabuild
sources have changed. To facilitate this, I introduce a new key,
metaBuildSourceOption which has three options:
1) IgnoreSourceChanges
2) WarnOnSourceChanges
3) ReloadOnSourceChanges
When the former is set, sbt will not check if the meta build sources
have changed. Otherwise, sbt will use the buildStructure / fileInputs to
get the ChangedFiles for the metabuild. If there are any changes, it
will either warn or reload the build depending on the value of
metaBuildSourceOption.
The mechanism for diffing the files is that I add a step to EvaluateTask
where, if the project has been loaded and
metaBuildSourceOption != IgnoreSourceChanges, we evaluate the needReload
task. If we need a reload, we return an error that indicates that a
Reload is necessary. When that error is detected, the MainLoop will
prepend "reload" to the pending commands for the state. Otherwise we
just print a warning and continue.
I benchmarked the overhead of this and it wasn't too bad. I generally
saw it taking 5-20ms to perform the check. Since this is only done once
per task evaluation run, I don't think it's a big deal. When
IgnoreSourceChanges is set, there is O(10us) overhead. If performance
does become a problem, we could add a global watch service and skip the
needReload evaluation if no files have been modified.
I removed the watchTrackMetaBuild key and made it so that the continuous
builds only track the meta build when
metaBuildSourceOption == ReloadOnSourceChanges
The persistentFileStampCache does seem to work pretty well but in case
users encounter issues, I add a boolean flag that allows the user to
turn this behavior off and always re-stamp every source file in every
task evaluation run.
Previously the persistent attribute map was only reset when the file
event monitor detected a change. This made it possible for the cache to
be inconsistent with the state of the file system*. To fix this, I add an
observer on the file tree repository used by the continuous build that
invalidates the cache entry for any path for which it detects a change.
Invalidating the cache does not stamp the file. That only happens either
when a task asks for the stamp for that file or when the file event
monitor reports an event and we must check if the file was updated or
not.
After this change, touching a source file will not trigger a build
unless the contents of the file actually changes.
I added a test that touches one source file in a project and updates the
content of the other. If the source file that is only ever touched ever
triggers a build, then the test fails.
* This could lead to under-compilation because ExternalHooks would not
detect that the file had been updated.
I had tried to be cute and only inject certain tasks if they're actually
used, but that made it so that dynamic tasks may not have be able to use
them.
The new io verion removes the PathFinder <-> Glob implicit translations.
It also has a number of small bug fixes related to directory listing via
FileTreeView.
The main project emits a number of deprecation warnings. I've isolated
the deprecation warnings related to Watch to the DeprecatedContinuous
file. I fixed the deprecation warnings where it was straightforward to
do so. After this change, there are three non-watch related changes
emitted:
1) Defaults.scala:3760 uses the deprecated InputTask.apply. This seems
fixable but I'm not in a hurry
2) oldLoadFailed and oldLastGrep are used by Main. I think this could
just be fixed by removing the deprecation warnings and setting them
private[sbt] since they will still be available in the shell.
I previously tried to fix https://github.com/sbt/sbt/issues/4608 in
fc715cab44 by finding the instance of
xsbt.boot.BootFilteredLoader in the classloader heirarchy. This was a
risky approach since it made a lot of assumptions about the classloaders
used to invoke xMain.run. Since the point is to filter out the scala
standard library jar, I reworked things to just find all the parents of
the scala provider loader and then walk the graph from the root
classloader until it finds the classloader that contains the scala
library. If no such classloader exists, it ends up returning the parent
of the scala provider library.
I also renamed the libraryLoader parameter to scalaProviderLoader since
that is what is actually passedin. It is actually the libraryLoader that
we want to exclude.
Ref https://github.com/sbt/sbt/issues/4661
local-preloaded-ivy contains dangling ivy.xml without JAR files.
We might include local-preloaded again once we have a preloaded in Maven layout.
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler clAdd scalaXml to testingProj dependencies
This also seemed to be missing once the sbt metabuild classpath stopped
leaking into the compiler classpath.
Previously the zincLmIntegrationProj would compile if the meta build classpath
leaked into the compilation classpath. I found that the project would
not compile a clean build running an sbt built off of origin/develop.
Ethan Atkins
eugene yokota
xuwei-k