The existing implementation of watch did not work with the thin client.
In sbt 1.3.0, watch was changed to be a blocking command that performed
manual task evaluation. This commit makes the implementation more
similar to < 1.3.0 where watch modifies the state and after running the
user specified command(s), it enters a blocking command. The new
blocking command is very similar to the shell command.
As part of this change, I also reworked some of the internals of watch
so that a number of threads are spawned for reading file and input
events. By using background threads that write to a single event queue,
we are able to block on the file events and terminal input stream rather
than polling. After this change, the cpu utilization as measured by ps
drops from roughly 2% of a cpu to 0.
To integrate with the network client, we introduce a new UITask that is
similar to the AskUserTask but instead of reading lines and adding execs
to the command queue, it reads characters and converts them into watch
commands that we also append to the command queue.
With this new implementation, the watch task that was added in 1.3.0 no
longer works. My guess is that no one was really using it. It wasn't
documented anywhere. The motivation for the task implementation was that
it could be called within another task which would let users define a
task that monitors for file changes before running. Since this had never
been advertised and is only of limited utility anyway, I think it's fine
to break it.
I also had to disable the input-parser and symlinks tests. I'm not 100%
sure why the symlinks test was failing. It would tend to work on my
machine but fail in CI. I gave up on debugging it. The input-parser test
also fails but would be a good candidate to be moved to the client test
in the serverTestProj. At any rate, it was testing a code path that was
only exercised if the user changed the watchInputStream method which is
highly unlikely to have been done in any user builds.
The WatchSpec had become a nuisance and wasn't really preventing from
any regressions so I removed it. The scripted tests are how we test
watch.
This project is used to create client executables. The implementation is
pure java but we can build graalvm native-images from the java main
class. There are two versions of the client. One of them uses the
ipcsocket jni implementation to connect to the sbt server while the
other uses jna. It is necessary to use jni for the graalvm native-image
tool to work. Otherwise the two approaches should be identical.
This commit makes it possible for the sbt server to render the same ui
to multiple clients. The network client ui should look nearly identical
to the console ui except for the log messages about the experimental
client.
The way that it works is that it associates a ui thread with each
terminal. Whenever a command starts or completes, callbacks are invoked
on the various channels to update their ui state. For example, if there
are two clients and one of them runs compile, then the prompt is changed
from AskUser to Running for the terminal that initiated the command
while the other client remains in the AskUser state. Whenever the client
changes uses ui states, the existing thread is terminated if it is
running and a new thread is begun.
The UITask formalizes this process. It is based on the AskUser class
from older versions of sbt. In fact, there is an AskUserTask which is
very similar. It uses jline to read input from the terminal (which could
be a network terminal). When it gets a line, it submits it to the
CommandExchange and exits. Once the next command is run (which may or
may not be the command it submitted), the ui state will be reset.
The debug, info, warn and error commands should work with the multi
client ui. When run, they set the log level globally, not just for the
client that set the level.
Fixes https://github.com/sbt/sbt/issues/3112
This unpacks Extracted as State's extension methods.
In addition this provides a way of responding via LSP.
The repo overrides scripted test relies on using the launcher to modify
the default resolvers. To support this, I extended the scripted launcher
to use the bundled sbt launcher if it is passed in via the
`-Dsbt.launch.jar` system property.
This commit makes it so that the scalaVersion, sbtVersion and classpath
are always passed in as parameters to any method that creates an sbt
server -- either for scripted or for the sbt server tests. By making
that change, I was able to change the implementation of scripted in the
sbt project to use publishLocalBin instead of publishLocal. This makes
the scripted tests start much faster (doc alone can easily take 30
second) with messing with the build to exclude slow tasks from
publishLocal.
As part of this change, I removed the test dependency on scriptedSbtRedux for
sbtProj and instead had scriptedSbtRedux depend on sbtProj. This allowed
me to remove some messy LocalProject logic in the resourceGenerators for
scriptedSbtReduxProj. I also had to remove a number of imports in the
scriptedSbtReduxProj because the definitions available in the sbt
package object became available.
I also removed the dependency on sbt-buildinfo and instead pass the
values from the build into test classes using scalatest properties. I
ran into a number of minor issues with the build info plugin, namely
that I couldn't get fullClasspathAsJars to reliably run as a BuildInfo
key. It also is somewhat more clear to me to just rely on the built in
scalatest functionality. The big drawback is that the scalatest
properties can only be strings, but that restriction isn't really a
problem here (strangely the TestData structure has a field configMap
which is effectively Map[String, Any] but Any is actually always String
given how the TestData is created as part of framework initialization.
Since scripted no longer publishes, scriptedUnpublished is now
effectively an alias for scripted.
To get publishLocalBin working, I had to copy private code from
IvyXml.scala into PublishBinPlugin. Once we publish a new version of
sbt, we can remove the copied code and invoke IvyXml.makeIvyXmlBefore
directly.
The swoval javafmt plugin uses the google java formatter (which I
believe is the only widely used java formatter) to format source files.
It does not provide an automatic javafmtOnCompile method like the
scalafmt plugin so I had to manually implement that functionality. In
general the java formatter is much faster than scalafmt so the impact of
having javafmtOnCompile set to true is very low.
This makes it possible to do mkIvyConfiguration.value.withXXX(...) for
all the methods in InlineIvyConfiguration. (I need this to remove
inter-project resolvers when fetching dotty from sbt-dotty to avoid
accidentally fetching a local project in the build of dotty itself).
Ref #4211Fixes#4395Fixes#4600
This is a reimplementation of `--addPluginSbtFile`. #4211 implemented the command to load extra `*.sbt` files as part of the global plugin subproject. That had the unwanted side effects of not working when `.sbt/1.0/plugins` directory does not exist. This changes the strategy to load the `*.sbt` files as part of the meta build.
```
$ sbt -Dsbt.global.base=/tmp/hello/global --addPluginSbtFile=/tmp/plugins/plugin.sbt
[info] Loading settings for project hello-build from plugin.sbt ...
[info] Loading project definition from /private/tmp/hello/project
sbt:hello> plugins
In file:/private/tmp/hello/
sbt.plugins.IvyPlugin: enabled in root
sbt.plugins.JvmPlugin: enabled in root
sbt.plugins.CorePlugin: enabled in root
sbt.ScriptedPlugin
sbt.plugins.SbtPlugin
sbt.plugins.SemanticdbPlugin: enabled in root
sbt.plugins.JUnitXmlReportPlugin: enabled in root
sbt.plugins.Giter8TemplatePlugin: enabled in root
sbtvimquit.VimquitPlugin: enabled in root
```
A number of users were reporting issues with deadlocking when using
1.3.2: https://github.com/sbt/sbt/issues/5116. This seems to be because
most of the sbt created classloaders were not actually parallel capable.
In order for a classloader to be registered as a parallel capable, ALL
of the parent classes except for object in the class hierarchy must be
registered as a parallel capable:
https://docs.oracle.com/javase/8/docs/api/java/lang/ClassLoader.html#registerAsParallelCapable--.
If a classloader is not registered as parallel capable, then a global
lock will be used internally for classloading and this can lead to deadlock.
It is impossible to register a scala 2 classloader as parallel capable
so I ported all of the classloaders to java.
This commit updates the java-serialization scripted test. Prior to the
port, the new version of the test would more or less always deadlock.
After this change, I haven't been able to reproduce a deadlock.
This had no significant performance impact when I reran
https://github.com/eatkins/scala-build-watch-performance
There have been a number of complaints about the new classloader closing
behavior. It is too aggressive about closing classloaders after test and
run. This commit softens the behavior by allowing a classloader to be
resurrected after close by creating a new zombie classloader that has
the same urls as the original classloader. After this commit, we always
close the classloaders when we are done, but they can still leak
file descriptors if a zombie is created.
To configure the behavior, I add the allowZombieClassLoaders key. If it
is false (which is default), we will warn but still allow them. If it
is true, then we silence the warning. In a later version of sbt, we can
change the semantics to be strict.
I verified after this change that I could add a shutdown hook in `run`
and it would be evaluated so long as I set `bgCopyClasspath := false`.
Otherwise the needed jars were deleted before the hooks could run.
Bonus: delete unused ResourceLoaderImpl class
Ethan Atkins
Eugene Yokota
Adrien Piquerez
Brice Jaglin
Dale Wijnand
Guillaume Martres
xuwei-k
Filipe Regadas
Charles O'Farrell