EvaluateTask was holding references to SafeState that could be quite
large. This was reported as #5992. In that project, I ran the `ci` task
and observed the OOM as reported. I took a heap dump prior to OOM and
got the retained size graph from visualvm (which took hours to compute).
The lastEvaluatedState was holding a reference to SafeState that was
1.7GB. The project max heap size was set to 2GB. Instead of using the
lastEvaluatedState, we can just use StandardMain.exchange.withState.
The cached instances of state were used for task cancellation and
completions. While it is possible that early on in booting
StandardMain.exchange.withState could return a null state, in practice
this won't happen because it is set early on during the sbt boot
commands.
After this change, I successfully ran the `ci` task in the #5992 issue
project with the same memory parameters as their ci config.
The ConsoleAppender formatEnabledInEnv field was being used both as an
indicator that ansi codes were supported and that color codes are
enabled. There are cases in which general ansi codes are not supported
but color codes are and these use cases need to be handled separately.
To make things more explicit, this commit adds isColorEnabled and
isAnsiSupported to the Terminal companion object so that we can be more
specific about what the requirements are (general ansi escape codes or
just colors). There are a few cases in ConsoleAppender itself where
formatEnabledInEnv was used to set flags for both color and ansi codes.
When that is the case, we use Terminal.isAnsiSupported because when that
is true, colors should at least work but there are terminals that
support color but not general ansi escape codes.
Some of the sbt scripted tests somewhat frequently hang in CI. I added a
patch that printed a stack trace of the sbt process every 30 seconds. I
discovered that the main thread was stuck in DefaultBackgroundJobService
shutdown. To avoid the hangs, I updated the awaitTermination methods to
take a timeout parameter and we timeout shutdown if 10 seconds have
elapsed.
I noticed that when using the scala 2.12 console with the thin client
that there was weird behavior for the first few seconds of the session.
When prompted with 'scala> ' I would type a letter, say v, and the
output would be 'scala>v' instead of 'scala> v'. It turned out that this
was because the NetworkChannel was returning a stale value for
isEchoEnabled. This happened because NetworkChannel has a method
getProperties that is rate limited under the assumption that the
properties rarely change. This made sense for things like
isAnsiSupported or isSuperShellEnabled but not isEchoEnabled. It is
straightforward to fix this by actually getting the terminal attributes
and checking if the echo flag is set.
It is possible for an InterruptedException to be thrown here because of
logic in NetworkClient. This seemed to be the root cause of the fix I
tried in ca251eb7c8 so I'm reverting that
commit.
Revert "Catch interrupted exception in shell"
This reverts commit ca251eb7c8.
In 64c0f0acdd, I attempted to safely close
all of the completion services when the user inputs ctrl+c. I have
noticed though that sometimes sbt crashes in CI with the
RejectedExecutionException thrown by submit. To avoid throwing when
there was no cancellation, I slightly modified the shutdown logic to not
shutdown the completion service whil still shutting down the underlying
thread pool.
It can be useful for plugin and build authors to have access to some of
the virtual terminal properties. For instance, when writing a task that
needs a password, the author may wish to put the terminal in raw mode
with echo disabled. This commit introduces a new Terminal trait at the
sbt level and a corresponding task, terminal, that provides a basic
terminal api. The Terminal returned by the terminal task will correspond
to the terminal that initiated the task so that it should work with sbtn
as well as in console mode.
Neither NetworkTerminal.getAttributes nor NetworkTerminal.setAttributes
worked correctly because they were sending the wrong json method name.
This wasn't noticeable because neither of these methods had previously
been used by sbt.
I noticed that no-op compile was slower in
https://github.com/sbt/sbt/issues/5508 using 1.4.0-RC2 than 1.4.0-RC1.
It took around 400ms with 1.4.0-RC2 and 200-250ms on RC1. Git bisect
brought me to 41afe9fbdb which I
remembered I'd been slightly concerned about from a performance
perspective but didn't get around to testing. The problem is that we
were blocking the task from running while determing whether or not we
should force a progress report. We can do that work on the background
thread instead so the task can begin running immediately.
The conditional for whether to make task progress events repeatable was
inverted. This wasn't actually noticeable because the function
doReport() was being schedule which had a guard to prevent it from
running more frequently than the report period.
Certain tasks may prefer to have the input set to raw mode and/or have
echo off. The specific use case is that it is difficult to get the
ammonite console to work correctly with the thin client. The problem is
that the ammonite console runs some tty commands. These commands will
only work on the tty of the thin client when the thin client itself has
launched the sbt server session (since they share the same tty). Once
the thin client that launched the server exits, the ammonite console
will never work again with that server session. A workaround is to
launch sbt separately and leave that server session open. Then, if the
run task is configured with canonical input set to false and echo
disabled, the thin client will work. In the future, it's possible that
ammonite could be updated to not rely on calling stty commands and then
the thin client could work with the ammonite console even after the
initial thin client session has exited provided canonical input and echo
are disabled.
There were a number of issues with swithcing between raw and canonical
issues that affected both the server and the thin client. These were
reported in #5863 and #5856. In both cases, there were issues with
reading input or having the input be displayed. Debugging those issues
revealed a number of issues with how we were using the jline 3 system
terminal and the hybrid interaction with the jline 2 terminal. This
commit eliminates all of our internal jline 2 usage. The only remaining
jline 2 usage is that we create and override the global terminal for the
scala console for scala versions < 2.13. By moving away from jline 2, I
was also able to fix#5828, which reported that the home, end and delete
keys were not working.
One of the big issues that this commit addresses is that the
NetworkClient was always performing blocking reads on System.in. This
was problematic because it turns out that you can't switch between raw
and canonical modes when there is a read present. To fix this, the
server now sends a message to the client when it wants to read bytes and
only then does the client create a background thread to read a single
byte.
I also figured out how to set the terminal type properly for the thin
client on windows where we had been manually setting the capabilities to
ansi, which only worked for some keys. This fix required switching to
the WindowsInputStream that I introduced in a prior commit. Before we
were using the jline 2 wrapped input stream which was converting some
system events, like home and end, to the wrong escape sequence mappings.
The remainder of the commit is mostly just converting from jline 2 apis
to jline 3 apis.
I verified that tab completions, the scala console, the ammonite console
and a run task that read from System.in all work with both the server
and the thin client on mac, linux and windows after these changes.
Fixes#5828, #5863, #5856
The old sbt launcher uses jansi 1.11, which is incompatible with jline3.
To work around this, we can use the jna terminal implementation for the
jline system terminal. This commit also switches to using the jline
TerminalBuilder for all system terminals except for the windows system
terminal with the thin client. The jline terminal builder uses
reflection that is difficult to make work with the thin client and it is
much easier to just manually construct the thin client. This is only
necessary for windows because on posix the thin client will fall back to
an implementation that shells out for stty commands.
The thin client needs to do its own success reporting because in batch
mode it's possible for the task to exit before success is logged by the
server. If the server also prints success, there can be double printing.
Unfortunately, the Prompt.Batch check is not reliable because MainLoop
will change the prompt to Running during task evaluation. The
interactive flag is set in the NetworkChannel when the client explicitly
registers itself as an interactive session, so this should be more
reliable.
I noticed in CI that sometimes the client tests exit with an interrupted
exception printed. I tracked it down the exception to the call to
getExec, which delegateds to CommandExchange.blockUntilNextExec.
In a continuous build in sbt 1.4.0-RC1, if the user enters an invalid
option, it causes the input thread to exit which means the watch would
no longer accept input commands (including <enter> to exit). This fixes
that behavior.
In sbt 1.4.0-RC1, if a user ran `sbt console`, the progress lines would
be printed after they had entered the console. This was because the
prompt state was incorrect. To get the prompt in the correct state, we
initialize the prompt to batch and then switch to pending when either
sbt enters the shell or the network client attaches in interactive mode.
We also will now immediately print progress as soon as we enter a skip
task to clear out the progress lines and display the warning about a
running task if there is another client connected while the task is
running.
The clean task was previously deleting the contents of directories that
were symlinked into the target directory. This was an oversight because
it never occurred to me that users might symlink a directory whose
contents they did not want deleted into the target directory.
Fixes https://github.com/sbt/sbt/issues/5822
Currently the entire shell gets stuck when there's a compilation error with pipelining.
This at least returns to sbt shell.
Together with https://github.com/sbt/zinc/pull/920 this fixes most of the mixed pipelining issues.
1. Previous values are carried from `compileScalaBackend` in `compileJavaTask`.
2. `compileJava / compileOptions ` now uses `compile / compileOptions` to avoid unintentional change of javac or scalac options.
3. Hooks up early compile analysis store.
Ref https://github.com/sbt/sbt/issues/5665
This adds `--server` command that is immediately filtered out in Main.scala.
The purpose of `--server` is so we can invoke thin client from `sbt` script at some point in the future when Bash script can parse `project/build.properties`.
`sbtn` would need to call `sbt` again to start the server, and at that point the shell script would need to actually invoke the server. The intent of `--server` is to be used as the tie breaker.
Also build users may want to sometimes call `sbt --server`.
I introduced the terminalShellPrompt so that we could generate a prompt
that was colored only if the terminal supported color. Rather than
expose the terminal implementation detail, we can just use a boolean
flag that toggles whether or not color is enabled and sbt can pass in
the value of terminal.isColorEnabled into the function.
It shouldn't be the case that a RejectedExecutionException is thrown
by TaskProgress. If that assumption is violated, log the exception but
don't crash sbt.
The play plugin seems to do out of band task evaluation on a stale State
object in the `run` task. As a result, when sbt tries to schedule tasks
to run, they tried to register the work with a closed TaskProgress
instance. There was no guard against this and it ended up causing a
RejectedExecutionException.
sbt 1.4.0 generates the shell prompt using the terminal properties for
the specific terminal for which the prompt is rendered. The mechanism
for doing this broke the prompt for projects that overrode the
shellPrompt key, notably the play plugin. After this change, the play
custom prompt is correctly rendered with 1.4.0-SNAPSHOT.
The RelayAppender should not log directly to console out since it is
supposed to be relaying json log messages to connected clients. This was
manifesting as double printing on some success messages.
Running publishLocal in the zinc project can cause gc thrashing with the
default parallelgc collector using jdk8 on my laptop. If I switch to
G1GC, it does not thrash even if I leave the heap the same size.
The java GarbageCollectorMXBean.getCollectionTime returns the cumulative
amount of time the collector has run during the jvm session. The GC
monitor is tracking how much time has been spent in garbage collection
during each task evaluation run. In order for this calculation to work
correctly, it is necessary to set the initial elapsed time to the bean's
current collection time when we create the gc monitor. Without doing
this, we can get completely incorrect results that are reporting based
on the total gc time for the entire process, not just in the last 10
seconds.
Should fix https://github.com/sbt/sbt/issues/5818
It is not uncommon in large projects for the jvm to silently be running
frequent full gcs in the background. This can slow progress to a crawl.
Usually the fix is to bump the -Xmx parameter but if the users do not
realize that their tasks are slow because of gc thrashing, they may not
think to do that. This PR adds a monitor that hooks into the jvm's event
notification system to keep track of how much time is spent in GC. If
the ratio of the amount of time in gc to the total elapsed time exceeds
some threshold, we emit a warning.
I was motivated to do this because publishLocal can take forever in the
zinc project because a 1G heap isn't big enough.
These tasks show up during task progress and they clutter up the
display. Since my understanding is that both of these tasks are more or
less just waiting for other work to complete, I don't think they are
helpful for debugging.
The zinc scripted project depends on all of the compiler bridges. As a
result the introduction of the strict scala binary version check in
f8139da192 broke zinc scripted. This
commit reverts to the old behavior in the non scala sandwich case.
I also switched to a for comprehension instead of a pattern match
because this is a rare case where I think it made the code significantly
more readable.
While in a continuous build, when the user enters ctrl+c into the sbt
server console (not a thin client connection) when sbt has been launched
in interactive mode, the server exits. This commit makes it so that
instead we just cancel the watch. As a result, if sbt was started in
batch mode, e.g. `sbt ~compile`, ctrl+c will still exit sbt but in
interactive mode ctrl+c will take the user back to the shell.
I occassionally end up in a state where watch input does not seem to be
read. To rule out the possibility that the background thread that reads
input has not successfully started, this commit makes it so that we
block until the thread signals that it has started via a CountDownLatch.
The diff is superficially big because of an indentation change at the
bottom.
The sbt Server is initialized with a callback onIncomingSocket. That
callback was created in CommandExchange and held references to a build
structure and a state. Neither the state nor structure would ever go out
of scope so they effectively leaked. It is possible for each
NetworkChannel to access a recent instance of state through the
CommandExchange.withState method. Using this, we can eliminate the
references to state and build structure in the onIncomingSocket
callback. In the sbt project, this reduced the memory utilization by
about 50mb on startup.
On linux and mac, entering ctrl+c will automatically kill any forked
processes that were created by the sbt server because sigint is
automatically forwarded to the child process. This is not the case on
windows where it is necessary to forcibly kill these processes.
The intellij import currentlly works by forking an sbt process and
writing command input through the process input stream. To make this
work, we need the SimpleTerminal (which is used when sbt is run with
-Dsbt.log.noformat=true) to be able to read input.
Attaching the input to the simple terminal caused watch tests to fail on
windows. This can be fixed by checking if the byte read from the input
stream is -1 and ignoring it if so.
The sbt.log.noformat parameter should be treated very similarly to
sbt.io.virtual. When it is true, we should just use the raw io streams
for the process. This came up because of
https://github.com/sbt/sbt/issues/5784 which reported that intellij
imports were not working and that ansi control characters were being
written to the output.
There can be race conditions where we try to interrupt and join a ui
thread before it becomes interruptible by blockign on a queue. To
workaround this, we can add the JoinThread class which adds an
extension method Thread.joinFor that takes a FiniteDuration parameter.
This variant of join will repeatedly interrupt and attempt to join the
thread for up to 10 milliseconds before retrying until the limit is
reached. If the limit is reached, we print a noisy error to the console.
I'm not 100% sure if we are leaking threads in the latest sbt version
but this gives me more piece of mind that either we are always
successfully joining the threads or we will be alerted if the joining
fails.
Ethan Atkins
eugene yokota
xuwei-k
Guillaume Martres
Taichi Yamakawa
Nader Ghanbari
Adrien Piquerez
Regis Desgroppes
Andrii