For the details about this PR, please see the blog post https://eed3si9n.com/sbt-remote-cache/.
* Add cache basics
* Refactor Attributed to use StringAttributeMap, which is Map[StringAttributeKey, String]
* Implement disk cache
* Rename Package to Pkg
* Virtualize packageBin
* Use HashedVirtualFileRef for packageBin
* Virtualize compile task
See https://eed3si9n.com/simplifying-sbt-with-common-settings/
Problem
-------
The behavior of bare settings is confusing in a multi-project build.
This is partly due to the fact that to use `ThisBuild` scoping
the build user needs to be aware of the task implementation,
and know if the task is already defined at project level.
Solution
--------
This changes the interpretation of the baresettings to be common
settings, which works similar to the way `ThisBuild` behaves in sbt 1.x,
but since this would be a simple append at project-level, it should
work for any tasks or settings.
The ConsoleAppender formatEnabledInEnv field was being used both as an
indicator that ansi codes were supported and that color codes are
enabled. There are cases in which general ansi codes are not supported
but color codes are and these use cases need to be handled separately.
To make things more explicit, this commit adds isColorEnabled and
isAnsiSupported to the Terminal companion object so that we can be more
specific about what the requirements are (general ansi escape codes or
just colors). There are a few cases in ConsoleAppender itself where
formatEnabledInEnv was used to set flags for both color and ansi codes.
When that is the case, we use Terminal.isAnsiSupported because when that
is true, colors should at least work but there are terminals that
support color but not general ansi escape codes.
Ref https://github.com/sbt/zinc/pull/744
This implements `ThisBuild / usePipelining`, which configures subproject pipelining available from Zinc 1.4.0.
The basic idea is to start subproject compilation as soon as pickle JARs (early output) becomes available. This is in part enabled by Scala compiler's new flags `-Ypickle-java` and `-Ypickle-write`.
The other part of magic is the use of `Def.promise`:
```
earlyOutputPing := Def.promise[Boolean],
```
This notifies `compileEarly` task, which to the rest of the tasks would look like a normal task but in fact it is promise-blocked. In other words, without calling full `compile` task together, `compileEarly` will never return, forever waiting for the `earlyOutputPing`.
This commit makes it possible for the sbt server to render the same ui
to multiple clients. The network client ui should look nearly identical
to the console ui except for the log messages about the experimental
client.
The way that it works is that it associates a ui thread with each
terminal. Whenever a command starts or completes, callbacks are invoked
on the various channels to update their ui state. For example, if there
are two clients and one of them runs compile, then the prompt is changed
from AskUser to Running for the terminal that initiated the command
while the other client remains in the AskUser state. Whenever the client
changes uses ui states, the existing thread is terminated if it is
running and a new thread is begun.
The UITask formalizes this process. It is based on the AskUser class
from older versions of sbt. In fact, there is an AskUserTask which is
very similar. It uses jline to read input from the terminal (which could
be a network terminal). When it gets a line, it submits it to the
CommandExchange and exits. Once the next command is run (which may or
may not be the command it submitted), the ui state will be reset.
The debug, info, warn and error commands should work with the multi
client ui. When run, they set the log level globally, not just for the
client that set the level.
This adds `Def.promise` a facility that wraps `scala.concurrent.Promise`. Project layer, there's an implicit for task-that-returns-promise (`Def.Initialize[Task[PromiseWrap[A]]]`) that would inject `await` method, which returns a task. This is a special task that is tagged with `Tags.Sentinel` so that it will bypass the concurrent restrictions. Since there's no CPU- or IO-bound work, this should be ok.
The purpose of this promise for long-running task to communicate with another task midway.
When `Def.task`, `:=`, `+=` etc contains `if` and only `if` expression automatically treat it as a conditional task even if the else clause contains `.value`.
This implements Selective functor for `Either[A, B]` "task" (`Initialize[Task[Either[A, B]]]`).
The selective functor allows an encoding of if-expression:
```
def ifS[A](
x: Def.Initialize[Task[Boolean]]
)(t: Def.Initialize[Task[A]])(e: Def.Initialize[Task[A]]): Def.Initialize[Task[A]]
```
The benefit of this approach is that task dependencies are still visible to inspect command.
This commit makes it so that the scalaVersion, sbtVersion and classpath
are always passed in as parameters to any method that creates an sbt
server -- either for scripted or for the sbt server tests. By making
that change, I was able to change the implementation of scripted in the
sbt project to use publishLocalBin instead of publishLocal. This makes
the scripted tests start much faster (doc alone can easily take 30
second) with messing with the build to exclude slow tasks from
publishLocal.
As part of this change, I removed the test dependency on scriptedSbtRedux for
sbtProj and instead had scriptedSbtRedux depend on sbtProj. This allowed
me to remove some messy LocalProject logic in the resourceGenerators for
scriptedSbtReduxProj. I also had to remove a number of imports in the
scriptedSbtReduxProj because the definitions available in the sbt
package object became available.
I also removed the dependency on sbt-buildinfo and instead pass the
values from the build into test classes using scalatest properties. I
ran into a number of minor issues with the build info plugin, namely
that I couldn't get fullClasspathAsJars to reliably run as a BuildInfo
key. It also is somewhat more clear to me to just rely on the built in
scalatest functionality. The big drawback is that the scalatest
properties can only be strings, but that restriction isn't really a
problem here (strangely the TestData structure has a field configMap
which is effectively Map[String, Any] but Any is actually always String
given how the TestData is created as part of framework initialization.
Since scripted no longer publishes, scriptedUnpublished is now
effectively an alias for scripted.
To get publishLocalBin working, I had to copy private code from
IvyXml.scala into PublishBinPlugin. Once we publish a new version of
sbt, we can remove the copied code and invoke IvyXml.makeIvyXmlBefore
directly.
To demonstrate [-Yno-lub](http://eed3si9n.com/stricter-scala-with-ynolub), this shows the code changes that removes lubing (Not all subprojects are done).
After I made the changes, I switched the Scala back to normal 2.12.10.
I am writing a plugin that uses mangled task keys that are very hard to
read. It is helpful to be able to override the show config for these
scopes so that they look reasonable in supershell and in error
reporting.
João Ferreira
xuwei-k
Adrien Piquerez
Eugene Yokota
Julien Richard-Foy
David Gregory
Ethan Atkins
Dale Wijnand
Josh Soref