For the details about this PR, please see the blog post https://eed3si9n.com/sbt-remote-cache/.
* Add cache basics
* Refactor Attributed to use StringAttributeMap, which is Map[StringAttributeKey, String]
* Implement disk cache
* Rename Package to Pkg
* Virtualize packageBin
* Use HashedVirtualFileRef for packageBin
* Virtualize compile task
In https://github.com/sbt/sbt/pull/5981 I tried to work around the spruious post-macro "a pure expression does nothing" warning (https://github.com/scala/bug/issues/12112) by trying to remove some pure-looking expressions out of the tree.
This quickly backfired when it was reported that sbt 1.4.3 was not evaluating some code. This backs out the macro-level manipulation, and instead try to silence the warning at the reporter level. This feels safer, and it seems to work just as well.
Ref https://github.com/scala/bug/issues/12112
Current app transformation macro creates a tree that looks like:
```scala
FullInstance.app[[T0[x]](T0[Int], T0[Int]), Unit](scala.Tuple2(task2, task1), (($p$macro$3: (Int, Int)) => {
<synthetic> val $q$macro$2: Int = $p$macro$3._1;
<synthetic> val $q$macro$1: Int = $p$macro$3._2;
{
($q$macro$1: Int);
($q$macro$2: Int);
()
}
}))(AList.tuple2[Int, Int])
```
Starting Scala 2.12.12 the compiler's "pure expression does nothing" has become more enthusiastic/accurate in its reach, and it started warning about the naked reference to `$q$macro$1` that appears to do nothing, even though in reality it would trigger the tasks and do something in the context of sbt. It's just _that_ particular line ends up macroed away into a pure expression.
A somewhat bizarre workaround is to make a fake call to a method just to satisfy this warning. I've chosen `scala.Predef.identity` here so it can be composed together with the existing expression nesting when they exist.
Using the scala reflect library always introduces significant
classloading overhead. We can eliminate the classloading overhead by
generating StringTypeTags at compile time instead.
This sped up average project loading time by a few hundred milliseconds
on my computer. The ManagedLoggedReporter in zinc is still using the
type tag based apis but after the next sbt release, we can upgrade the
zinc apis. We also could consider breaking binary compatibility.
To demonstrate [-Yno-lub](http://eed3si9n.com/stricter-scala-with-ynolub), this shows the code changes that removes lubing (Not all subprojects are done).
After I made the changes, I switched the Scala back to normal 2.12.10.
In my recent changes to watch, I have been moving towards a world in
which sbt manages the file inputs and outputs at the task level. The
main idea is that we want to enable a user to specify the inputs and
outputs of a task and have sbt able to track those inputs across
multiple task evaluations. Sbt should be able to automatically trigger a
build when the inputs change and it also should be able to avoid task
evaluation if non of the inputs have changed.
The former case of having sbt automatically watch the file inputs of a
task has been present since watch was refactored. In this commit, I
make it possible for the user to retrieve the lists of new, modified and
deleted files. The user can then avoid task evaluation if none of the
inputs have changed.
To implement this, I inject a number of new settings during project
load if the fileInputs setting is defined for a task. The injected
settings are:
allPathsAndAttributes -- this retrieves all of the paths described by
the fileInputs for the task along with their attributes
fileStamps -- this retrieves all of the file stamps for the files
returned by allPathsAndAttributes
Using these two injected tasks, I also inject a number of derived tasks,
such as allFiles, which returns all of the regular files returned by
allPathsAndAttributes and changedFiles, which returns all of the regular
files that have been modified since the last run.
Using these injected settings, the user is able to write tasks that
avoid evaluation if the inputs haven't changed.
foo / fileInputs += baseDirectory.value.toGlob / ** / "*.scala"
foo := {
foo.previous match {
case Some(p) if (foo / changedFiles).value.isEmpty => p
case _ => fooImpl((foo / allFiles).value
}
}
To make this whole mechanism work, I add a private task key:
val fileAttributeMap = taskKey[java.util.HashMap[Path, Stamp]]("...")
This keeps track of the stamps for all of the files that are managed by
sbt. The fileStamps task will first look for the stamp in the attribute
map and, only if it is not present, it will update the cache. This
allows us to ensure that a given file will only be stamped once per task
evaluation run no matter how the file inputs are specified. Moreover, in
a continuous build, I'm able to reuse the attribute map which can
significantly reduce latency because the default file stamping
implementation used by zinc is fairly expensive (it can take anywhere
between 300-1500ms to stamp 5000 8kb source files on my mac).
I also renamed some of the watch related keys to be a bit more clear.
The newest version of io repackages a number of classes into the
sbt.nio.* packages. It also changes some of the semantics of glob
related apis. This commit updates all of the usages of the updated apis
within sbt but should have no functional difference.
Eugene Yokota
xuwei-k
Adrien Piquerez
Julien Richard-Foy
Ethan Atkins
Dale Wijnand
Ethan Atkins