It takes about a second to load scala.reflect.runtime.universe. If we
lazy load here, we can load scala.relect.runtime.universe in the
background to speed up the sbt start up time. See
0ebb7a5662.
The previous implementation of supershell log line interlacing with
regular line interlacing relied on state in a global object. A somewhat
better approach is for each appender to hold a reference to a state
object. Every time tasks run, new appenders are created, so the state
should always reflect the current progress state.
Supershell actually works quite well in no color mode. On the sbt side,
we still want to disable supershell automatically if the output is not a
terminal or no color is set, but this commit allows the user to force
supershell through -Dsbt.supershell or the useSuperShell setting even
when no color is set.
With this commit, I improved the padding management so that padding is
now added above the progress report. Whenever a line is logged at the
info or greater level, we can reduce the padding level by one since that
line has effectively filled in the padding.
With the current supershell implementation, the progress display
flickers when there is heavy console logging during task evaluation.
This is because the console appender clears out the task progress and it
isn't restored until the next periodic super shell report (which
runs every 100ms by default). To remove the flickering, I reworked the
implementation to interlace the log lines with progress reports. In
order to ensure that the log lines remained contiguous, I had to apply
padding at the bottom of the supershell region whenever the new report
contained fewer lines than the old report. The report shifts down as new
log lines are appended. This isn't optimal, but I think removing
the flickering while preserving contiguous log lines is worth it.
During akka startup, addLocal was caused twice and prior to this change,
it took roughly 200ms per call on my computer. After this change, it
took about 100ms.
At the suggestion of @eed3si9n, instead of specifying the file cache
size in bytes, we now specify it in a formatted string. For example,
instead of specifying 128 megabytes in bytes (134217728), we can specify
it with the string "128M".
In some cases, comp.append could be an empty string. This would happen
if a parser was something like `(token(foo) <~ ;).+ <~ fo.?` because there were
no completions for the `fo` available anchor. The effect of this was
that tab would never complete foo;f to foo;foo, even though that was the
only possible completion. It would, _display_, foo as a possible
completion though.
This came up because the multi parser has a similar parser to that
described above and it broke tab completion to the right of a semi
colon.
We run into issues if we naively split the command input on ';' and
treat each part as a separate command unless the ';' is inside of a
string because it is also valid to have ';'s inside of braced
expressions, e.g. `set foo := { val x = 1; x + 1 }`. There was no parser
for expressions enclosed in braces. I add one that should parse any
expression wrapped in braces so long as each opening brace is matched by a
closing brace. The parser returns the original expression. This allows
the multi parser to ignore ';' inside of '{...}'.
I had to rework the scripted tests to individually run 'reload' and
'setUpScripted' because the new parser rejects setUpScripted because it
isn't a valid command until reload has run.
In my recent changes to watch, I have been moving towards a world in
which sbt manages the file inputs and outputs at the task level. The
main idea is that we want to enable a user to specify the inputs and
outputs of a task and have sbt able to track those inputs across
multiple task evaluations. Sbt should be able to automatically trigger a
build when the inputs change and it also should be able to avoid task
evaluation if non of the inputs have changed.
The former case of having sbt automatically watch the file inputs of a
task has been present since watch was refactored. In this commit, I
make it possible for the user to retrieve the lists of new, modified and
deleted files. The user can then avoid task evaluation if none of the
inputs have changed.
To implement this, I inject a number of new settings during project
load if the fileInputs setting is defined for a task. The injected
settings are:
allPathsAndAttributes -- this retrieves all of the paths described by
the fileInputs for the task along with their attributes
fileStamps -- this retrieves all of the file stamps for the files
returned by allPathsAndAttributes
Using these two injected tasks, I also inject a number of derived tasks,
such as allFiles, which returns all of the regular files returned by
allPathsAndAttributes and changedFiles, which returns all of the regular
files that have been modified since the last run.
Using these injected settings, the user is able to write tasks that
avoid evaluation if the inputs haven't changed.
foo / fileInputs += baseDirectory.value.toGlob / ** / "*.scala"
foo := {
foo.previous match {
case Some(p) if (foo / changedFiles).value.isEmpty => p
case _ => fooImpl((foo / allFiles).value
}
}
To make this whole mechanism work, I add a private task key:
val fileAttributeMap = taskKey[java.util.HashMap[Path, Stamp]]("...")
This keeps track of the stamps for all of the files that are managed by
sbt. The fileStamps task will first look for the stamp in the attribute
map and, only if it is not present, it will update the cache. This
allows us to ensure that a given file will only be stamped once per task
evaluation run no matter how the file inputs are specified. Moreover, in
a continuous build, I'm able to reuse the attribute map which can
significantly reduce latency because the default file stamping
implementation used by zinc is fairly expensive (it can take anywhere
between 300-1500ms to stamp 5000 8kb source files on my mac).
I also renamed some of the watch related keys to be a bit more clear.
Ref https://github.com/sbt/sbt/issues/4583
This moves the super shell rendering to ConsoleAppender with several improvements.
Instead of scrolling up, supershell is now changed to normal scrolling down, with more traditional cursor position. Before printing out the logs, last known progress reports are wiped out. In addition, there's now 5 lines of blank lines to accomodate for `println(...)` by tasks.
The usingTerminal method synchronizes the JLine object which can lead to
deadlock if multiple threads call it. When we just to want to read the
attributes of the terminal, but not modify it, there doesn't seem to be
any reason to use a lock.
I was seeing a number of compiler warnings about the type parameter
Scope:
Settings.scala:55:12: type parameter Scope defined in trait Init shadows class Scope defined in package util. You may want to rename your type parameter, or possibly remove it.
I'm not sure why I wasn't seeing these before, but the fix is simple.
Although this is technically in the internal package, it is exposed to
users when they write a custom input task. I do not think that we should
prevent users/plugin authors from writing their own parser
implementations if there is a different library they prefer. By my
count, there are 21 implementations of this interface in sbt, so it's
unlikely that there is much benefit from a pattern matching perspective.
While the AnyLeft and AnyRight types are necessary to make the extension
class work, I don't want to leak the AnyLeft or AnyRight traits into the
public api. It wasn't neceessary to annotate `some`, but it's good
practice to annotate anything public anyway.
This implements a new sbt.color flag that takes always/auto/never/true/false value as a replacement of current sbt.log.format=true/false flag.
When neither flags are set, the default behavior is to enable color when the terminal supports ANSI and it detects an stdout console (as opposed to redirects).
Fixes https://github.com/sbt/sbt/issues/4284
Dotty has its own logic for displaying problems with the proper file
path, position, and caret, but if we store this information in
Problem#message we end up with duplicated information in the output
since Zinc will prepend/append similar things (see
sbt.internal.inc.ProblemStringFormats). So far, we worked around this in
Dotty by using an empty position in the sbt bridge reporter, but this
means that crucial semantic information that could be used by a Build
Server Protocol implementation and other tools is lost. This commit
allows us to avoid by adding an optional `rendered` field to `Problem`:
when this field is set, its value controls what the user sees, otherwise
we fallback to the default behavior (the logic to do this will be added to
Zinc after this PR is merged and a new release of sbt-util is made).
Positions in the Language Server Protocol and Build Server Protocol are
line/column-based instead of offset-based, so this is more convenient.
Computing the line/column from the offset is possible but requires
reading the source file.
A position now has a start, an end, and a point (the existing `offset`),
just like it does in the Scala compiler. This information is especially
useful for displaying squiggly lines in an IDE.
This commit and the next one are required for https://github.com/sbt/zinc/pull/571
Make sure that we generate valid ID (according to matches(ID, s)) so
that we properly test the new validID implementation.
And that's what led to the bug fix. :)
This showed up in profiling. It's known that TypeTags are expensive. Even
more so if the reflect universe is accessed during startup when the
class loading and JIT compiler are busy enough with other stuff.
It turned up in profiling sessions. Previously, it used parser combinators
which are somewhat slow especially when the JVM is still cold.
The grammar for ID is simple enough to afford this handwritten parser.
There are just too many instances in which sbt's code relies on
the `lastModified`/`setLastModified` semantics, so instead of moving
to `get`/`setModifiedTime`, we use new IO calls that offer the new
timestamp precision, but retain the old semantics.
Fixessbt/sbt#1812
This adds unified slash syntax for both sbt shell and the build.sbt DSL.
Instead of the current `<project-id>/config:intask::key`,
this adds `<project-id>/<config-ident>/intask/key` where <config-ident> is the Scala identifier notation for the configurations like `Compile` and `Test`.
This also adds a series of implicits called `SlashSyntax` that adds `/` operators to project refererences, configuration, and keys such that the same syntax works in build.sbt.
These examples work for both from the shell and in build.sbt.
Global / cancelable
ThisBuild / scalaVersion
Test / test
root / Compile / compile / scalacOptions
ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/scalacOptions
Zero / Zero / name
The inspect command now outputs something that can be copy-pasted:
> inspect compile
[info] Task: sbt.inc.Analysis
[info] Description:
[info] Compiles sources.
[info] Provided by:
[info] ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/compile
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:326
[info] Dependencies:
[info] Compile/manipulateBytecode
[info] Compile/incCompileSetup
[info] Reverse dependencies:
[info] Compile/printWarnings
[info] Compile/products
[info] Compile/discoveredSbtPlugins
[info] Compile/discoveredMainClasses
[info] Delegates:
[info] Compile/compile
[info] compile
[info] ThisBuild/Compile/compile
[info] ThisBuild/compile
[info] Zero/Compile/compile
[info] Global/compile
[info] Related:
[info] Test/compile
This is modification of crash log event logging that was added in sbt/util#85.
Instead of using the hardcoded 0 as the default value, this introduces `setTrace(..)` to `ConsoleAppender` like `BasicLogger`. Also the default value is set to `Int.MaxValue` that will display the full stack trace.
Fixessbt/sbt#3343
Fixes sbt/sbt#3348
Ref #101
The new logger, based on log4j separates the concern of the log producer (Logger) and the handlers that takes actions (Appender, e.g for displaying on Console). As such filtering of color should be performed only in the ConsoleAppender.
Ref #3282
We used to wrap InputStream so it will inject Thread.sleep, which then allows the thread to be cancelled, emulating a non-blocking readLine. This trick doesn't seem to work for Windows.
For non-Cygwin, actually just removing the wrapping does the job, but I couldn't get it to work for Cygwin.
To test, run some command via network, and then type `show name` into the terminal. On Cygwin, it will not respond.
`F0`, `F1` and `Maybe` have become useless since Java 8 introduced
`Supplier`, `Function` and `Optional` in the default Java 8 standard
library.
Therefore, they are not necessary anymore. This change is required to
change some Zinc's and sbt APIs. They are not widely used, so the
changes will be small.
While running scripted, you see
```
ERROR StatusLogger Unable to create custom ContextSelector. Falling back to default.
java.lang.ClassCastException: Cannot cast org.apache.logging.log4j.core.async.AsyncLoggerContextSelector to org.apache.logging.log4j.core.selector.ContextSelector
at java.lang.Class.cast(Class.java:3369)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOf(LoaderUtil.java:201)
at org.apache.logging.log4j.util.LoaderUtil.newCheckedInstanceOfProperty(LoaderUtil.java:226)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.createContextSelector(Log4jContextFactory.java:97)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.<init>(Log4jContextFactory.java:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:94)
at org.apache.logging.log4j.spi.ThreadContextMapFactory.createThreadContextMap(ThreadContextMapFactory.java:73)
at org.apache.logging.log4j.ThreadContext.init(ThreadContext.java:223)
at org.apache.logging.log4j.ThreadContext.<clinit>(ThreadContext.java:202)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createDefaultInjector(ContextDataInjectorFactory.java:83)
at org.apache.logging.log4j.core.impl.ContextDataInjectorFactory.createInjector(ContextDataInjectorFactory.java:67)
at org.apache.logging.log4j.core.lookup.ContextMapLookup.<init>(ContextMapLookup.java:34)
at org.apache.logging.log4j.core.lookup.Interpolator.<init>(Interpolator.java:117)
at org.apache.logging.log4j.core.config.AbstractConfiguration.<init>(AbstractConfiguration.java:125)
at org.apache.logging.log4j.core.config.DefaultConfiguration.<init>(DefaultConfiguration.java:46)
at org.apache.logging.log4j.core.layout.PatternLayout$Builder.build(PatternLayout.java:650)
at org.apache.logging.log4j.core.layout.PatternLayout.createDefaultLayout(PatternLayout.java:487)
at sbt.internal.util.ConsoleAppender.<init>(ConsoleAppender.scala:245)
```
This aims to workaround the casting error during PatternLayout.createDefaultLayout()
that was originally used for ConsoleAppender.
The stacktrace shows it's having issue initializing default DefaultConfiguration.
Since we currently do not use Layout inside ConsoleAppender, the actual pattern is not relevant.
Looks like the reason that util-cache depended on util-collection was to
define the sjson-new formats (HListFormats) for util-collection's HList.
Given that util-collection already depends on sjsonnew, HListFormats can
also be defined in util-collection.
All that was left then was (a) HListFormatSpec requires
sjsonnewScalaJson, so that was added in test scope, and (b) HListFormats
had to be dropped from sbt.util.CacheImplicits - HListFormats will have
to be imported and/or mixed-in where required downstream. For importing
convenience I defined a companion object.
Simply
import OptJsonWriter.OptOut._
And you'll get the implicit lift, but not the implicit fallback.
You get an ambiguous compile error like this:
[error] /d/sbt-util/internal/util-collection/src/main/scala/sbt/util/OptJsonWriter.scala:28: ambiguous implicit values:
[error] both method conflictingFallback1 in trait OptOut0 of type [A]=> sbt.util.NoJsonWriter[A]
[error] and method conflictingFallback2 in trait OptOut0 of type [A]=> sbt.util.NoJsonWriter[A]
[error] match expected type sbt.util.OptJsonWriter[Foo]
[error] val x = implicitly[OptJsonWriter[Foo]]
[error] ^
Uses TypeTag to recover the full name of type parameter, which is calculated by StringTypeTag. This is sent along in ObjectEvent.
On the other end, we can lookup typeclass instances using the tag key.
* Fixes [sbt/sbt#2781]
* When using `<TAB>` completion the failed errors were always
computed for mathcing projects even if there was no failure,
leading to excessive computation of Levenshtein distances
and a large lag (seconds) on builds with many matching
projects.
sbt's shell provided completion only for keys that were relative to a
defined project, but didn't provide completion for keys that belong to
the build definition only.
This commit fixes this issue by defining a new kind of `Parser` (from
which completions are generated) which runs its input simultaneously on
distinct parsers. We now define a parser for project-level keys and
another parser for build-level keys. These two parsers are eventually
combined, and we get the completions of both parsers.
Fixessbt/sbt#2460
Thread sleeping interferes with scripted test when the build cannot be
loaded. The scripted test gets stuck, and jstack shows
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at sbt.internal.util.InputStreamWrapper.read(LineReader.scala:138)
at
jline.internal.NonBlockingInputStream.read(NonBlockingInputStream.java:2
45)
....
at sbt.internal.util.JLine$.withJLine(LineReader.scala:118)
at sbt.internal.util.JLine.readLine(LineReader.scala:18)
at
sbt.BuiltinCommands$.sbt$BuiltinCommands$$doLoadFailed(Main.scala:460)
Looks like the reason that util-cache depended on util-collection was to
define the sjson-new formats (HListFormats) for util-collection's HList.
Given that util-collection already depends on sjsonnew, HListFormats can
also be defined in util-collection.
All that was left then was (a) HListFormatSpec requires
sjsonnewScalaJson, so that was added in test scope, and (b) HListFormats
had to be dropped from sbt.util.CacheImplicits - HListFormats will have
to be imported and/or mixed-in where required downstream. For importing
convenience I defined a companion object.
sbt/util#45 implemented caching using sjson-new. Now many of the functions take `CacheStore` that abstracts the caching ability.
sbt/sbt#3109 demonstrates that setting up CacheStore requires boilerplate involving concepts introduced in sbt 1.
This change adds back overrides using File by making assumption that majority of the time we would want standard JSON converter.
Simply
import OptJsonWriter.OptOut._
And you'll get the implicit lift, but not the implicit fallback.
You get an ambiguous compile error like this:
[error] /d/sbt-util/internal/util-collection/src/main/scala/sbt/util/OptJsonWriter.scala:28: ambiguous implicit values:
[error] both method conflictingFallback1 in trait OptOut0 of type [A]=> sbt.util.NoJsonWriter[A]
[error] and method conflictingFallback2 in trait OptOut0 of type [A]=> sbt.util.NoJsonWriter[A]
[error] match expected type sbt.util.OptJsonWriter[Foo]
[error] val x = implicitly[OptJsonWriter[Foo]]
[error] ^
Uses TypeTag to recover the full name of type parameter, which is calculated by StringTypeTag. This is sent along in ObjectEvent.
On the other end, we can lookup typeclass instances using the tag key.
This introduces ManagedLogger, which is a wrapper around Log4j2's async logging.
Log4j2 separates the notion of logger (the code that collects events) and appender (the code that acts on events). The old code is kept around intentionally to minimize breakage during transition.
Fixes#61
* move it back into style
* use that in Tracked (uncomment some code)
* specify the style in the spec test
* must define absolute file.. (doesn't pass with relative, which is wrong imo)
* Make Nothing a singleton
* Make Maybe's constructor private
* Optimise equals to use reference equality first
* Optimise Just.equals by having proper, non-anonymous subclasses
* Having non-anonymous subclasses makes them have nicer classnames
* Give Just a value() method
* Fixes [sbt/sbt#2781]
* When using `<TAB>` completion the failed errors were always
computed for mathcing projects even if there was no failure,
leading to excessive computation of Levenshtein distances
and a large lag (seconds) on builds with many matching
projects.
Ethan Atkins
Eugene Yokota
Josh Soref
xuwei-k
Ethan Atkins
Dale Wijnand
Guillaume Martres
Kazuhiro Sera
Jason Pickens
Jason Zaugg
Johannes Rudolph
Dale Wijnand
Seth Tisue
Antonio Cunei
Leonard Ehrenfried
Simon Schäfer
kczulko
tom.walford
Olli Helenius
Martin Duhem
jvican