Previously, we were leaking the internal details of incremental
compilation to users by defining FileTree(DataView|Repository)[Stamp].
To avoid this, I introduce the new class FileCacheEntry that is quite
similar to Stamp except defined using scala Options rather than java
Optionals. The implementation class just delegates to an actual Stamp
and I provided a private[sbt] ops class that adds a
method `stamp` to FileCacheEntry. This will usually just extract the
stamp from the implementation class. This allows us to use
FileCacheEntry almost interchangeably with Stamp while still avoiding
exposing users to Stamp.
In the FileTreeDataView use case, we were previously working with
FileTreeDataView[Stamped], which actually contained a lot of redundant
information because FileTreeDataView.Entry[_] has a toTypedPath method
that could be used to read the path related fields in Stamped. Instead,
we can just return the Stamp itself in FileTreeDataView.list* methods
and convert to Stamped.File where needed (i.e. in ExternalHooks).
Also move BasicKeys.globalFileTreeView to Keys since it isn't actually
used in the main-command project.
Resident compilation actually works pretty well most of the time*,
but if there ever is an issue with the cached compilation, we should be
able to easily clear the cache.
* I've only had issues when package objects are involved
If the managedSources task writes into an unmanaged source directory,
that would cause an infinite loop. I don't think it's worth doing out of
band task execution to try and prevent this.
The community build was broken for some projects because I broke builds
that relied on the unscoped definition of `runner`. To preserve legacy
behavior, I restore the old unscoped behavior and append the new scoped
runners that used the layered classloaders. This makes more sense
because the layered classloaders were specifically designed for the
Runtime and Test configurations and may not make sense in other
contexts.
I noticed that sometimes when running scripted tests that I'd run out of
metaspace. I believe that this may be due to the caffeine cache leaking
classloaders. Regardless, it's a good idea to clear the cache whenever
we shutdown the command exchange or reload the state.
Previously, the ClassLoaderLayeringStrategy was set globally. This
didn't really make sense because the Runtime and Test configs had
different strategies available (Test being a superset of Runtime).
Instead, we now set the layering strategy in the Runtime and Test
configurations directly. In doing this, we can eliminate the Default
ClassLoaderLayeringStrategy. Previously this had existed so that we
could set the layering strategy globally and have it do the right thing
in both test and runtime.
To implement this, I factored out the logic for generating the layered
classloader in the test task and shared it with the runtime task. I did
this because I realized that Test / run is a thing. Previously I had
been operating under the assumption that the runner would never include
the test dependencies. Once I realized this, it made sense to combine
the logic in both tasks.
As a bonus, I only allow the layering strategies that explicitly make
sense to be set in each configuration. If the user sets an invalid
strategy, an error will be thrown that specifies the valid strategies
for the task.
I also added ScalaInstance as an option for the runtime layer. It was an
oversight that this was left out.
In code review, @eed3si9n suggested that I switch to a more verbose and
descriptive naming scheme. In addition to trying to make layers more
descriptive, I also made the various layer case objects extend the
relevant layers so it's more clear what the layer should look like.
I was seeing spurious travis failures and I finally tracked it down to
the fact that in some cases, the project metabuild would sometimes use a
caching file tree repository instead of a polling repository. This
caused problems because the caching repository can take a few milliseconds
to detect changes in a directory. Because scripted copies the project
sources to the temporary test directory, it was possible for the project
meta build compilation to be initiated before the cache was aware of all
of the files.
The reason this happened was because scripted would create a state where
the remaining commands looked like:
List(sbtPopOnFailure, resumeFromFailure, notifyUsersAboutShell, iflast shell, ~compile, < 41684)
The ~compile command was causing the continuous flag to get set to true
which caused the default file tree repository task to return the caching
version. The reason for the continuous flag was so that when sbt is
started in a non-interactive mode where the command is to be repeated,
then we use the caching file tree repository. To support this use case,
we just need to check that the last command begins with `~`, not that
_any_ command begins with `~`.
We want the user to be able to invalidate the classloader cache in the
event that it somehow gets in a bad state. The cache is, however,
defined in multiple configurations, so there are in fact many
ClassLoaderCache instances that are managed by sbt. To make this sane, I
add a global cache that is keyed by a TaskKey[_] and can return
arbitrary data back. Invalidating all of the ClassLoaderCache instances
is then as straightforward as just replacing the TaskRepository
instance.
I also went ahead and unified the management of the global file tree
repository. Instead of having to specifically clear the file tree
repository or the classloader cache, the user can now invalidate both
with the new clearCaches command.
Using the data structures that I added in the previous commits, it is
now possible to rework the run and test task to use (configurable)
layered class loaders. The layering strategy is globally set to
LayeringStrategy.Default. The default strategy leads to what is
effectively a three layered ClassLoader for the both the test and run
tasks. The first layer contains the scala instance (and test framework
loader in the test task). The second layer contains all of the
dependencies for the configuration while the third layer contains the
project artifacts.
The layering strategy is very easily changed both at the Global or
Configuration level, e.g. adding
Test / layeringStrategy := LayeringStrategy.Flat
to the project build.sbt will make the test task not even use the scala
instance and instead a create a single layer containing the full
classpath of the test task.
I also tried to ensure that all of the ClassLoaders have good toString
overrides so that it's easy to see how the ClassLoader is constructed
with, e.g. `show testLoader`, in the sbt console.
In this commit, the ClassLoaderCache instances are settings. In the next
commit, I make them tasks so that we can easily clear out the caches
with a command.
This introduces a new trait LayeringStrategy that is used to configure
how sbt constructs the ClassLoaders used by the run and test tasks. In
addition to defining the various options, I try to give a good high
level overview of the problem that the LayeringStrategy is intended to
address in its scaladoc.
In order to speed up the start up time of the test and run tasks, I'm
introducing a ClassLoaderCache that can be used to avoid reloading the
classes in the project dependencies (which includes the scala library).
I made the api as minimal as possible so that we can iterate on the
implementation without breaking binary compatibility. This feature will
be gated on a feature flag, so I'm not concerned with the cache class
loaders being useable in every user configuration. Over time, I hope
that the CachedClassLoaders will be a drop in replacement for the
existing one-off class loaders*.
The LayeredClassLoader was adapted from the NativeCopyLoader. The main
difference is that the NativeCopyLoader extracts named shared libraries
into the task temp directory to ensure that the ephemeral libraries are
deleted after each task run. This is a problem if we are caching the
ClassLoader so for LayeredClassLoader I track the native libraries that
are extracted by the loader and I delete them either when the loader is
explicitly closed or in a shutdown hook.
* This of course means that we both must layer the class loaders
appropriately so that the project code is in a layer above the cached
loaders and we must correctly invalidate the cache when the project, or
its dependencies are updated.
I am going to be introducing multiple caches throughout sbt and I am
going to build these features out using this simple Repository
interface. The idea is that we access data by some key through the
Repository. This allows us to use the strategy pattern to easily switch
the runtime implementation of how to get the data.
I am going to add a classloader cache to improve the startup performance
of the run and test tasks. To prevent the classloader cache from having
unbounded size, I'm adding a simple LRUCache implementation to sbt. An
important characteristic of the implementation of the cache is that when
entries are evicted, we run a callback to cleanup the entry. This allows
us to automatically cleanup any resources created by the entry.
This is a pretty naive implementation that uses an array of entries that
it manipulates as elements are removed/accessed. In general, I expect
these caches to be quite small <= 4 elements, so the storage overhead /
performance of the simple implementation should be quite good. If
performance ever becomes an issue, we can specialzed LRUCache.apply to
use a different implementation for caches with large limits.
The `sbt-server` was prepending a new probem and not appending.
The result was a `textDocument/publishDiagnostics` notification
containing a inverted list of problems compare to what was show in the
sbt console.
It drives me crazy that in intellij when I do the go to class task that
TestBuild.Keys comes up before Keys. Given how central Keys is to sbt,
it doesn't seem like a good idea to alias that particular class name.
It was becoming a pain to work on these files in intellij because the
auto-import feature would implicitly optimize all of the imports in
these files, leading to a large diff. I'd then have to go and manually
add the import that I care about. This change does add some wildcard
imports, which I don't always love, but these files are so unwieldy
already that I think it's worth it to have the imports follow the format
preferred by intellij.
It has long been a frustration of mine that it is necessary to prepend
multiple commands with a ';'. In this commit, I relax that restriction.
I had to reorder the command definitions so that multi comes before act.
This was because if the multi command did not have a leading semicolon,
then it would be handled by the action parser before the multi command
parser had a shot at it. Sadness ensued.
In #4446, @japgolly reported that in some projects, if a parent project
was broken, then '~' would immediately exit upon startup. I tracked it
down to this managed sources filter. The idea of this filter is to avoid
getting stuck in a build loop if managedSources writes into an unmanaged
source directory. If the (managedSources in ThisScope).value line
failed, however, it would cause the watchSources, and by delegation,
watchTransitiveSources task to fail. The fix is to only create this
filter if the managedSources task succeeds.
I'm not 100% sure if we shouldn't just get rid of this filter entirely
and just document that '~' will probably loop if a build writes the
result of managedSources into an unmanaged source directory.
Fixes#4437
Until now, sbt was resolved twice once by the launcher, and the second time by the metabuild.
This excludes sbt from the metabuild graph, and instead uses app classpath from the launcher.
Fixes#3436
This implements isMetaBuild setting that is explicitly for meta build only,
unlike sbtPlugin setting which can be used for both meta build and plugin development purpose.
I noticed that when using the latest nightly, triggered execution would
fail to work if I switched projects with, e.g. ++2.10.7. This was
because the background thread that filled the file cache was incorrectly shutdown.
To fix this, we just need to close whatever view is cached in the
globalFileTreeView attribute in the exit hook rather than the view
created by the method.
After making this change and publishing a local SNAPSHOT build, I was
able to switch projects with ++ and have triggeredExecution continue to
work.
On windows* it was possible to get into a loop where the build would
continually restart because for some reason the build.sbt file would get
touched during test (I did not see this behavior on osx). Thankfully,
the repository keeps track of the file hash and when we detect that the
build file has been updated, we check the file hash to see if it
actually changed.
Note that had this bug shipped, it would have been fixable by overriding
the watchOnEvent task in user builds.
The loop would occur if I ran ~filesJVM/test in
https://github.com/swoval/swoval. It would not occur if I ran
test:compile, so the fact that the build file is being touched seems
to be related to the test run itself.
It was possible that on startup, when this function was first invoked,
that the default boot commands are present. This was a problem because
the global file repository is instantiated using the value of this task.
When we start a continuous build, this task gets run again to evaluate
again.
When sbt is started without an implicit task list, then the task is
implicitly shell as indicated by the command "iflast shell". We can use
this to determine whether or not to use the global file system cache or
not.
Ideally we use the FileTreeRepository for interactive sessions by
default. A continuous build is effectively interactive, so I'd like that
case to also use the file tree repository. To avoid breaking scripted
tests, many of which implicitly expect file tree changes to be
instantaneously available, we set interactive to true only if we are not
in a scripted run, which can be verified by checking that the commands
contains "setUpScripted".
Sometimes a user may want to rerun their task even if the source files
haven't changed. Presently this is a little annoying because you have to
hit enter to stop the build and then up arrow or <ctrl+r> plus enter to
rebuild. It's more convenient to just be able to press the 'r' key to
re-run the task.
To implement this, I had to make the watch task set up a jline terminal
so that System.in would be character buffered instead of line buffered.
Furthermore, I took advantage of the NonBlockingInputStream
implementation provided by jline to wrap System.in. This was necessary
because even with the jline terminal, System.in.available doesn't return
> 0 until a newline character is entered. Instead, the
NonBlockingInputStream does provide a peek api with a timeout that will
return the next unread key off of System.in if there is one available.
This can be use to proxy available in the WrappedNonBlockingInputStream.
To ensure maximum user flexibility, I also update the watchHandleInput Key to
take an InputStream and return an Action. This setting will now receive
the wrapped System.in, which will allow the user to create their own
keybindings for watch actions without needing to use jline themselves.
Future work might make it more straightforward to go back to a line
buffered input if that is what the user desires.
For projects with a large number of files, zinc has to do a lot of work
to determine which source files and binaries have changes since the last
build. In a very simple project with 5000 source files, it takes roughly
750ms to do a no-op compile using the default incremental compiler
options. After this change, it takes about 200ms. Of those 200ms, 50ms
are due to the update task, which does a partial project resolution*.
The implementation is straightforward since zinc already provides an api
for overriding the built in change detection strategy. In a previous
commit, I updated the sources task to return StampedFile rather than
regular java.io.File instances. To compute all of the source file
stamps, we simply list the sources and if the source is in fact an
instance of StampedFile, we don't need to compute it, otherwise we
generate a StampedFile on the fly. After building a map of stamped files
for both the sources files and all of the binary dependencies, we simply
diff these maps with the previous results in the changedSources,
changedBinaries and removedProducts methods.
The new ExternalHooks are easily disabled by setting
`externalHooks := _ => None`
in the project build.
In the future, I could see moving ExternalHooks into the zinc project so
that other tools like bloop or mill could use them.
* I think this delay could be eliminated by caching the UpdateResult so
long as the project doesn't depend on any snapshot libraries. For a
project with a single source, the no-op compile takes O(50ms) so caching
the project resolution would make compilation start nearly
instantaneous.
I realized that using the cache has the potential to cause issues for
batch processing in CI if some tasks assume that a file created by one
task will immediately be visible in the other. With the cache, there is
typically on O(10ms) latency between a file being created and appearing
in the cache (at least on OSX). When manually running commands, that
latency doesn't matter.
It is not always possible to monitor a directory using OS file system
events. For example, inotify does not work with nfs. To work around
this, I add support for a hybrid FileTreeViewConfig that caches a
portion of the file system and monitors it with os file system
notification, but that polls a subset of the directories. When we query
the view using list or listEntries, we will actually query the file
system for the polling directories while we will read from the cache for
the remainder. When we are not in a continuous build (~ *), there is no
polling of the pollingDirectories but the cache will continue to update
the regular directories in the background. When we are in a continuous
build, we use a PollingWatchService to poll the pollingDirectories and
continue to use the regular repository callbacks for the other
directories.
I suspect that #4179 may be resolved by adding the directories for which
monitoring is not working to the pollingDirectories task.
Now that we have the fileTreeView task, we can generalized the process
of collecting files from the view (which may or may not actually cache
the underlying file tree). I moved the implementation of collectFiles
and addBaseSources into the new FileManagement object because Defaults
is already too large of a file. When we query the view, we also need to
register the directory we're listing because if the underlying view is a
cache, we must call register before any entries will be available.
Because FileTreeDataView doesn't have a register method, I implement
registration with a simple implicit class that pattern matches on the
underlying type and only calls register if it is actually a
FileRepository.
A side effect of this change is that the underlying files returned by
collectFiles and appendBaseSources are StampedFile instances. This is so
that in a subsequent commit, I can add a Zinc external hook that will
read these stamps from the files in the source input array rather than
compute the stamp on the fly. This leads to a substantial reduction in
Zinc startup time for projects with many source files. The file filters
also may be applied more quickly because the isDirectory property (which
we check for all source files) is read from a cached value rather than
requiring a stat.
I had to update a few of the scripted tests to use the `1.2.0`
FileTreeViewConfig because those tests would copy a file and then
immediately re-compile. The latency of cache invalidation is O(1-10ms),
but not instantaneous so it's necessary to either use a non-caching
FileTreeView or add a sleep between updates and compilation. I chose the
former.
Every time that the compile task is run, there are potentially a large
number of iops that must occur in order for sbt to generate the source
file list as well as for zinc to check which files have changed since
the last build. This can lead to a noticeable delay between when a build
is started (either manually or by triggered execution) and when
compilation actually begins. To reduce this latency, I am adding a
global view of the file system that will be stored in
BasicKeys.globalFileTreeView.
To make this work, I introduce the StampedFile trait, which augments the
java.io.File class with a stamp method that returns the zinc stamp for
the file. For source files, this will be a hash of the file, while for
binaries, it is just the last modified time. In order to gain access to
the sbt.internal.inc.Stamper class, I had to append addSbtZinc to the
commandProj configurations.
This view may or may not use an in-memory cache of the file system tree
to return the results. Because there is always the risk of the cache
getting out of sync with the actual file system, I both make it optional
to use a cache and provide a mechanism for flushing the cache. Moreover,
the in-memory cache implementation in sbt.io, which is backed by a
swoval FileTreeRepository, has the property that touching a monitored
directory invalidates the entire directory within the cache, so the
flush command isn't even strictly needed in general.
Because caching is optional, the global is of a FileTreeDataView, which
doesn't specify a caching strategy. Subsequent commits will make use of
this to potentially speed up incremental compilation by caching the
Stamps of the source files so that zinc does not need to compute the
hashes itself and will allow for continuous builds to use the cache to
monitor events instead of creating a new, standalone FileEventMonitor.
There may be instances where the user may wish to stop the watch if an
error occurs running the task. To facilitate this, I add boolean
parameter, lastStatus, to watchShouldTerminate. The value is computed by
modifying the state used to run the task to have a custom onFailure
command. If the task fails, the returned state will have the onFailure
command will be enqueued at the head of the remaining commands. The
result of the task then becomes true if the custom onFailure is not
present in the remaining commands and false if it is. We don't actually
run this command, so it's just implemented with the identity function.
I also updated Watched.watch to return an Action instead of Unit. This
enables us to return a failed state if Watched.watch returns
HandleError.
This commit reworks Watched to be more testable and extensible. It also
adds some small features. The previous implementation presented a number
of challenges:
1) It relied on external side effects to terminate the watch, which was
difficult to test
2) It exposed irrelevant implementation details to the user in the
methods that exposed the WatchState as a parameter.
3) It spun up two worker threads. One was to monitor System.in for user
input. The other was to poll the watch service for events and write
them to a queue. The user input thread actually broke '~console'
because nearly every console session will hit the <enter> key, which
would eventually cause the watch to stop when the user exited the
console.
To address (1), I add the shouldTerminate method to WatchConfig. This
takes the current watch iteration is input and if the function returns
true, the watch will stop.
To address (2), I replace the triggeredMessage and watchingMessage keys
with watchTriggeredMessage and watchStartMessage. The latter two keys
are functions that do not take the WatchState as parameters. Both
functions take the current iteration count as a parameter and the
watchTriggeredMessage also has a parameter for the path that triggered
the build.
To address (3), I stop using the sbt.internal.io.EventMonitor and
instead use the sbt.io.FileEventMonitor. The latter class is similar to
the former except that it's polling method accepts a duration, which may
be finite or infinite) and returns all of the events that occurred since
it was last polled. By adding the ability to poll for a finite amount of
time, we can interleave polling for events with polling System.in for
user input, all on the main thread. This eliminates the two extraneous
threads and fixes the '~console' use case I described before.
I also let the user configure the function that reads from System.in via
the watchHandleInput method. In fact, this method need not read from
System.in at all since it's just () => Watched.Action. The reason that
it isn't () => Boolean is that I'd like to leave open the option for the
ability to trigger a build via user input, not just terminating the
watch. My initial idea was to add the ability to type 'r' to re-build in
addition to <enter> to exit. This doesn't work without integrating
jline though because the input is buffered. Regardless, for testing
purposes, it gives us the ability to add a timeout to the watch by
making handleInput return true when a deadline expires.
The tests are a bit wonky because I still need to rely on side effects
in the logging methods to orchestrate the sequence of file events that
I'd like to test. While I could move some of this logic into a
background thread, there still needs to be coordination between the
state of the watch and the background thread. I think it's easier to
reason about when all of the work occurs on the same thread, even if it
makes these user provided functions impure.
I deprecated all of the previous watch related keys that are no longer
used with the new infrastructure. To avoid breaking existing builds, I
make the watchConfig task use the deprecated logging methods if they are
defined in the user's builds, but sbt will not longer set the default
values. For the vast majority of users, it should be straightforward to
migrate their builds to use the new keys. My hunch is that the of the
deprecated keys, only triggeredMessage is widely used (in conjunction
with the clear screen method) and it is dead simple to replace it with
watchTriggeredMessage.
Note: The FileTreeViewConfig class is not really necessary for this commit.
It will become more important in a subsequent commit which introduces an
optional global file system cache.
This commit makes watch event logging work in the '~' command. The
previous design of the command made this difficult, so there is a
significant re-design of the implementation of '~'. I believe that this
redesign will allow the feature to be maintained and improved more
easily moving forward. With the redesign, it is now possible to test the
business logic of the watch command (and I add a rudimentary test that I
will build upon in subsequent commits).
A bonus of this redesign is that now if the user tries to watch an
invalid command, the watch will immediately terminate with an error
rather than get stuck waiting for events when the task can never
possibly succeed.
The previous implementation of the '~' command makes it difficult
to dynamically control the implementation arguments because it is
implemented in the command project which makes it unable to depend on
any task keys that are defined in the build. It works around this by
putting all of it's configuration in the Watched attribute which is
stored globally. This would not have been necessary if the function had
been defined in the main project where it could just extract the value
of the watched task rather than relying on the global attribute value.
Moreover, because it cannot depend on tasks, it makes it nigh impossible
to use the logging framework within the '~' command.
Another issue with the previous implementation is that it's somewhat
difficult to reason about. The executeContinuously has effectively two
entry points: one for the first time the command is run and one for each
subsequent invocation when a new build is triggered. The successive
invocations are triggered by prepending commands to run to the previous
state. This is made recursive by prepending the initial command (that
was prefixed with '~'. Which branch we're in is determined by checking
for the existence of a temporary attribute, that we must ensure that we
remove when the build is stopped. This makes a lot of behavior non-local and
difficult for an outsider who is less familiar with sbt to understand.
Broadly, this refactor does two things:
1) Move the definition of continuous from BasicCommands to BuiltInCommands
2) Re-work the implementation to be executed in code rather than using
the sbt dsl.
The first part is simple. We just add an implementation of continuous to
BuiltInCommands and remove it from the list of BasicCommands. We need to
leave in the legacy implementation for binary compatibility. I also
moved all of the actual implementation logic into Watched, which makes
maintenance easier since most of the logic is in one place.
The second part is more complicated. Rather than rely on the sbt dsl
(e.g. `(ClearOnFailure :: next :: FailureWall :: repeat :: s)`) to
parse and run the command. We manually parse the command and generate a
task of type `() => State`. We don't actually need to do anything with
the generated state because we're going to return the original state at
the end of the command no matter what. With this task, we can then
create a tail recursive function that repeatedly executes the task until
the watch is terminated.
The parsing is handled in the Watch.command method (which is where I
moved the refactored BasicCommands.continuous implementation). The
actual task running and monitoring is handled in Watched.watch. This
method has no reference to the sbt state, which makes it testable. It sets
up an event monitor and then delegates the recursive monitoring to a
small nested function, Watched.watch.impl. One nice thing about this
approach is that it is very easy to reason about the life cycle of the
EventMonitor. The recursive call is within a try { } finally { } where
the monitor and stdin are guaranteed to be cleared at the end.
Adding support for a custom (and default) watch logger is trivial with
the new infrastructure and is done via the watchLogger TaskKey.
There was a small reporting race condition that was introduced by the
change to (2). Because the new implementation is able to bypass command
parsing for triggered builds, the watch message would usually end up
being printed before the task outcome was fully logged. To work around
this, I made the watch and triggered messages be logged rather than
printed directly to stdout. As a result, the only user visible result of
this change should be that instead of seeing:
"1. Waiting for source changes in project foo... (press enter to interrupt)",
users will now see:
"[info] 1. Waiting for source changes in project foo... (press enter to interrupt)".
There have been reports that often a new build will be triggered
immediately after the previous build even when none of the files have
been modified since the start of the last build. This can happen when,
for example, a program implements save with a rename. When that occurs,
a deletion watch event may trigger the build but the corresponding
creation event may be detected outside of the current 40ms window. By
bumping this value to 500ms, we hopefully prevent the majority of these
false triggers. For unusual workflows in which this longer quarantine
period is an issue, the setting can be overridden.
This change makes the temporary shared library that is created by
the swoval file-tree-views library to be extracted into the sbt global
base directory rather than the temp file. This way if there is a leak of
shared libraries, they can easily be found in ~/.sbt rather than in,
say, /tmp (or the osx/windows equivalent location). The extracted shared
library objects will be in the path ~/.sbt/swoval-jni. There is a
shutdown hook that removes them as well as a garbage collection process
that runs in the background whenever the swoval library is loaded, so
these shouldn't leak uncontrollably.
I had to turn off -Xfatal-warnings in commandProj because after updating
io, commandProj depends on the deprecated EventMonitor class. In #4335,
I stop using EventMonitor, but deprecate the Watched class which is both
defined and used (as an unused attribute key) in commandProj. I think we
can probably get rid of Watched in 1.4.x and certainly in a hypothetical
2.x, so hopefully we can restore -Xfatal-warnings sooner than later.
I also had to replace uses of IO.classLocationFile with
IO.classLocationPath to avoid compilation failures due to
-Xfatal-warnings.
Fixes#4362
This implements an instance of ExecuteProgress that is enabled by default to report progress on the shell.
Combined with the scroll up logger, this takes over the bottom lines of the terminal screen and display a count up clock of the currently executing task.
The end goal is to rewrite Dotty's compiler-bridge in Java (this is easy
since the zinc-specific phases are in the compiler itself) to simplify
the bootstrapping process.
Change CommandExchange#publishEvent to not broadcast ExecStatusEvent and
instead check the channelName, this matches what was already done in
CommandExchange#publishEventMessage
Fixes#4293
Ref #4231, #4065
This fixes the regression on sbt 1.2.0 that displays a lot of warnings about configurations.
The warning was added in #4231 in an attempt to fix#4065. This actually highlights somewhat loose usage of configurations among the builds in the wild, and the limitation on the current slash syntax implementation.
I think we can remove this warning for now, and try to fix#4065 by making slash syntax more robust. In particular, we need to memorize the mapping between the configuration name and Scala identifier across the entire build, and use that in the shell.
Ref https://github.com/sbt/sbt/issues/4121
sbt already has the facility to trim stack traces. This sets the trace level of the background run, which fixes the upper half of the `run` stacktrace.
```
[error] (run-main-0) java.lang.Exception
[error] java.lang.Exception
[error] at Hello$.delayedEndpoint$Hello$1(Hello.scala:5)
[error] at Hello$delayedInit$body.apply(Hello.scala:1)
[error] at scala.Function0.apply$mcV$sp(Function0.scala:34)
[error] at scala.Function0.apply$mcV$sp$(Function0.scala:34)
[error] at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
[error] at scala.App.$anonfun$main$1$adapted(App.scala:76)
[error] at scala.collection.immutable.List.foreach(List.scala:389)
[error] at scala.App.main(App.scala:76)
[error] at scala.App.main$(App.scala:74)
[error] at Hello$.main(Hello.scala:1)
[error] at Hello.main(Hello.scala)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:498)
[error] java.lang.RuntimeException: Nonzero exit code: 1
[error] at sbt.Run$.executeTrapExit(Run.scala:127)
[error] at sbt.Run.run(Run.scala:77)
[error] at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1254)
[error] at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1249)
[error] at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:377)
[error] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] at scala.util.Try$.apply(Try.scala:209)
[error] at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:299)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (Compile / run) Nonzero exit code: 1
```
The bottom half requires a similar fix to the foreground log.
Fixes https://github.com/sbt/sbt/issues/3508
This forks an instance of sbt in the background when it's not running already.
```
$ time sbt -client compile
Getting org.scala-sbt sbt 1.2.0-SNAPSHOT (this may take some time)...
:: retrieving :: org.scala-sbt#boot-app
confs: [default]
79 artifacts copied, 0 already retrieved (28214kB/130ms)
[info] entering *experimental* thin client - BEEP WHIRR
[info] server was not detected. starting an instance
[info] waiting for the server...
[info] waiting for the server...
[info] server found
> compile
[success] completed
sbt -client compile 9.25s user 2.39s system 33% cpu 34.893 total
$ time sbt -client compile
[info] entering *experimental* thin client - BEEP WHIRR
> compile
[success] completed
sbt -client compile 3.55s user 1.68s system 107% cpu 4.889 total
```
There's also a special case for aliases that will try to resolve
the target of the alias to a task key if possible and display the
output of that key if found.
see https://github.com/sbt/sbt/issues/2881
Fixes https://github.com/sbt/sbt/issues/1502
This adds `--addPluginSbtFile=<file>` command, which adds the given .sbt file to the plugin build.
Using this mechanism editors or IDEs can start a build with required plugin.
```
$ cat /tmp/extra.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.7")
$ sbt --addPluginSbtFile=/tmp/extra.sbt
...
sbt:helloworld> plugins
In file:/xxxx/hellotest/
...
sbtassembly.AssemblyPlugin: enabled in root
```
I spent a lot of time debugging why it took 5 seconds to run tests each
time. It turns out that if the hostname is not set explicitly on os x,
then getaddrinfo takes 5 seconds to try (and fail) to resolve the dns
entry for the localhostname. This is easily fixed by setting the
hostname, but it is not at all easy to figure out that a slow hostname
lookup is the reason why tests are slow to start.
I don't know if this is a common issue on other platforms, so only issue
the warning on OS X.
It turns out that `syntaxAnalyzer.UnitParser()` in global now also
needs to be synchronized. The alternative is adding `synchronizeNames = true`
in the global constructor, but that already proved unreliable in the
case of #3743 (see comment https://github.com/sbt/sbt/issues/3170#issuecomment-355218833)
When source generators write into the unmanaged source directory, bad
things can happen. Continuous builds will loop indefinitely and
compiling will fail because the generated sources get added to the
source list twice, causing the incremental compiler to complain about
compiling classes it has already seen. My two-pronged solution is to
de-duplicate the sources task and to filter out managed source files in
watch sources. The drawback to the latter is that it causes the source
generation task to be executed twice per compile.
In https://github.com/sbt/io/pull/142, I add a new api for watching for
source file events. This commit updates sbt to use the new EventMonitor
based api. The EventMonitor has an anti-entropy parameter, so that
multiple events on the same file in a short window of time do not
trigger a build. I add a key to tune it.
The implementation of executeContinuously is pretty similar. The main
changes are that shouldTerminate now blocks (EventMonitor spins up a
thread to check the termination condition) and that the
EventMonitor.watch method only returns a Boolean. This is because
the event monitor contains mutable state. It does, however, have a
state() method that returns an immutable snapshot of the state.
The existing filter caused SourceModificationWatch.watch to ignore
deleted files because !file.exists implies !file.isFile. The intention
of the filter was to exclude directories that had a name ending in
".scala".
Fixes#3849
This brings back the 0.13 logic:
```scala
def setGlobalLogLevel(s: State, level: Level.Value): State = {
s.globalLogging.full match {
case a: AbstractLogger => a.setLevel(level)
case _ => ()
}
s.put(BasicKeys.explicitGlobalLogLevels, true).put(Keys.logLevel.key, level)
}
```
* 1.1.x:
Update mimaPreviousArtifacts/sbt.version
Introduce SBT_GLOBAL_SERVER_DIR env var to override too long paths
Handle very long socket file paths on UNIX
Conflicts:
project/build.properties
This works around the name conflict between sbt.test package and sbt.Keys.test.
1. sbt.test package is renamed to sbt.scriptedtest. This allows 1.0 plugins and builds to use `test` to mean `Keys.test`.
2. To keep binary compatibility for sbt 0.13 scripted, I am adding `sbt.test.ScriptedRunner` and `sbt.test.ScriptedTests` in `scripted-plugin` artifact.
3. Another affected user is Giter8 plugin that uses ScriptedPlugin. Since the intereactions are limited to `sbt.ScriptedPlugin.*`, we should be fine here. - https://github.com/foundweekends/giter8/blob/v0.11.0-M2/plugin/src/main/scala-sbt-1.0/giter8/SBTCompat.scala
Fixes#3538
This brings in `sbt.ScriptedPlugin` as `sbt.plugins.ScriptedPlugin` into sbt mothership.
In addition, `sbt.plugins.SbtPlugin` is added that enables the scripted plugin and `sbtPlugin := true`.
This allows plugin authors to bring in scripted plugin by writing:
```scala
lazy val root = (project in file("."))
.enablePlugins(SbtPlugin)
```
Fixes#3841
This fixes console task that internally uses JLine. When `console` is started from batch mode, the tab is printed as is. This is because JLine is not initialized yet.
Calling `usingTerminal` initializes and restores the terminal afterwards.
There are just too many instances in which sbt's code relies on
the `lastModified`/`setLastModified` semantics, so instead of moving
to `get`/`setModifiedTime`, we use new IO calls that offer the new
timestamp precision, but retain the old semantics.
Previously I was seeing the error upon the first scripted test. I thought it was because Main was somehow not early enough. It might just be because scripted technically runs as part of the build.
Ref sbt/io#110
Fixes#3823
When you launch a second instance of sbt on a build, prior to this change it was displaying `java.io.IOException: sbt server is already running` on every command. This make it a bit less aggressive, and just display a warning once.
```
[warn] Is another instance of sbt is running on this build?
[warn] Running multiple instances is unsupported
```
Even with `publishArtifact := false` the user is still forced to define a (dummy) resolver that's never used, e.g. `publishTo := { Some("publishMeNot" at "https://publish/me/not") }`
Otherwise the following error is thrown:
```
publish
[error] java.lang.RuntimeException: Repository for publishing is not specified.
[error] at scala.sys.package$.error(package.scala:27)
[error] at sbt.Classpaths$.$anonfun$getPublishTo$1(Defaults.scala:2436)
[error] at scala.Option.getOrElse(Option.scala:121)
[error] at sbt.Classpaths$.getPublishTo(Defaults.scala:2436)
[error] at sbt.Classpaths$.$anonfun$ivyBaseSettings$48(Defaults.scala:1917)
```
This is to avoid it initialising Log4J2 (via SLF4J), which we initialise
ourselves programmatically in LogExchange. Also there's no need to
removeAll in initialState.
Fixes#3787
Ref https://github.com/sbt/io/pull/96
Under RFC 8089, both u1 and u3 are legal, but many of the other platforms expect traditional u3.
This will increase the compatibility/usability of sbt server, for example to integrate with Vim.
Although in theory the fix in #3776 should be preferable to
synchronize templateStats() manually, it turns out that we
still get errors in some tests. So, reverting to a
synchronized section while we investigate.
This reverts commit ee90917cc4.
Fixes#3786
To configure the log level of the server, this introduces a new task key named `serverLog`. The idea is to set this using `Global / serverLog / logLevel`. It will also check the global log level, and if all else fails, fallback to Warn.
```
lazy val level: Level.Value = (s get serverLogLevel) orElse (s get logLevel) match {
case Some(x) => x
case None => Level.Warn
}
```
`NGUnixDomainSocket` throws `java.io.IOException` instead of `SocketException`, probably because `SocketException` does not expose the contructor with a `Throwable` parameter.
To allow clients to disconnect, we need to catch `IOException`.
This is an implementation of `textDocument/definition` request.
Supports types only, and only in case when type is found in Zinc Analysis. When source(s) are found then editor opens potential source(s).
This simple implementation does not use semantic data.
During the processing of `textDocument/didSave`, we will start collecting the location of Analysis files via `lspCollectAnalyses`.
Later on, when the user asked for `textDocument/definition`, sbt server will invoke a Future call to lspDefinition, which direct reads the files to locate the definition of a class.
In addition to TCP, this adds sbt server support for IPC (interprocess communication) using Unix domain socket and Windows named pipe.
The use of Unix domain socket has performance and security benefits.
The creation of a backgroundLog was always using Debug as the
logging level for console and backing. This commit sets the
levels to those used by the caller. Fixes#3655
This adds a new option `dev` to the `reboot` command, which deletes the only the current sbt artifacts from the boot directory. `reboot dev` reads actively from `build.properties` instead of using the current state since `reboot` can restart into another sbt version.
In general, `reboot dev` is intended for the local development of sbt.
Fixes#3590
Using a recursive Source meant that ~ looked into target. If you have
any source generators and use ~ with anything the invokes them, like
~compile, that means that the act of generating sources triggers ~ to
re-execute compile (perhaps only on macOS where the NIO WatchService
just polls, after an initial delay).
Requires sbt/io#78
Fixes#3501
This adds a sbt.watch.mode system property that if set to 'polling' will
use PollingWatchService instead of WatchServiceAdapter (nio).
On macOS this will default to 'polling' and on all others 'nio'.
This is a temporary workaround for users affected by #3527
This adds a sbt.watch.mode system property that if set to 'polling' will
use PollingWatchService instead of WatchServiceAdapter (nio).
On macOS this will default to 'polling' and on all others 'nio'.
This is a temporary workaround for users affected by #3527
In ca71b4b902 I went about fixing the
inexhaustive matching in Scope's resolveProjectBuild and
resolveProjectRef. Looking back the change was wrong.
For resolveProjectBuild the new implementation is less wrong, but still
not great, seeing as it doesn't actually do any build resolving.
For resolveProjectRef the new implementation now blows up instead of
lies. Which means it's less leneant, more "fail-fast".
isProjectThis is unused; remnant of the pre-AutoPlugin days when build
settings where defined in Plugin.settings.
toString added for REPL testing:
```
scala> Zero / Zero / Zero / name
res0: sbt.SlashSyntax.ScopeAndKey[sbt.SettingKey[String]] = Zero / Zero / Zero / name
```
Prior to this change `Zero / Zero / Zero / name` broke as folllows:
```
scala> Zero / Zero / Zero / name
Zero / Zero / Zero / name
<console>:18: error: inferred type arguments [sbt.Zero.type] do not conform to method /'s type parameter bounds [K <: sbt.SlashSyntax.Key[K]]
Zero / Zero / Zero / name
^
```
This is the first cut for the Language Server Protocol on top of server that is still work in progress.
With this change, sbt is able to invoke `compile` task on saving files in VS Code.
```
Provided by:
ProjectRef(uri("...."), "root") / Test / test
Dependencies:
Test / executeTests
Test / test / streams
Test / state
Test / test / testResultLogger
```
Fixessbt/sbt#1812
This adds unified slash syntax for both sbt shell and the build.sbt DSL.
Instead of the current `<project-id>/config:intask::key`,
this adds `<project-id>/<config-ident>/intask/key` where <config-ident> is the Scala identifier notation for the configurations like `Compile` and `Test`.
This also adds a series of implicits called `SlashSyntax` that adds `/` operators to project refererences, configuration, and keys such that the same syntax works in build.sbt.
These examples work for both from the shell and in build.sbt.
Global / cancelable
ThisBuild / scalaVersion
Test / test
root / Compile / compile / scalacOptions
ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/scalacOptions
Zero / Zero / name
The inspect command now outputs something that can be copy-pasted:
> inspect compile
[info] Task: sbt.inc.Analysis
[info] Description:
[info] Compiles sources.
[info] Provided by:
[info] ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/compile
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:326
[info] Dependencies:
[info] Compile/manipulateBytecode
[info] Compile/incCompileSetup
[info] Reverse dependencies:
[info] Compile/printWarnings
[info] Compile/products
[info] Compile/discoveredSbtPlugins
[info] Compile/discoveredMainClasses
[info] Delegates:
[info] Compile/compile
[info] compile
[info] ThisBuild/Compile/compile
[info] ThisBuild/compile
[info] Zero/Compile/compile
[info] Global/compile
[info] Related:
[info] Test/compile
This implements JSON-based port file. Thoughout the lifetime of the sbt server there will be `cwd / "project" / "target" / "active.json"`, which contains `url` field.
Using this `url` the potential client, such as IDEs can find out which port number to hit.
Ref #3508
Adds JVM flag `sbt.server.autostart` to enable/disable the automatic starting of sbt server with the sbt shell.
This also adds a new command `startServer` to manually start the server.
If the read buffer contains more that 2 messages, we need to consume them all before blocking on socket read again. For that we have to loop until the buffer does not contain anymore the message delimiter character.
Same problem in the client ServerConnection code.
Fixes#2776
This allows cross building commands. When issuing a command, it detects
whether there is likely to be any Scala version incompatibilities, by
checking whether all projects have the same Scala cross version
configuration, if not, it outputs a big fat warning.
This no longer injects scalaVersion at the project level, which was interfering with crossScalaVersions delegation to ThisBuild scope.
Fixessbt/sbt#3353
Both the default settings and ^^ together sets the correct scalaVersion based on `sbtVersion in pluginCrossBuild`, but frequently people set up `scalaVersion` on sbt plugin's subproject, which disables the feature.
This change appends the scalaVersionSetting on ^^ so scalaVersion gets switched to 2.12.2 on ^^ 1.0.0-RC2 etc.
Fixes#3205
Ref #3282
We used to wrap InputStream so it will inject Thread.sleep, which then allows the thread to be cancelled, emulating a non-blocking readLine. This trick doesn't seem to work for Windows.
For non-Cygwin, actually just removing the wrapping does the job, but I couldn't get it to work for Cygwin.
To test, run some command via network, and then type `show name` into the terminal. On Cygwin, it will not respond.
This commit adapts `Watched` so that it supports the new `WatchService`
infrastructure introduced in sbt/io. The goal of this infrastructure is
to provide and API for and several implementations of services that
monitor changes to the file system.
The service to use to monitor the file system can be configured with the
key `watchService`.
This undeprecates the syntax, but at the same times moves it out of
implicit scope, therefore requiring a 'import TupleSyntax._' to opt-in
to the old syntax.
Before, we were not preserving the value `insideXXX`. This commit makes
sure that we handle much more complex scenarios and we report them
successfully. Have a look at the tests.
This ports sbt-cross-building's cross (`^`) and switch (`^^`) commands.
Instead of making it a plugin, the default settings are now changed
to use `sbtVersion in pluginCrossBuild` for the sbt dependency.
In sbt 0.13.15, in addition to notifying the user about the existence of
sbt's shell, a feature was added to allow the user to switch to sbt's
shell - a more pro-active approach to just displaying a message.
Unfortunately sbt is often unintentionally invoked in shell scripts in
"interactive mode" when no interaction is expected by, for exmaple,
invoking `sbt package` instead of `sbt package < /dev/null`. In that
case hitting [ENTER] would silently trigger sbt to run its shell,
easily wrecking the script. In addition to that I was unhappy with the
implementation as it created a tight coupling between sbt's command
processing abstraction to sbt's shell command.
If you want to stay in sbt's shell after running a task like `package`
then invoke sbt like so:
sbt package shell
Fixes#3091
This is a change in strategy.
The motivation is the need to find a good balance between:
+ informing the uninformed that would benefit from this information, &
+ not spamming the already informed
Making it dependent on "compile" being present in remainingCommands will
probably make it trigger for, for example, Maven users who are used to
running "mvn compile" and always run "sbt compile", and who therefore
are unneccesarily suffering terribly slow compile speeds by starting up
the jvm and sbt every time.
Fixes#3091Fixes#3097
This commit does the following things:
* Removes the boolean from the instance context passes to the linter.
* Prohibits the use of value inside anonymous functions.
* Improves the previous check of `value` inside if.
The improvements have occurred thanks to the fix of an oversight in the
traverser. As a result, several implementation of tasks have been
rewritten because of new compilation failures by both checks.
Note that the new check that prohibits the use of value inside anonymous
functions ignores all the functions whose parameters have been
synthesized by scalac (that can happen in a number of different
scenarios, like for comprehensions). Other scripted tests have also been
fixed.
Running `.value` inside an anonymous function yields the following
error:
```
[error] /data/rw/code/scala/sbt/main-settings/src/test/scala/sbt/std/TaskPosSpec.scala:50:24: The evaluation of `foo` inside an anonymous function is prohibited.
[error]
[error] Problem: Task invocations inside anonymous functions are evaluated independently of whether the anonymous function is invoked or not.
[error]
[error] Solution:
[error] 1. Make `foo` evaluation explicit outside of the function body if you don't care about its evaluation.
[error] 2. Use a dynamic task to evaluate `foo` and pass that value as a parameter to an anonymous function.
[error]
[error] val anon = () => foo.value + " "
[error] ^
```
`.value` inside the if of a regular task is unsafe. The wrapping task
will always execute the value, no matter what the if predicate yields.
This commit adds the infrastructure to lint code for every sbt DSL
macro. It also adds example of neg tests that check that the DSL checks
are in place.
The sbt checks yield error for this specific case because we may want to
explore changing this behaviour in the future. The solutions to this are
straightforward and explained in the error message, that looks like
this:
```
EXPECTED: The evaluation of `fooNeg` happens always inside a regular task.
PROBLEM: `fooNeg` is inside the if expression of a regular task.
Regular tasks always evaluate task inside the bodies of if expressions.
SOLUTION:
1. If you only want to evaluate it when the if predicate is true, use a dynamic task.
2. Otherwise, make the static evaluation explicit by evaluating `fooNeg` outside the if expression.
```
Aside from those solutions, this commit also adds a way to disable any
DSL check by using the new `sbt.unchecked` annotation. This annotation,
similar to `scala.annotation.unchecked` disables compiler output. In our
case, it will disable any task dsl check, making it silent.
Examples of positive checks have also been added.
There have been only two places in `Defaults.scala` where this check has
made compilation fail.
The first one is inside `allDependencies`. To ensure that we still have
static dependencies for `allDependencies`, I have hoisted up the value
invocation outside the if expression. We may want to explore adding a
dynamic task in the future, though. We are doing unnecessary work there.
The second one is inside `update` and is not important because it's not
exposed to the user. We use a `taskDyn`.
This change is necessary in the cases where we have global
initialization issues that have no position, like:
```
[info] [error] scala.reflect.internal.MissingRequirementError: object scala in compiler mirror not found.
```
Before, it was failing with a `sys.error` exception. Now we will report
these issues with a console reporter that is not meant to be
thread-safe.
Fixes#3178
While working on the Scopes and Scope Delegation document, I noticed that the term Global in sbt is used for two different meaning.
1. Universal fallback scope component `*`
2. An alias for GlobalScope
This disambiguates the two by renaming ScopeAxis instance to Zero.
Since this is mostly internal to sbt code, the impact to the user should be minimal.
The `cachedUpdate` implementation does not need to be in `Defaults`
since it's not using any of the tasks/settings defined there, that's
`updateTask`'s job.
This commit moves the utilities required by `updateTask` to the
`sbt.internal.librarymanagement` namespace.
This commit does some changes to the implementation with the purpose of
making this code more readable. I find that this rewrite was necessary
as I was implementing the dependency lock file.
This commit has two goals:
* Simplify the `load` API endpoints, removing the unused ones to shorten
the surface of the API.
* Add documentation to the main `load` methods.
Sbt has a feature to show timed logs for every operation at startup.
However, its output is cluttered and users cannot read how much time
single methods consume nor if they call other methods.
This commit improves the status quo by adding indentation.
This commit reduces the complexity around `loadPluginDefinition` et al.
`pluginDefinitionLoader` is not used anywhere in sbt, so the extra
definitions are removed.
Both the implementation of `loadPluginDefinition` and
`pluginDefinitionLoader` are reduced to a bare minimum where the
components at hand (definition classpath, dependency classpath) are
properly defined.
Documentation to the three methods has been added.
It mainly does three things:
* Clean up the implementation, removing unused values like
`globalPluginDefs`.
* Make the implementation more readable.
* And the big one: Remove the creation of a classloader that we were
instantiating but whose value we were throwing away. This has an impact
in performance, but I'm yet to benchmark how much it is.
Previous commit used `synchronized` to ensure that the global reporter
was not reporting errors from other parsing sessions. Theoretically,
though, sbt could invoke parsing in parallel, so it's better to ensure
we remove the `synchronized` block, which could also be preventing some
JVM optimizations.
The following commit solves the issue by introducing a reporter id.
A reporter id is a unique identifier that is mapped to a reporter. Every
parsing session gets its own identifier, which then is reused for
recursive parsing. Error reports between recursive parses cannot collide
because the reporter is cleaned in `parse`.
The previous implementation was instantiating a toolbox to parse every
time it parsed a sbt file (and even recursively!!!).
This is inefficient and translates to instantiating a `ReflectGlobal`
every time we want to parse something.
This commit takes another approach:
1. It removes the dependency on `ReflectGlobal`.
2. It reuses the same `Global` and `Run` instances for parsing.
This is an efficient as it can get without doing a whole overhaul of it.
I think that in the future we may want to reimplement it to avoid the
recursive parsing to work around Scalac's bug.
This change was proposed by Jason in case that the new parsing mechanism
implemented later on has to be reverted. This change provides a good
baseline, but it's far from ideal with regard to readability of the
parser and performance.
The previous implementation was using the Scala runtime universe to
check whether a plugin had or not an `autoImport` member. This is a bad
idea for the following reasons:
* The first time you use it, you class load the whole Scalac compiler
universe. Not efficient. Measurements say this is about a second.
* There is a small overhead of going through the reflection API.
There exists a better approach that consists in checking if `autoImport`
exists with pure Java reflection. Since the class is already class
loaded, we check for:
* A class file named after the plugin FQN that includes `autoImport$` at
the end, which means that an object named `autoImport` exists.
* A field in the plugin class that is named `autoImport`.
This complies with the plugin sbt specification:
http://www.scala-sbt.org/1.0/docs/Plugins.html#Controlling+the+import+with+autoImport
We need to communicate the error states in the thread, so I added a `Future[Unit]` called `ready`.
If something goes wrong during the startup, like if the port is already taken, this can be used to communicate back to the main thread, and display the error accordingly.
+ Don't notify ScriptMain users by moving the logic to xMain
+ Only trigger shell if shell is a defined command
+ Use existing Shell/BootCommand strings instead of new ones
The sbt/sbt-launcher-package doesn't invoke sbt with the "shell"
command. sbt has a mechanism for handling this in its "boot" command
that adds an "iflast shell" to the commands. Handle this when displaying
the "Executing in batch mode" warning.
Fixes#3004
Notify & enable users to stay in sbt's shell on the warm JVM by hitting
[ENTER] while sbt is running.
Looks like this; first I run 'sbt about', then I hit [ENTER]:
$ sbt about
[info] !!! Executing in batch mode !!! For better performance, hit [ENTER] to remain in the sbt shell
[info] Loading global plugins from /Users/dnw/.dotfiles/.sbt/0.13/plugins
[info] Loading project definition from /s/t/project
[info] Set current project to t (in build file:/s/t/)
[info] This is sbt 0.13.14-SNAPSHOT
[info] The current project is {file:/s/t/}t 0.1.0-SNAPSHOT
[info] The current project is built against Scala 2.12.1
[info] Available Plugins: sbt.plugins.IvyPlugin, sbt.plugins.JvmPlugin, sbt.plugins.CorePlugin, sbt.plugins.JUnitXmlReportPlugin, sbt.plugins.Giter8TemplatePlugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.6
>
>
Fixes#2987
Have sbt.version set in project/build.properties is a best practice
because it makes the build more deterministic and reproducible.
With this change sbt, after ensuring that the base directory is probably
an sbt project, writes out sbt.version in project/build.properties if it
is missing.
Fixes#754
Before this commit, using dotty in your sbt project required to add:
scalaCompilerBridgeSource := ("ch.epfl.lamp" % "dotty-sbt-bridge" %
scalaVersion.value % "component").sources()
in your build.sbt. We might as well automatically do this, this reduces
the boilerplate for using dotty in your project to:
scalaOrganization := "ch.epfl.lamp"
scalaVersion := "0.1.1-SNAPSHOT"
scalaBinaryVersion := "2.11" // dotty itself is only published as a
// 2.11 artefact currently
java.lang.Class#newInstance deprecated since Java 9
http://download.java.net/java/jdk9/docs/api/java/lang/Class.html#newInstance--
```
Deprecated. This method propagates any exception thrown by the nullary constructor, including a checked exception. Use of this method effectively bypasses the compile-time exception checking that would otherwise be performed by the compiler. The Constructor.newInstance method avoids this problem by wrapping any exception thrown by the constructor in a (checked) InvocationTargetException.
The call
clazz.newInstance()
can be replaced by
clazz.getDeclaredConstructor().newInstance()
The latter sequence of calls is inferred to be able to throw the additional exception types InvocationTargetException and NoSuchMethodException. Both of these exception types are subclasses of ReflectiveOperationException.
Creates a new instance of the class represented by this Class object. The class is instantiated as if by a new expression with an empty argument list. The class is initialized if it has not already been initialized.
```
This adds a macro-level hack to support += op for sourceGenerators and resourceGenerators using RHS of Initialize[Task[Seq[File]]].
When the types match up, the macro now calls `.taskValue` automatically.
This setting controls the maximum width of the ASCII graphs printed
by commands like `inspect tree`. Default value corresponds to the
previously hardcoded value of 40 characters.
Copies products to the workind directory, and the rest to the serviceTempDir of this service, both wrapped in SHA-1 hash of the file contents. This is intended to mimize the file copying and accumulation of the unused JAR file. Since working directory is wiped out when the background job ends, the product JAR is deleted too. Meanwhile, the rest of the dependencies are cached for the duration of this service.
Fixes#2460Fixes#2851
Ref #2707, #2708, #2469
Unlike the previous attempts at fixing the handling of build-level
keys, this change does not change the main parsing logic, which uses
`getKey` to retrieve the key from the key map.
The fact that shell worked pre-0.13.11 means that the parsing was ok.
What this changes is just the "example" keys supplied to the parser so
the tab completion works.
Fixes#2761
With sbt 0.13.13-RC1 rediscovered that the dependency pulled in from
Giter8 was affecting the plugins. To avoid this, this change splits up
the template resolver implementation to another module called
sbt-giter8-resolver, and it will be downloaded using Ivy into
`~/.sbt/0.13/templates/`, and then launched reflectively using Java as
the interface.
This adds `new` command, which helps create a new build definition. The
`new` command is extensible via a mechanism called the template
resolver,
which evaluates the arbitrary arguments passed to the command to find
and run a template.
As a reference implementation [Giter8][g8] is provided as follows:
sbt new eed3si9n/hello.g8
This will run eed3si9n/hello.g8 using Giter8.
[g8]: http://www.foundweekends.org/giter8/
The compiler interface subclasses `scala.tools.nsc.Global`,
and loading this new subclass before each `compile` task forces
HotSpot JIT to deoptimize larges swathes of compiled code. It's
a bit like SBT has rigged the dice to always descend the longest
ladder in a game of Snakes and Ladders.
The slowdown seems to be larger with Scala 2.12. There are a number
of variables at play, but I think the main factor here is that
we now rely on JIT to devirtualize calls to final methods in traits
whereas we used to emit static calls. JIT does a good job at this,
so long as classloading doesn't undo that good work.
This commit extends the existing `ClassLoaderCache` to encompass
the classloader that includes the compiler interface JAR. I've
resorted to adding a var to `AnalyzingCompiler` to inject the
dependency to get the cache to the spot I need it without binary
incompatible changes to the intervening method signatures.
Ref #2634
updateSbtClassifiers uses an artificially created dependency graph set
in classifiersModule. The problem is that ivyScala instance is reused
from the outer scope that has the user project's scalaVersion as
demonstrated as follows:
scala> val is = (ivyScala in updateSbtClassifiers).eval
is: Option[sbt.IvyScala] =
Some(IvyScala(2.9.3,2.9.3,List(),true,false,true,org.scala-lang))
This change fixes#2686 by redefining ivyScala with scalaVersion and
scalaBinaryVersion scoped to updateSbtClassifiers task. The existing
scripted test was modified to reproduce the bug.
Def.make could take 10099ms for 100 subprojects. This would display
logs probably for projects with more than 10 subprojects, which might
pause for a few seconds during load.
LogManager implementation is modified to use ManagedLogger, which can swap out backing Appenders without re-creating the log instance.
The State was also changed to track `currentCommand: Option[Exec]`. `Exec` knows the origin of the command invocation, and using that we can now send the network-originated events only to the network clients.
Combined together, this implements log splitting between the sbt clients (channels).
This is the beginning of a lightweight client, which talks to the
server over Contraband-generated JSON API. Given that the server is
started on port 5173:
```
$ cd /tmp/bogus
$ sbt client localhost:5173
> compile
StatusEvent(Processing, Vector(compile, server))
StatusEvent(Ready, Vector())
StatusEvent(Processing, Vector(, server))
StatusEvent(Ready, Vector())
```
This adds support to generate synthetic subprojects from an auto plugin.
In addition, a method called `projectOrigin` is added to distinguish
Organic, BuildExtra, ProjectExtra, and GenericRoot.
Forward-port of #2717 and #2738
eugene yokota
Ethan Atkins
Thomas Droxler
Dale Wijnand
Ólafur Páll Geirsson
Arnout Engelen
Alexandre Archambault
Andrea Peruffo
Yusuke Izawa
kai-chi
falmarri
Guillaume Martres
Justin Kaeser
Kazuhiro Sera
Julien Sirocchi
Philippus
Eugene Platonov
xuwei-k
Jason Zaugg
Jason Pickens
Guillaume Poirier
Filipe Regadas
Anthony Garo
veera venky
Mark Canlas
Martynas Mickevičius
Antonio Cunei
alodavi
tiqwab
Øyvind Høisæther
Alexander Samsig
Liu Fengyun
Seth Tisue
Dale Wijnand
Alexey Alekhin
Nafer Sanabria
Alistair Johnson
Jason Steenstra-Pickens
tmiyamon
TATSUNO Yasuhiro
Rui Gonçalves
Maksym Fedorov
Deokhwan Kim
Steve Waldman
Antonio Cunei
Michael Pollmeier
wpopielarski
OlegYch
Roman Iakovlev
PanAeon
Michael Stringer
Age Mooij
Răzvan Flavius Panda
Guillaume Bort
Martin Duhem
James Roper
jvican
valydia
Eric K Richardson
Devis Lucato
Miles Sabin
David Perez
Josh Soref
Paul Draper
Anatoly Fayngelerin
Johan Andrén