I had previously not though there was much reason to support commands in
continuous builds. This was primarily because there were a number of
questions regarding semantics. Commands cannot have fileInputs
specifically assigned to them because they don't have an associated
scope. They also can arbitrarily modify state so what is the expectation
when running ~foo where foo is a command that, for example, replaces
itself. I settled on the following semantics:
1) Commands run in a continuous build cannot modify the sbt execution
state which is to say that the state that is returned by continuous
is the same that was passed in (unless a reload occurred or we exited
the build with an exception)
2) Any global watchTriggers or fileInputs apply to a watched command.
They automatically inherit any fileInputs that are queried when
running tasks in a command. So, for example, ~+compile does what
you'd expect.
The implementation is fairly straightforward. If we can successfully
parse a command, but we cannot parse a scopedKey from it, we assign it a
private ScopedKey. When computing the watch settings for that key, we
will select the global settings through delegation. This is how it picks
up the global watchTriggers.
To run the command, I had to rework the task evaluation portion because
a command may return a state with additional commands to run. The cross
build command works this way. We recursively run all of the commands
starting with the original until we run out of commands to run. As part
of this work, I was able to remove the three argument version of
Command.processCommand that I'd previously added to support my old
approach to evaluating commands. This was a nice bonus.
I added scripted tests that check that global watchTriggers are picked
up and that commands that delegate to a command that uses fileInputs
automatically pick up those inputs during the watch. I also added a test
that approximates the ~+compile use case and ensures that the failure
semantics are what we expect and that the task runs for all defined
scala versions.
@olegych reported in https://github.com/sbt/sbt/issues/4722 that
sometimes even when a build was triggered during watch that no
recompilation would occur. The cause of this was that we never
invalidated the file stamp cache for managed sources or output files.
The optimization of persisting the source file stamps between task
evaluations in a continuous build only really makes sense for unmanaged
sources. We make the implicit assumption that unmanaged sources are
infrequently updated and generally one at a time. That assumption does
not hold for unmanaged source or output files.
To fix this, I split the fileStampCache into two caches: one for
unmanaged sources and one for everything else. We only persist the
unmanagedFileStampCache during continuous builds. The
managedFileStampCache gets invalidated every time.
I added a scripted test that simulates changing a generated source file.
Prior to this change, the test would fail because the file stamp was not
invalidated for the new source file content.
Fixes#4722
Given that there are io differences between windows and posix systems,
we should aim to run the tests that do a lot of io on windows. There are
a few tests that don't work because of some platform specific issues so
I added a filter that excludes these tests on windows in ScriptedTests.
* 1.2.x: (28 commits)
More bumping up the 2.12 version to 2.12.8 in 1.2.x
Bump the 2.12 version to 2.12.8 in 1.2.x
define whitesourceOnPush
lm 1.2.4
1.2.7-SNAPSHOT
implement TestConsoleLogger
bump util, lm, and zinc
Bump scalatest to 3.0.6-SNAP5
Bump log4j2 to 2.11.1
drop notification override
Ignore files in scripted group dirs
Fix '~' for dependent projects with a broken parent
util 1.2.3, zinc 1.2.4
lm 1.2.2
Adjust the tests
Set withMetadataDirectory by default
Fix single repo emulation script
add onLoadMessage
check PluginCross.scala consisntency
Bump modules
...
It was reported in https://github.com/sbt/sbt/issues/4608 that there was
a regression that tests run against scala 2.11 would fail. This was
because the interface loader incorrectly contained the scala library. To
fix this, I needed to find the xsbt.boot.BootFilteredLoader in the
classloading hierarchy and put the sbt testing interface library in
between that loader and the scala library loader.
It is possible with the new layering strategies that tests may fail if a
java package private class is accessed across classloader layers. This
will result in an IllegalAccessError that is hard to debug. With this
commit, I add an error message that will be displayed if run throws an
IllegalAccessError that suggests that the user try the
ScalaInstance layering strategy or the flat layering strategy.
I'm not sure if this is a huge benefit or not, but it's nice to have the
option to run the scripted tests in parallel. The default behavior
should be the same as before.
Previously, the ClassLoaderLayeringStrategy was set globally. This
didn't really make sense because the Runtime and Test configs had
different strategies available (Test being a superset of Runtime).
Instead, we now set the layering strategy in the Runtime and Test
configurations directly. In doing this, we can eliminate the Default
ClassLoaderLayeringStrategy. Previously this had existed so that we
could set the layering strategy globally and have it do the right thing
in both test and runtime.
To implement this, I factored out the logic for generating the layered
classloader in the test task and shared it with the runtime task. I did
this because I realized that Test / run is a thing. Previously I had
been operating under the assumption that the runner would never include
the test dependencies. Once I realized this, it made sense to combine
the logic in both tasks.
As a bonus, I only allow the layering strategies that explicitly make
sense to be set in each configuration. If the user sets an invalid
strategy, an error will be thrown that specifies the valid strategies
for the task.
I also added ScalaInstance as an option for the runtime layer. It was an
oversight that this was left out.
Normally I'd include these with the previous commit, but the diff is so
large that I put them in their own commit. The tests handle 5 scenarios:
1) akka-actor-system -- a project that has Akka as a dependency and a
simple main method that creates and terminates an ActorSystem. What
is interesting about this test is that if scriptedBufferLog := false,
we notice that the first call to run is slow, but subsequent calls to
run and test are fast. The test does at least ensure that recycling
the runtime layer in test works ok.
2) jni -- verifies that a project with native libraries will be able to
load the library with each run. It actually swaps out the underlying
library so that the it really ensures that the library is reloaded
between runs.
3) library-mismatch -- verifies that the layered classloaders can work
when the test dependencies are incompatible with the runtime
dependencies. In this test, the test dependencies use an api in a
library called foo-lib that isn't available in the version used by
the runtime dependencies. Because of this incompatibility, the test
will not work if Test / layeringStrategy := LayeringStrategy.Full.
4) scalatest -- verifies that a test runs using the scalatest framework
5) utest -- verifies that a test runs using the utest framework
The reason for (4) and (5) is to ensure that both the in sourced test
frameworks and external frameworks work with the new loaders.
Fixes#4241Fixes#4242
This introduces a new subproject named scripted-sbt-redux. The purpose of this new subproject is to workaround the 'sbt.test` package vs `Keys.test` key confusion (#4242) while maintaining the forward compatibility of 0.13.17's sbt cross testing ^^ (#4241).
The new subproject uses `sbt.scriptedtest` package name, and that's the one that will be used by the mothership.
Meanwhile "scripted-sbt" subproject will also be published for compatibility purpose.
Ethan Atkins
Eugene Yokota
Dale Wijnand