Commit Graph

312 Commits

Author SHA1 Message Date
Eugene Yokota 0ee7d11afe Migrate updateTask via tuple syntax 2023-01-15 23:45:35 -05:00
Eugene Yokota 0c6eb093ac Rename tuple syntax to use mapN 2023-01-15 23:45:35 -05:00
Eugene Yokota fe0de6ca30 Fix input task macro 2023-01-15 23:45:35 -05:00
Eugene Yokota d802c2ba04 valStart to change synthetic variable names 2023-01-15 23:45:35 -05:00
Eugene Yokota 73c7236c2c Refactor to use Type instead of TypeRepr 2023-01-15 23:45:35 -05:00
Eugene Yokota 61d4fe2d30 Implement input task helper Def.input 2023-01-15 23:45:35 -05:00
Eugene Yokota cec131f6ed Fix conditional task 2023-01-15 23:45:34 -05:00
Eugene Yokota 81a05430bd Split buildfile parsing to a subproject 2023-01-15 23:45:30 -05:00
Eugene Yokota 66fa46a912 Fix InputTask macro 2023-01-15 23:44:28 -05:00
Eugene Yokota c7fa4937ef Bump to Scala 3.1.3 2023-01-15 23:44:25 -05:00
Eugene Yokota f3dd2dff36 Starting input task 2023-01-15 23:43:59 -05:00
Eugene Yokota b715917d95 Inline append macros 2023-01-15 23:43:59 -05:00
Eugene Yokota a1e50f8a2e Append macros 2023-01-15 23:43:59 -05:00
Eugene Yokota 9c1e8abb91 transform can be an inline 2023-01-15 23:43:59 -05:00
Eugene Yokota 37f6ee6184 Setting macro 2023-01-15 23:43:59 -05:00
Eugene Yokota 6662dbad26 InputWrapper can be inline method 2023-01-15 23:43:59 -05:00
Eugene Yokota 340721c4a1 KeyMacro 2023-01-15 23:43:57 -05:00
David Gregory dd497063e5
Add Remove instances for Set and Map 2022-03-30 19:03:14 +01:00
eugene yokota b3aae681bf
Merge pull request #6711 from xuwei-k/fix-scala-2-13-warn
fix Scala 2.13 warnings
2021-11-14 21:53:57 -05:00
xuwei-k 535b15b83e fix Scala 2.13 warnings 2021-11-14 22:59:34 +09:00
xuwei-k aa8b1141f8 Update scalatest 2021-11-14 22:03:59 +09:00
Eugene Yokota 695261a068 Migrate most of the internal usage of in 2021-02-22 00:09:36 -05:00
Eugene Yokota 833024b1d9 `in` is no longer in
Ref https://github.com/sbt/sbt/pull/6309

This deprecates the sbt 0.13 style `key in (Compile, intask)` syntax.
2021-02-21 23:38:56 -05:00
Eugene Yokota 8f5759e48d Cross build to Scala 2.13 2021-01-10 20:24:05 -05:00
Eugene Yokota a6d03c819c Backtick the unicode letters in method 2021-01-10 14:21:50 -05:00
Ethan Atkins 78620cd902 Manage ansi codes and color codes separately
The ConsoleAppender formatEnabledInEnv field was being used both as an
indicator that ansi codes were supported and that color codes are
enabled. There are cases in which general ansi codes are not supported
but color codes are and these use cases need to be handled separately.
To make things more explicit, this commit adds isColorEnabled and
isAnsiSupported to the Terminal companion object so that we can be more
specific about what the requirements are (general ansi escape codes or
just colors). There are a few cases in ConsoleAppender itself where
formatEnabledInEnv was used to set flags for both color and ansi codes.
When that is the case, we use Terminal.isAnsiSupported because when that
is true, colors should at least work but there are terminals that
support color but not general ansi escape codes.
2020-10-21 08:01:12 -07:00
Eugene Yokota c85e5b8bc0 Upgrade scala version to 2.12.12
This introduces `Def.unit(...)` to workaround the pure expression does nothing warning.
2020-08-11 23:33:12 -04:00
Eugene Yokota 002f97cae7 Build pipelining
Ref https://github.com/sbt/zinc/pull/744

This implements `ThisBuild / usePipelining`, which configures subproject pipelining available from Zinc 1.4.0.

The basic idea is to start subproject compilation as soon as pickle JARs (early output) becomes available. This is in part enabled by Scala compiler's new flags `-Ypickle-java` and `-Ypickle-write`.

The other part of magic is the use of `Def.promise`:

```
earlyOutputPing := Def.promise[Boolean],
```

This notifies `compileEarly` task, which to the rest of the tasks would look like a normal task but in fact it is promise-blocked. In other words, without calling full `compile` task together, `compileEarly` will never return, forever waiting for the `earlyOutputPing`.
2020-08-06 02:31:01 -04:00
Ethan Atkins ba345dd797 Add multi-client ui to server
This commit makes it possible for the sbt server to render the same ui
to multiple clients. The network client ui should look nearly identical
to the console ui except for the log messages about the experimental
client.

The way that it works is that it associates a ui thread with each
terminal. Whenever a command starts or completes, callbacks are invoked
on the various channels to update their ui state. For example, if there
are two clients and one of them runs compile, then the prompt is changed
from AskUser to Running for the terminal that initiated the command
while the other client remains in the AskUser state. Whenever the client
changes uses ui states, the existing thread is terminated if it is
running and a new thread is begun.

The UITask formalizes this process. It is based on the AskUser class
from older versions of sbt. In fact, there is an AskUserTask which is
very similar. It uses jline to read input from the terminal (which could
be a network terminal). When it gets a line, it submits it to the
CommandExchange and exits. Once the next command is run (which may or
may not be the command it submitted), the ui state will be reset.

The debug, info, warn and error commands should work with the multi
client ui. When run, they set the log level globally, not just for the
client that set the level.
2020-06-24 19:40:17 -07:00
Eugene Yokota 9821b83048 Forward type to the conditional tasks to fix Seq[Nothing]
Fixes https://github.com/sbt/sbt/issues/5625
2020-06-17 18:08:59 -04:00
eugene yokota 2b557851ca
Merge pull request #5041 from jsoref/https
Https
2020-06-14 17:34:58 -04:00
eugene yokota 975e3db43e
Merge pull request #5558 from eed3si9n/wip/selective
Selective functor
2020-06-10 17:39:13 -04:00
Eugene Yokota b78e4f0919 implement Def.promise
This adds `Def.promise` a facility that wraps `scala.concurrent.Promise`. Project layer, there's an implicit for task-that-returns-promise (`Def.Initialize[Task[PromiseWrap[A]]]`) that would inject `await` method, which returns a task. This is a special task that is tagged with `Tags.Sentinel` so that it will bypass the concurrent restrictions. Since there's no CPU- or IO-bound work, this should be ok.

The purpose of this promise for long-running task to communicate with another task midway.
2020-06-10 15:16:25 -04:00
Eugene Yokota b43f663b0e Automatic conversion to Def.taskIf
When `Def.task`, `:=`, `+=` etc contains `if` and only `if` expression automatically treat it as a conditional task even if the else clause contains `.value`.
2020-05-19 15:02:57 -04:00
Eugene Yokota 45e8426d27 def taskIf[A](a: A): Initialize[Task[A]]
Make `Def.taskIf` accept pure `A` type, as opposed to `Def.task { ... }`.
2020-05-17 23:44:35 -04:00
Eugene Yokota c6f62293f1 Def.taskIf macro
Def.taskIf accepts an if-expression or a block ending in an if-expression.
2020-05-17 23:36:04 -04:00
Eugene Yokota 2feecf8a1f Selective functor
This implements Selective functor for `Either[A, B]` "task" (`Initialize[Task[Either[A, B]]]`).
The selective functor allows an encoding of if-expression:

```
  def ifS[A](
      x: Def.Initialize[Task[Boolean]]
  )(t: Def.Initialize[Task[A]])(e: Def.Initialize[Task[A]]): Def.Initialize[Task[A]]
```

The benefit of this approach is that task dependencies are still visible to inspect command.
2020-05-17 23:36:04 -04:00
eugene yokota 0b12862caf
Merge pull request #5439 from dwijnand/introduce-Taskable
Introduce Taskable & toTaskable
2020-02-28 10:43:16 -05:00
eugene yokota 9c384dca15
Merge pull request #5435 from dwijnand/tweak-Def.displayRelative
Tweak Def.displayRelative
2020-02-28 09:57:04 -05:00
Dale Wijnand 23702b6432 Add toTaskable to Init/+Task, for tuple syntax 2020-02-17 12:22:53 +00:00
Dale Wijnand 722022ae97 Add conversions from Init/+Task to Taskable 2020-02-17 12:22:53 +00:00
Dale Wijnand fdfdd1ca47 Extract Taskable from ScopedTaskable 2020-02-17 12:22:53 +00:00
Dale Wijnand 2608f8ade6 Document parts of AList & components of TupleSyntax
Also, extract the AList.SplitK type lambda helper for call-site reuse.
2020-02-17 07:15:37 +00:00
Dale Wijnand c5b684984c Tweak Def.displayRelative
My driving use case was LocalProject: it should display the same as a
fully resolved ProjectRef.  Similarly ThisBuild should display as such.
2020-02-15 17:17:03 +00:00
Dale Wijnand 4635af182a Add some tests around Def.displayRelative2 2020-02-15 17:17:03 +00:00
Ethan Atkins ae4d3aecb8 Explicitly set scripted and server test classpath
This commit makes it so that the scalaVersion, sbtVersion and classpath
are always passed in as parameters to any method that creates an sbt
server -- either for scripted or for the sbt server tests. By making
that change, I was able to change the implementation of scripted in the
sbt project to use publishLocalBin instead of publishLocal. This makes
the scripted tests start much faster (doc alone can easily take 30
second) with messing with the build to exclude slow tasks from
publishLocal.

As part of this change, I removed the test dependency on scriptedSbtRedux for
sbtProj and instead had scriptedSbtRedux depend on sbtProj. This allowed
me to remove some messy LocalProject logic in the resourceGenerators for
scriptedSbtReduxProj. I also had to remove a number of imports in the
scriptedSbtReduxProj because the definitions available in the sbt
package object became available.

I also removed the dependency on sbt-buildinfo and instead pass the
values from the build into test classes using scalatest properties. I
ran into a number of minor issues with the build info plugin, namely
that I couldn't get fullClasspathAsJars to reliably run as a BuildInfo
key. It also is somewhat more clear to me to just rely on the built in
scalatest functionality. The big drawback is that the scalatest
properties can only be strings, but that restriction isn't really a
problem here (strangely the TestData structure has a field configMap
which is effectively Map[String, Any] but Any is actually always String
given how the TestData is created as part of framework initialization.

Since scripted no longer publishes, scriptedUnpublished is now
effectively an alias for scripted.

To get publishLocalBin working, I had to copy private code from
IvyXml.scala into PublishBinPlugin. Once we publish a new version of
sbt, we can remove the copied code and invoke IvyXml.makeIvyXmlBefore
directly.
2020-01-19 09:04:26 -08:00
Eugene Yokota 54b3405f42 apply -Yno-lub
To demonstrate [-Yno-lub](http://eed3si9n.com/stricter-scala-with-ynolub), this shows the code changes that removes lubing (Not all subprojects are done).

After I made the changes, I switched the Scala back to normal 2.12.10.
2019-10-13 23:46:23 -04:00
Josh Soref c9d5cbe808 https://www.scala-sbt.org 2019-09-05 14:11:13 -04:00
Ethan Atkins 09f4fcf94b Add test case and scaladoc 2019-08-19 11:49:09 -07:00
Ethan Atkins 8fbf1907ce Allow custom show for a scope
I am writing a plugin that uses mangled task keys that are very hard to
read. It is helpful to be able to override the show config for these
scopes so that they look reasonable in supershell and in error
reporting.
2019-08-19 11:23:43 -07:00
Ethan Atkins f126206231 Fix incremental task evaluation semantics
While writing documentation for the new file management/incremental
task evaluation features, I realized that incremental task evaluation
did not have the correct semantics. The problem was that calls to
`.previous` are not scoped within the current task. By this, I mean that
say there are tasks foo and bar and that the defintion of bar looks like

bar := {
    val current = foo.value
    foo.previous match {
        case Some(v) if v == current => // value hasn't changed
        case _ => process(current)
    }
}

The problem is that foo.previous is stored in
effectively (foo / streams).value.cacheDirectory / "previous". This
means that it is completely decoupled from foo. Now, suppose that the
user runs something like:
> set foo := 1
> bar // processes the value 1
> set foo := 2
> foo
> bar // does not process the new value 2 because foo was called, which updates the previous value

This is not an unrealistic scenario and is, in fact, common if the
incremental task evaluation is changed across multiple processing steps.
For example, in the make-clone scripted test, the linkLib task processes
the outputs of the compileLib task. If compileLib is invoked separately
from linkLib, then when we next call linkLib, it might not do anything
even if there was recompilation of objects because the objects hadn't
changed since the last time we called compileLib.

To fix this, I generalizedthe previous cache so that it can be keyed on
two tasks, one is the task whose value is being stored (foo in the
example above) and the other is the task in which the stored task value
is retrieved (bar in the example above). When the two tasks are the
same, the behavior is the same as before.

Currently the previous value for foo might be stored somewhere like:

base_directory/target/streams/_global/_global/foo/previous

Now, if foo is stored with respect to bar, it might be stored in

base_directory/target/streams/_global/_global/bar/previous-dependencies/_global/_gloal/foo/previous

By storing the files this way, it is easy to remove all of the previous
values for the dependencies of a task.

In addition to changing how the files are stored on disk, we have to store
the references in memory differently. A given task can now have multiple
previous references (if, say, two tasks bar and baz both depend on the
previous value). When we complete the results, we first have to collect
all of the successful tasks. Then for each successful task, we find all
of its references. For each of the references, we only complete the
value if the scope in which the task value is used is successful.

In the actual implemenation in Previous.scala, there are a number places
where we have to cast to ScopedKey[Task[Any]]. This is due to
limitations of ScopedKey and Task being type invariant. These casts are
all safe because we never try to get the value of anything, we only use
the portion of the apis of these types that are independent of the value
type. Structural typing where ScopedKey[Task[_]] gets inferred to
ScopedKey[Task[x]] forSome x is a big part of why we have problems with
type inference.
2019-08-09 12:18:22 -07:00
Ethan Atkins a3ac4c76a6 Bump scalafmt
Intellij had issues resolving 2.0.0-RCX so it will be nice to be using
the latest.
2019-07-18 12:40:21 -07:00
Ethan Atkins cad89d17a9 Add support for in memory cache store
It can be quite slow to read and parse a large json file. Often, we are
reading and writing the same file over and over even though it isn't
actually changing. This is particularly noticeable with the
UpdateReport*. To speed this up, I introduce a global cache that can be
used to read values from a CacheStore. When using the cache, I've seen
the time for the update task drop from about 200ms to about 1ms. This
ends up being a 400ms time savings for test because update is called for
both Compile / compile and Test / compile.

The way that this works is that I add a new abstraction
CacheStoreFactoryFactory, which is the most enterprise java thing I've
ever written. We store a CacheStoreFactoryFactory in the sbt State.
When we make Streams for the task, we make the Stream's
cacheStoreFactory field using the CacheStoreFactoryFactory. The
generated CacheStoreFactory may or may not refer to a global cache.

The CacheStoreFactoryFactory may produce CacheStoreFactory instances
that delegate to a Caffeine cache with a max size parameter that is
specified in bytes by the fileCacheSize setting (which can also be set
with -Dsbt.file.cache.size). The size of the cache entry is estimated by
the size of the contents on disk. Since we are generally storing things
in the cache that are serialized as json, I figure that this should be a
reasonable estimate. I set the default max cache size to 128MB, which is
plenty of space for the previous cache entries for most projects. If the
size is set to 0, the CacheStoreFactoryFactory generates a regular
DirectoryStoreFactory.

To ensure that the cache accurately reflects the disk state of the
previous cache (or other cache's using a CacheStore), the Caffeine cache
stores the last modified time of the file whose contents it should
represent. If there is a discrepancy in the last modified times (which
would happen if, say, clean has been run), then the value is read from
disk even if the value hasn't changed.

* With the following build.sbt file, it takes roughly 200ms to read and
parse the update report on my compute:

libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.3"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.1"

This is because spark-sql has an enormous number of dependencies and the
update report ends up being 3MB.
2019-07-11 17:45:16 -07:00
Dale Wijnand e86a63ca1b
Drop the remaining TupleSyntax usage 2019-05-29 14:43:18 +01:00
Ethan Atkins 81ce14d58c Allow calling TaskKey.previous in input tasks
I discovered that it wasn't possible to call .previous in an input task.
While I understand why you can't call .previous on an InputKey, I think
it makes sense to allow calling .previous on a TaskKey within an input
task.
2019-04-25 15:58:01 -07:00
Eugene Yokota 1e157b991a apply formatting 2019-04-20 03:23:54 -04:00
Ethan Atkins 9cdeb7120e Add StateTransform class
This commit cleans up the approach for transforming the sbt state upon
completion of a task returning State. I add a new approach where a task
can return an instance of StateTransform, which is just a wrapper around
State. I then update EvaluateTask to apply this stateTransform rather
than the (optional) state transformation that may be stored in the Task
info parameter. By requiring that the user return StateTransform rather
than State directly, we ensure that no existing tasks that depend on the
state transformation function embedded in the Task info break. In sbt 2,
I could see the possibility of making this automatic (and probably
removing the state transformation function via attribute).

The problem with using the transformState attribute key is that it is
applied non-deterministically. This means that if you decorate a task
returning State, then the state transformation may or may not be
correctly applied.

I tracked this non-determinism down to the stateTransform
method in EvaluateTask. It iterates through the task result map and
chains all of the defined transformState attribute values. Because the
result is a map, this order is not specified. This chaining is arguably
a bad design because State => State does not imply commutivity. Indeed,
the problem here was that my state transformation functions were
constant functions, which are obviously non-commutative. I believe that
this logic likely written under the assumption that there would be no
more than one of these tranformations in a given result map.
2019-03-30 16:39:10 -07:00
Ethan Atkins e868c43fcc Refactor Watched
This is a huge refactor of Watched. I produced this through multiple
rewrite iterations and it was too difficult to separate all of the
changes into small individual commits so I, unfortunately, had to make a
massive commit. In general, I have tried to document the source code
extensively both to facilitate reading this commit and to help with
future maintenance.

These changes are quite complicated because they provided a built-in
like api to a feature that is implemented like a plugin. In particular,
we have to manually do a lot of parsing as well as roll our own
task/setting evaluation because we cannot infer the watch settings at
project build time because we do not know a priori what commands the
user may watch in a given session. The dynamic setting and task
evaluation is mostly confined to the WatchSettings class in Continuous.
It feels dirty to do all of this extraction by hand, but it does seem to
work correctly with scopes.

At a high level this commit does four things:
1) migrate the watch implementation to using the InputGraph to collect
   the globs that it needs to monitor during the watch
2) simplify WatchConfig to make it easier for plugin authors to write
   their own custom watch implementations
3) allow configuration of the watch settings based on the task(s) that
   is/are being run
4) adds an InputTask implemenation of watch.

Point #1 is mostly handled by Point #3 since I had to overhaul how _all_
of the watch settings are generated. InputGraph already handles both
transitive inputs and triggers as well as legacy watchSources so not
much additional logic is needed beyond passing the correct scoped keys
into InputGraph.

Point #3 require some structural changes. The watch settings cannot in
general be defined statically because we don't know a priori what tasks
the user will try and watch. To address this, I added code that will
extract the task keys for all of the commands that we are running. I
then manually extract the relevant settings for each command. Finally, I
aggregate those settings into a single WatchConfig that can be used to
actually implement the watch. The aggregation is generally
straightforward: we run all of the callbacks for each task and choose
the next watch state based on the highest priority Action that is
returned by any of the callbacks.

Because I needed Extracted to pull out the necessary settings, I was
forced to move a lot of logic out of Watched and into a new singleton,
Continuous, that exists in the main project (Watched is in the command
project). The public footprint of Continuous is tiny. Even though I want
to make the watch feature flexible for plugin authors, the
implementation and api remain a moving target so I do not want to be
limited by future binary compatibility requirements. Anyone who wants to
live dangerously can access the private[sbt] apis via reflection or by
adding custom code to the sbt package in their plugin (a technique I've
used in CloseWatch).

Point #2 is addressed by removing the count and lastStatus from the
WatchConfig callbacks. While these parameters can be useful, they are
not necessary to implement the semantics of a watch. Moreover, a status
boolean isn't really that useful and the sbt task engine makes it very
difficult to actually extract the previous result of the tasks that were
run. After this refactor, WatchConfig has a simpler api. There are fewer
callbacks to implement and the signatures are simpler. To preserve the
_functionality_ of making the count accessible to the user specifiable
callbacks, I still provided settings like watchOnInputEvent that accept
a count parameter, but the count is actually tracked externally to
Watched.watch and incremented every time the task is run.

Moreover, there are a few parameters of the watch: the logger and
transitive globs, that cannot be provided via settings. I provide
callback settings like watchOnStart that mirror the WatchConfig
callbacks except that they return a function from Continuous.Arguments
to the needed callback. The Continuous.aggregate function will check if
the watchOnStart setting is set and if it is, will pass in the needed
arguments. Otherwise it will use the default watchOnStart implementation
which simulates the existing behavior by tracking the iteration count in
an AtomicInteger and passing the current count into the user provided
callback. In this way, we are able to provide a number of apis to the
watch process while preserving the default behavior.

To implement #4, I had to change the label of the `watch` attribute key
from "watch" to "watched". This allows `watch compile` to work at the
sbt command line even thought it maps to the watchTasks key. The actual
implementation is almost trivial. The difference between an
InputTask[Unit] and a command is very small. The tricky part is that the
actual implementation requires applying mapTask to a delegate task that
overrides the Task's info.postTransform value (which is used to
transform the state after task evaluation). The actual postTransform
function can be shared by the continuous task and continuous command.
There is just a slightly different mechanism for getting to the state
transformation function.
2019-03-30 16:38:56 -07:00
Ethan Atkins ed06e18fab Add InputGraph
This commit adds functionality to traverse the settings graph to find
all of the Inputs settings values for the transitive dependencies of the
task. We can use this to build up the list of globs that we must watch
when we are in a continuous build. Because the Inputs key is a setting,
it is actually quite fast to fetch all the values once the compiled map
is generated (O(2ms) in the scripted tests, though I did find that it
took O(20ms) to generate the compiled map).

One complicating factor is that dynamic tasks do not track any of
their dynamic dependencies. To work around this, I added the
transitiveDependencies key. If one does something like:
foo := {
  val _ = bar / transitiveDependencies
  val _ = baz / transitiveDependencies
  if (System.getProperty("some.prop", "false") == "true") Def.task(bar.value)
  else Def.task(baz.value)
}
then (foo / transitiveDependencies).value will return all of the inputs
and triggers for bar and baz as well as for foo.

To implement transitiveDependencies, I did something fairly similar to
streams where if the setting is referenced, I add a default
implementation. If the default implementation is not present, I fall
back on trying to extract the key from the commandLine. This allows the
user to run `show bar / transitiveDependencies` from the command line
even if `bar / transitiveDependencies` is not defined in the project.

It might be possible to coax transitiveDependencies into a setting, but
then it would have to be eagerly evaluated at project definition time
which might increase start up time too much.  Alternatively, we could
just define this task for every task in the build, but I'm not sure how
expensive that would be. At any rate, it should be straightforward to
make that change without breaking binary compatibility if need be. This
is something to possibly explore before the 1.3 release if there is any
spare time (unlikely).
2019-03-30 16:38:44 -07:00
Eugene Yokota adfb2ece1a install official sbt based on project/build.properties 2019-03-25 02:10:48 -04:00
Ethan Atkins 571b179574 Add dsl for collecting globs
Right now, the sbt.internal.io.Source is something of a second class
citizen within sbt. Since sbt 0.13, there have been extension classes
defined that can convert a file to a PathFinder but no analog has been
introduced for sbt.internal.io.Source.

Given that sbt.internal.io.Source was not really intended to be part of
the public api (just look at its package), I think it makes sense to
just replace it with Glob. In this commit, I add extension
methods to Glob and Seq[Glob] that make it possible to easily
retrieve all of the files for a particular Glob within a task. The
upshot is that where previously, we'd have had to write something like:

watchSources += Source(baseDirectory.value / "src" / "main" / "proto", "*.proto", NothingFilter)

now we can write

watchGlobs += baseDirectory.value / "src" / "main" / "proto" * "*.proto"

Moreover, within a task, we can now do something like:
foo := {
  val allWatchGlobs: Seq[File] = watchGlobs.value.all
  println(allWatchSources.mkString("all watch source files:\n", "\n", ""))
}
Before we would have had to manually retrieve the files.

The implementation of the dsl uses the new GlobExtractor class which
proxies file look ups through a FileTree.Repository. This makes it so
that, by default, all file i/o using Sources will use the default
FileTree.Repository. The default is a macro that returns
`sbt.Keys.fileTreeRepository.value: @sbtUnchecked`. By doing it this
way, the default repository can only be used within a task definition
(since it delegates to `fileTreeRepository.value`). It does not,
however, prevent the user from explicitly providing a
FileTree.Repository instance which the user is free to instantiate
however they wish.

Bonus: optimize imports in Def.scala and Defaults.scala
2019-03-22 07:53:41 -07:00
Ethan Atkins 82cfbe83a7 Fix doc warning
The scaladoc task was warning:
'Could not find any member to link for "LinterLevelLowPriority"'. Given
that LinterLevelLowPriority is a package private trait, it shouldn't be
mentioned by name.
2019-01-31 17:04:40 -08:00
Ethan Atkins df6f3bd888 Switch to regular comment instead of scaladoc
Having this as scaladoc adds an unmoored doc comment compiler warning
because no documentation is generated for the class that is being
documented.
2019-01-31 17:04:40 -08:00
Dale Wijnand 32b342e9a8
Mention SAM conversions 2019-01-29 15:47:50 +00:00
Dale Wijnand 53c6299c94
Implement Append for Function1 2019-01-29 09:12:11 +00:00
Dale Wijnand 14bffefef9
Cleanup Append 2019-01-27 16:20:59 +00:00
Ethan Atkins 01b2e86a54 Fix Def cannot be used inside a taskDyn #3110
The illegalReference check did not actually validate whether the illegal
reference actually referred to an M[_] (which is pretty much always
Initialize[_]]). The means by which this failure was induces were fairly
obscure and go through multiple levels of macro transformations that I
attempt to explain in the comment in IllegalReferenceSpec.

Fixes #3110
2018-12-11 21:49:42 -08:00
Ethan Atkins d42b3ee2fd Make the TaskLinterDSL warn by default
I am generally of the opinion that a linter should not abort progress by
default. I do, however, think that it should be on by default, making
warn a happy compromise.
2018-12-11 18:21:46 -08:00
Ethan Atkins 32792f2b9d Make the task linter configurable
The user should be able to configure whether or not the task linting is
strictly enforced. In my opinion, the linter is generally pretty good
about warning users that a particular definition may not behave the way
the user expects. Unfortunately, it is fairly common that the user
explicitly wants this behavior and making the linter abort compilation
is a frustrating user experience.

To fix this, I add the LinterLevel trait to a new sbt.dsl package in the
core-macros project. The user can configure the linter to:
1) abort when an issue is detected. This is the current default behavior.
2) print a warning when an issues is detected
3) skip linting all together

Linter configuration is managed by importing the corresponding implicit
object from the LinterLevel companion object.

Fixes #3266
2018-12-11 18:21:44 -08:00
Ethan Atkins 2e027640d3 Add solutions for dynamic task evaluation
There are many cases where one would want to force evaluation of the
task even when contained in a lambda (see
https://github.com/sbt/sbt/issues/3266). The @sbtUnchecked annotation is
one way to disable the linter that prevents this, but it is obscure. If
the annotation is to exist, I think it should be presented as a
solution.
2018-12-11 12:45:49 -08:00
Ethan Atkins 1c2bab093b Switch to more functional style in BaseTaskLinterDSL
I found it somewhat difficult to reason about the state of the tree
Traverser because of the usage of mutable variables and data structures.
For example, I determined that the uncheckedWrappers were never used
non-locally so a Set didn't seem to be the right data structure. It was
reasonably straightforward to switch to a more functional style by
parameterizing the method local traverser class.

Bonus:
  - remove unused variable disableNoValueReport
  - add scaladoc to document the input parameters of the traverser class
    so that it's a bit easier to understand for future maintainers.
2018-12-11 12:45:49 -08:00
Ethan Atkins ab2df045ab Lint TaskLinterDSL for intellij
This cleans up all of the yellow intellij warnings for me. It still
complains about some typos, but those manifest as green squiggled lines
which are less annoying.
2018-12-11 12:45:49 -08:00
Eugene Yokota ca7c7d3841 Fix resolver for compiler bridge
I noticed that we can't resolve the compiler bridge out of snapshot repo.
2018-10-05 04:11:08 -04:00
Eugene Yokota 2389106aa6 Use linesIterator for JDK 11 compat 2018-09-27 12:41:47 -04:00
Eugene Yokota ef49a95b7d address more warnings 2018-09-18 17:45:24 -04:00
Eugene Yokota 4ff4f6e45e Update header 2018-09-14 04:53:36 -04:00
xuwei-k ae1fdff968 use SAM type 2018-07-09 13:06:34 +09:00
Eugene Yokota 14a31634e7 Code formatting 2018-06-27 21:06:40 -04:00
eugene yokota eb942f8e5f
Merge pull request #4003 from eed3si9n/wip/opt-delegation3
optimize scope delegation
2018-06-27 15:31:21 -04:00
Eugene Yokota 28e90ea09b Merge branch '1.1.x' into wip/merge-1.1.x 2018-04-29 14:31:30 -04:00
eugene yokota 851ce6ed83
Merge pull request #4090 from eed3si9n/wip/unused-task
Fixes linter that detects missing .value
2018-04-26 12:25:09 -04:00
Eugene Yokota 06d7be0365 handle X / y 2018-04-26 04:44:12 -04:00
Dale Wijnand 8f4b8abb7b
Run scalafmt & test:scalafmt 2018-04-24 16:12:10 +01:00
Dale Wijnand dfff1ed928
Merge pull request #4032 from eed3si9n/wip/servertest
improve server testing
2018-04-18 07:12:53 +02:00
Eugene Yokota 190dc23f70 Fixes linter that detects missing .value
Fixes #4079

#3216 introduced a linter that checks against missing `.value`, but the tree only checked for `Ident`. This doesn't work because in reality the symbols of build.sbt are transformed to `$somehash.npmInstallTask` where `somehash` is the wrapper object we create. Similarly for the built-in keys, they are presented as `sbt.Keys.compile`.

With this change unused task will fail to load the build with the following message:

```
/sbt-4079/build.sbt:26: error: The key `compile` is not being invoked inside the task definition.

Problem: Keys missing `.value` are not initialized and their dependency is not registered.

Solution: Replace `compile` by `compile.value` or remove it if unused.

  compile
  ^
/sbt-4079/build.sbt:27: error: The key `npmInstallTask` is not being invoked inside the task definition.

Problem: Keys missing `.value` are not initialized and their dependency is not registered.

Solution: Replace `npmInstallTask` by `npmInstallTask.value` or remove it if unused.

  npmInstallTask
  ^
```
2018-04-11 01:31:38 -04:00
Dale Wijnand 54a6262f7f
Merge pull request #4041 from dwijnand/Scoped.equals
Give Scoped a default equals/hashCode implementation
2018-04-10 06:10:55 +01:00
Dale Wijnand 3692db9068
Give Scoped a default equals/hashCode implementation 2018-04-04 15:04:53 +01:00
Dale Wijnand 8c1337455d
Fix migrate URL
Fixes #4062
2018-04-04 14:26:03 +01:00
Dale Wijnand af3eec7c31
Merge pull request #4045 from dwijnand/Deprecate-Scope.transformTaskName
Deprecate Scope.transformTaskName
2018-03-28 16:09:39 +01:00
eugene yokota 02acf9380e
Merge pull request #4042 from dwijnand/Reuse-Scoped.scoped-methods
Re-use Scoped.scoped* methods
2018-03-28 09:41:09 -04:00
Dale Wijnand 2e86eed151
Split BuildSettingsInstances out of SlashSyntaxSpec.scala 2018-03-27 16:32:34 +01:00
Dale Wijnand 165aac858e
Merge pull request #4040 from dwijnand/dedup/SlashSyntaxSpec
Dedup keys in SlashSyntaxSpec
2018-03-27 14:54:59 +01:00
Dale Wijnand 35decb6ee5
Deprecate Scope.transformTaskName 2018-03-27 09:59:07 +01:00
Dale Wijnand d19329666a
Re-use Scoped.scoped* methods 2018-03-27 09:24:12 +01:00
Dale Wijnand 3c66f39744
Allow SlashSyntaxSpec to be Scalafmt formatted
Previously:

    [warn] /d/sbt/main-settings/src/test/scala/sbt/SlashSyntaxSpec.scala:44: error: illegal start of simple expression
    [warn] )
    [warn]     ^
2018-03-27 09:22:07 +01:00
Dale Wijnand 42b61a4f8e
Dedup keys in SlashSyntaxSpec 2018-03-27 09:22:06 +01:00
Dale Wijnand 51ff72872b
De-generalise expectValue in SlashSyntaxSpec 2018-03-27 09:22:06 +01:00
Dale Wijnand 006527d246
Re-order SlashSyntaxSpec arbitraries 2018-03-27 09:22:06 +01:00
Eugene Yokota 67e1e4a9ff improve server testing
previously I was using separate thread in a forked test to test the server, but that is not enough isolation to run multiple server tests.
This adds `RunFromSourceMain.fork(workingDirectory: File)`, which allows us to run a fresh sbt on the given working directory. Next, I've refactored the stateful client-side buffer to a class `TestServer`.
2018-03-24 12:09:41 -04:00
Eugene Yokota d4cdb11b53 typo fix 2018-03-13 17:47:11 +09:00
Eugene Yokota d2f2a90d5e handroll for-expression
Ref https://github.com/sbt/sbt/pull/3979

Based on #3979 this handrolls the for-express using while.
2018-03-10 18:43:43 -05:00
Dale Wijnand 10275489d8
Merge pull request #3987 from dwijnand/document-Rich-Init-classes
Document RichInitX classes
2018-03-07 09:44:19 +00:00
Dale Wijnand 621ad2b553
Cleanup TaskNegSpec 2018-03-06 11:59:27 +00:00
Dale Wijnand d26085155d
Cleanup InputWrapper imports 2018-03-06 11:54:11 +00:00
Dale Wijnand 77abb9ee29
Cleanup InputConvert 2018-03-06 11:54:10 +00:00
Dale Wijnand 3921c45563
Document RichInitX classes 2018-03-06 10:37:42 +00:00
Dale Wijnand 90cd60f3b9
Merge branch '1.1.x' into merge-1.1.x-into-1.x
* 1.1.x:
  Use Java's redirectInput rather than sys.process's connectInput
  Re-write toolboxClasspath to use sbt-buildinfo
  Cleanup generateToolboxClasspath
  Upgrade to sbt-buildinfo 0.8.0
  Fix how fullClasspath is defined in TestBuildInfo
  delete buildinfo.BuildInfo from sbt main

Conflicts:
	project/plugins.sbt
2018-02-28 10:30:53 +00:00
Dale Wijnand 27fe5a6957
Re-write toolboxClasspath to use sbt-buildinfo 2018-02-23 13:20:19 +00:00
Colin Dean 3e11b3f000 Fixes link to documentation for deprecated 0.10/0.12 DSL syntax
Fixes sbt/website#558
2018-01-24 23:13:00 -05:00
Dale Wijnand 113656aed1
Remove compile warnings 2018-01-16 11:17:01 +00:00
Dale Wijnand a90832b593
Remove all warnings from mainProj 2017-12-14 15:40:03 +00:00
Dale Wijnand 2390fdfac6
Remove all warnings from mainSettingsProj 2017-12-14 13:16:23 +00:00
Dale Wijnand 9f1d60be60
Rewrite to polymorphic function syntax 2017-10-25 10:23:46 +01:00
Dale Wijnand f662fdda8e
Rewrite to function syntax 2017-10-25 10:22:48 +01:00
Simon Schäfer 93e08b7ee7 Fix warnings about unused pattern vars in various projects 2017-10-19 13:07:24 +02:00
Dale Wijnand 668ace98ee
Remove some duplication when only Scoped is required 2017-10-08 19:07:38 +01:00
Dale Wijnand 5e0b080f51
Fix some minor errors 2017-10-07 18:42:26 +01:00
Dale Wijnand ef4828cfc2
Add slash syntax for AttrKey that parity with "in" syntax 2017-10-07 13:12:17 +01:00
Dale Wijnand d19c350687
Correct name of WithoutScope properties 2017-10-07 13:12:16 +01:00
Dale Wijnand 8af3110c00
Add .previous tests to SlashSyntaxTest 2017-10-07 13:12:16 +01:00
Dale Wijnand fb28c201c8
Make scoped keys more frequent 2017-10-07 13:12:16 +01:00
Dale Wijnand 74ac7d9e07
Drop redudant label in expectValue 2017-10-07 13:12:15 +01:00
Dale Wijnand 025075efd0
Dedup, cleanup & cover all key types 2017-10-07 13:12:15 +01:00
Dale Wijnand 165dc794ca
Extract gen{Input,Setting,Task}Key 2017-10-07 13:12:15 +01:00
Dale Wijnand 18d615701f
Remove WithScope wrapping 2017-10-07 13:12:15 +01:00
Dale Wijnand 0bcd7c3b6d
Remove boilerplate around sbtSlashSyntaxRichConfiguration 2017-10-07 12:08:32 +01:00
Dale Wijnand e5898111fe
Replace CanScope with Scoped.ScopingSetting 2017-10-07 12:07:19 +01:00
Eugene Yokota 2a6385fd94 Remove thunk for slash syntax
Ref #3606, #3611, and #3613

This removes unnecessary thunk for slash syntax.
The semantics using this approach is strictly better than the previous `in (ref, config, task)`. By removing the thunk, we retain `(a / b) / c == a / b / c`.

See the following example:

```scala
scala> import sbt._, Keys._
scala> val t: TaskKey[Unit] = (test in Test)
t: sbt.TaskKey[Unit] = TaskKey(This / Select(ConfigKey(test)) / This / test)

scala> ThisBuild / t
ThisBuild / t
res1: sbt.TaskKey[Unit] = TaskKey(Select(ThisBuild) / Select(ConfigKey(test)) / This / test)

scala> ThisBuild / t / name
ThisBuild / t / name
res2: sbt.SettingKey[String] = SettingKey(Select(ThisBuild) / Select(ConfigKey(test)) / Select(test) / name)
```

so far so good? Now look at this:

```
scala> scala> name in (ThisBuild, t)
name in (ThisBuild, t)
res3: sbt.SettingKey[String] = SettingKey(Select(ThisBuild) / This / Select(test) / name)
```

`Test` configuration knowledge is lost! For `in (..)` maybe it was ok because mostly we don't use unscoped keys, but that's the difference between `in (..)` and `/`.

Fixes #3605
2017-10-06 15:47:46 -04:00
Dale Wijnand 5b03379693
Extract withScope 2017-10-06 17:41:06 +01:00
Dale Wijnand 9b526f54bf
Rarely include custom scopes in key generators 2017-10-06 15:41:28 +01:00
Dale Wijnand db87e4c871
Add SlashSyntaxSpec to validate syntax parity 2017-10-06 15:07:18 +01:00
Dale Wijnand 97ddc1ffb7
Copy the non-runtime parts of project/unified to SlashSyntaxTest 2017-10-06 15:07:17 +01:00
Dale Wijnand f22843f91c
Move SlashSyntax to the settings component 2017-10-06 09:52:34 +01:00
eugene yokota 1251817cf2 Merge pull request #3612 from dwijnand/scope-toString
Handle Global in Scope#toString
2017-10-05 17:43:25 -04:00
Dale Wijnand 2a5ba9475d
Handle Global in Scope#toString
Print "Global" instead of "Zero / Zero / Zero".
2017-10-05 20:29:42 +01:00
Dale Wijnand f4b2fc4228
Correct handling of resolving ThisProject
In ca71b4b902 I went about fixing the
inexhaustive matching in Scope's resolveProjectBuild and
resolveProjectRef. Looking back the change was wrong.

For resolveProjectBuild the new implementation is less wrong, but still
not great, seeing as it doesn't actually do any build resolving.

For resolveProjectRef the new implementation now blows up instead of
lies. Which means it's less leneant, more "fail-fast".

isProjectThis is unused; remnant of the pre-AutoPlugin days when build
settings where defined in Plugin.settings.
2017-10-05 19:00:48 +01:00
Dale Wijnand a41727fb17
Add, configure & enforce file headers 2017-10-05 09:03:40 +01:00
Eugene Yokota 60f2498c0a Implement toString for keys
toString added for REPL testing:

```
scala> Zero / Zero / Zero / name
res0: sbt.SlashSyntax.ScopeAndKey[sbt.SettingKey[String]] = Zero / Zero / Zero / name
```
2017-10-05 02:46:09 -04:00
Eugene Yokota b0306b738e Add whitespaces in Show of scoped keys
```
Provided by:
    ProjectRef(uri("...."), "root") / Test / test
Dependencies:
    Test / executeTests
    Test / test / streams
    Test / state
    Test / test / testResultLogger
```
2017-09-28 03:34:49 -04:00
Eugene Yokota 33a01f3ceb Unified slash syntax
Fixes sbt/sbt#1812

This adds unified slash syntax for both sbt shell and the build.sbt DSL.
Instead of the current `<project-id>/config:intask::key`,
this adds `<project-id>/<config-ident>/intask/key` where <config-ident> is the Scala identifier notation for the configurations like `Compile` and `Test`.

This also adds a series of implicits called `SlashSyntax` that adds `/` operators to project refererences, configuration, and keys such that the same syntax works in build.sbt.

These examples work for both from the shell and in build.sbt.

    Global / cancelable
    ThisBuild / scalaVersion
    Test / test
    root / Compile / compile / scalacOptions
    ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/scalacOptions
    Zero / Zero / name

The inspect command now outputs something that can be copy-pasted:

    > inspect compile
    [info] Task: sbt.inc.Analysis
    [info] Description:
    [info] 	Compiles sources.
    [info] Provided by:
    [info] 	ProjectRef(uri("file:/xxx/helloworld/"),"root")/Compile/compile
    [info] Defined at:
    [info] 	(sbt.Defaults) Defaults.scala:326
    [info] Dependencies:
    [info] 	Compile/manipulateBytecode
    [info] 	Compile/incCompileSetup
    [info] Reverse dependencies:
    [info] 	Compile/printWarnings
    [info] 	Compile/products
    [info] 	Compile/discoveredSbtPlugins
    [info] 	Compile/discoveredMainClasses
    [info] Delegates:
    [info] 	Compile/compile
    [info] 	compile
    [info] 	ThisBuild/Compile/compile
    [info] 	ThisBuild/compile
    [info] 	Zero/Compile/compile
    [info] 	Global/compile
    [info] Related:
    [info] 	Test/compile
2017-09-28 01:01:43 -04:00
eugene yokota 45765583a6 Merge branch '1.0.x' into 1.0.x 2017-09-15 19:47:08 -04:00
Răzvan Flavius Panda 0124a8ad0e Fix unused imports warnings 2017-09-15 16:35:08 +01:00
Dale Wijnand 936733b2b1
Cleanup 2017-09-13 16:03:51 +01:00
Dale Wijnand d3d105515a
Don't warn on by-name keys
.. which is apparently what the type of bare vals in *.sbt files are.

Fixes #3299, again.
2017-09-11 14:46:33 +01:00
Dale Wijnand 8d9a3a35d2
Fix warnings in TaskLinterDSL.scala 2017-09-11 14:45:40 +01:00
eugene yokota 33d3ba9d7c Merge pull request #3438 from Duhemm/source-appender
[1.0.x] `Append` instance to add `File` to `Seq[Source]`
2017-08-23 18:31:58 -04:00
Dale Wijnand 4f0d2f9ffd
Scalafmt 1.2.0 2017-08-14 15:47:52 +01:00
Martin Duhem 5815f1db0a
`Append` instance to add `File` to `Seq[Source]` 2017-08-14 15:47:17 +02:00
Dale Wijnand 805b76f3d4
Add back, re-configure & re-enable Scalafmt 2017-08-10 16:35:23 +01:00
Eugene Yokota 467617a4b9 Implement withRank 2017-07-25 01:50:53 -04:00