Commit Graph

9384 Commits

Author SHA1 Message Date
Grzegorz Kossakowski 4b0123f9ec Merge pull request #1002 from gkossakowski/deps-by-treewalking
Extract dependencies by tree walking
2013-11-27 06:51:05 -08:00
Mark Harrah e16864f2e7 Drop Thread.getName from use in computeID(thread) in TrapExit SecurityManager to fix #997. 2013-11-27 08:53:52 -05:00
Mark Harrah ad60e6c794 Basic predicate combinators for Tags 2013-11-26 22:46:50 -05:00
Mark Harrah d38450b41f Command to run multiple tasks concurrently: 'all a b'. Fixes #628. 2013-11-26 22:46:49 -05:00
Grzegorz Kossakowski 331fffbb19 Add test for trait as a first parent scenario in dep tracking.
The documentation of `Relations.inheritance` mentions an oddity of Scala's
type checker which manifests itself in what is being tracked by that
relation in case of traits being first parent for a class/trait.

Add a test case which verifies that this oddity actually exists and it's
not harmful because it doesn't break an invariant between `memberRef`
and `inheritance` relations.
2013-11-26 18:39:24 +01:00
Grzegorz Kossakowski 2226fccd4f Test `memberRef` and `inheritance` in DependencySpecification.
Flip `memberRefAndInheritanceDeps` flag to true which allows us to
test `memberRef` and `inheritance` relations instead of `direct` and
`publicInherited` as it was previously done.

There a few changes to extracted dependencies from public members:

  * F doesn't depend on C by inheritance anymore. The dependency on
    C was coming from self type. This shows that dependencies from self
    types are not considered to be dependencies introduces by inheritance
    anymore.
  * G depends on B by member reference now. This dependency is introduced
    by applying type constructor `G.T` and expanding the result of the
    application.
  * H doesn't depend on D by inheritance anymore. That dependency was
    introduced through B which inherits from D. This shows that only
    parents (and not all base classes) are included in `inheritance`
    relation.

NOTE: The second bullet highlights a bug in the old dependency tracking
logic. The dependency on B was recorded in `publicInherited` but not in
`direct` relation. This breaks the contract which says that
`publicInherited` is a subset of `direct` relation.

This a change to dependencies extracted from non-public members:

  * C depends on A by inheritance and D depends on B by inheritance now;
    both changes are of the same kind: dependencies introduced by
    inheritance are tracked for non-public members now. This is necessary
    for name hashing correctness algorithm
2013-11-26 18:39:24 +01:00
Grzegorz Kossakowski 4da31cd27d Extract source code dependencies by tree walking.
Previously incremental compiler was extracting source code
dependencies by inspecting `CompilationUnit.depends` set. This set is
constructed by Scala compiler and it contains all symbols that given
compilation unit refers or even saw (in case of implicit search).
There are a few problems with this approach:

  * The contract for `CompilationUnit.depend` is not clearly defined
    in Scala compiler and there are no tests around it. Read: it's
    not an official, maintained API.
  * Improvements to incremental compiler require more context
    information about given dependency. For example, we want to
    distinguish between dependency on a class when you just select
    members from it or inherit from it. The other example is that
    we might want to know dependencies of a given class instead of
    the whole compilation unit to make the invalidation logic more
    precise.

That led to the idea of pushing dependency extracting logic to
incremental compiler side so it can evolve indepedently from Scala
compiler releases and can be refined as needed. We extract
dependencies of a compilation unit by walking a type-checked tree
and gathering symbols attached to them.

Specifically, the tree walk is implemented as a separate phase that
runs after pickler and extracts symbols from following tree nodes:

  * `Import` so we can track dependencies on unused imports
  * `Select` which is used for selecting all terms
  * `Ident` used for referring to local terms, package-local terms
            and top-level packages
  * `TypeTree` which is used for referring to all types

Note that we do not extract just a single symbol assigned to `TypeTree`
node because it might represent a complex type that mentions
several symbols. We collect all those symbols by traversing the type
with CollectTypeTraverser. The implementation of the traverser is inspired
by `CollectTypeCollector` from Scala 2.10. The
`source-dependencies/typeref-only` test covers a scenario where the
dependency is introduced through a TypeRef only.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski 533a5b8c23 Add specification for extracted source dependencies.
Add specs2 specification (unit test) which documents current dependency
extraction logic's behavior. It exercises `direct` and `publicInherited`
relations.

This test is akin to `source-dependencies/inherited-dependencies` scripted
test. We keep both because this test will diverge in next commit to test
`memberRef` and `inheritance` relations.

The idea behind adding this test and then modifying the
`memberRefAndInheritanceDeps` flag so we test `memberRef` and `inheritance`
is that we can show precisely the differences between those two dependency
tracking mechanisms.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski 2551eb2a63 Do not add source dependencies on itself.
Adding source dependency on itself doesn't really bring any value so
there's no reason to do it. We avoided recording that kind of dependencies
by performing a check in `AnalysisCallback` implementation. However, if we
have another implementation like `TestCallback` used for testing we do
not benefit from that check.

Therefore, the check has been moved to dependency phase were dependencies
are collected.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski de1c5a4aed Add support for unit testing of extracted source dependencies.
Add `extractDependenciesFromSrcs` method to ScalaCompilerForUnitTest
class which allows us to unit test dependency extraction logic.

See the comment attached to the method that explain the details of
how it should be used.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski 89914975e1 Refactor ScalaCompilerForUnitTesting.
Refactor ScalaCompilerForUnitTesting by introducing a new method
`extractApiFromSrc` which better describes the intent than
`compileSrc`. The `compileSrc` becomes a private, utility method.

Also, `compileSrc` method changed it's signature so it can take
multiple source code snippets as input. This functionality will
be used in future commits.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski aac19fd02b Extract source code dependencies by tree walking.
Previously incremental compiler was extracting source code
dependencies by inspecting `CompilationUnit.depends` set. This set is
constructed by Scala compiler and it contains all symbols that given
compilation unit refers or even saw (in case of implicit search).
There are a few problems with this approach:

  * The contract for `CompilationUnit.depend` is not clearly defined
    in Scala compiler and there are no tests around it. Read: it's
    not an official, maintained API.
  * Improvements to incremental compiler require more context
    information about given dependency. For example, we want to
    distinguish between dependency on a class when you just select
    members from it or inherit from it. The other example is that
    we might want to know dependencies of a given class instead of
    the whole compilation unit to make the invalidation logic more
    precise.

That led to the idea of pushing dependency extracting logic to
incremental compiler side so it can evolve indepedently from Scala
compiler releases and can be refined as needed. We extract
dependencies of a compilation unit by walking a type-checked tree
and gathering symbols attached to them.

Specifically, the tree walk is implemented as a separate phase that
runs after pickler and extracts symbols from following tree nodes:

  * `Import` so we can track dependencies on unused imports
  * `Select` which is used for selecting all terms
  * `Ident` used for referring to local terms, package-local terms
            and top-level packages
  * `TypeTree` which is used for referring to all types

Note that we do not extract just a single symbol assigned to `TypeTree`
node because it might represent a complex type that mentions
several symbols. We collect all those symbols by traversing the type
with CollectTypeTraverser. The implementation of the traverser is inspired
by `CollectTypeCollector` from Scala 2.10. The
`source-dependencies/typeref-only` test covers a scenario where the
dependency is introduced through a TypeRef only.
2013-11-26 18:39:23 +01:00
Grzegorz Kossakowski b8371691f2 Introduce `memberRef` and `inheritance` relations.
Introduce an alternative source dependency tracking mechanism that is
needed by upcoming name hashing algorithm. This new mechanism is
implemented by introducing two new source dependency relations called
`memberRef` and `inheritance`.

Those relations are very similar to existing `direct` and
`publicInherited` relations in some subtle ways. Those differences
will be highlighted in the description below.

Dependencies between source files are tracked in two distinct
categories:

  * dependencies introduced by inheriting from a class/trait
    defined in other source file
  * dependencies introduced by referring (selecting) a member
    defined in other source file (that covers all other
    kinds of dependencies)

Due to invalidation algorithm implementation details sbt would need to
track inheritance dependencies of public classes only. Thus, we had
relation called `publicInherited`. The name hashing algorithm which
improves invalidation logic will need more precise information about
dependencies introduced by inheritance including dependencies of non-public
classes. That's one difference between `inheritance` and `publicInherited`
relations.

One surprising (to me) thing about `publicInherited` is that it includes
all base classes of a given class and not just parents. In that sense
`publicInherited` is transitive. This is a bit irregular because
everything else in Relations doesn't include transitive dependencies.

Since we are introducing new relations we have an excellent chance to
make things more regular. Therefore `inheritance` relation is
non-transitive and includes only extracted parent classes.

The access to `direct`, `publicInherited`, `memberRef` and `inheritance`
relations is dependent upon the value of `memberRefAndInheritanceDeps`
flag. Check documentation of that flag for details.

The two alternatives for source dependency tracking are implemented by
introduction of two subclasses that implement Relations trait and one
abstract class that contains some common logic shared between those two
subclasses. The two new subclasses are needed for the time being when we
are slowly migrating to the name hashing algorithm which requires
subtle changes to dependency tracking as explained above. For some time we
plan to keep both algorithms side-by-side and have a runtime switch which
allows to pick one. So we need logic for both old and new dependency
tracking to be available. That's exactly what two subclasses of
MRelationsCommon implement. Once name hashing is proven to be stable and
reliable we'll phase out the old algorithm and the old dependency tracking
logic.
2013-11-26 18:39:13 +01:00
Mark Harrah 8857a4fb9a Add -Dsbt.cli.nodelegation option to experiment with no delegation for running/showing tasks/settings from the command line.
With this set to true, the following is no longer allowed for example:

> compile:update
2013-11-25 21:03:40 -05:00
Mark Harrah 13041bfb9a Don't mark ExceptionInInitializerError as incompatible plugins: rethrow the cause instead. Fixes #1007. 2013-11-25 21:03:40 -05:00
Mark Harrah 5f7a327e5e Add -Dsbt.cli.nodelegation option to experiment with no delegation for running/showing tasks/settings from the command line.
With this set to true, the following is no longer allowed for example:

> compile:update
2013-11-25 21:03:40 -05:00
Grzegorz Kossakowski c5317a1f72 Work harder on generating unique values in generator for Analysis.
The TestCaseGenerators uses global set for ensuring that certain generated
values are unique. This is not the best design because the more properties
you check the harder is to generate new sample inputs because of already
accumulated values. This results in:

[info] + Analysis.Simple Merge and Split: OK, proved property.
[info] ! Analysis.Complex Merge and Split: Gave up after only 8 passed tests. 93 tests were discarded.

I don't have an ambition to reduce the scope of this global set but at
least I wanted to make generators to work a bit harder on generating
samples.

Instead of using `suchThat` method for filtering out non-unique samples
we use `retryUntil` that never gives up (therefore it might not
terminate). We had to upgrade to latest (1.11.1) version of scalacheck
in order to have an access to `retryUntil` method.

Also, I overridden the `identifier` to delegate to original
`Gen.identifier` but with minimal size set to be to '3'. This means,
the generated identifier will be of size 3 or larger which is needed in
order to avoid collisions.
2013-11-25 18:50:49 +01:00
Ikenna Nwaiwu 5174f44dc3 Update Tasks.rst
Updating typo in documentation
2013-11-25 08:22:16 -05:00
Mark Harrah 818f4f96fb Docs: add section on Task.taskDyn. Fixes #999. 2013-11-24 18:24:15 -05:00
Mark Harrah fcb35f3b8f Add Initialize[Task[T]].taskValue: Task[T] to allow selecting the Task for use by *Generators. Fixes #866. 2013-11-24 18:24:15 -05:00
Grzegorz Kossakowski a9a15748cc Improve failure message in `inherited-dependencies` test.
When the `source-dependencies/inherited-dependencies` test fails we
get a dump of a big collection of all dependencies with absolute
file paths printed. This is not very readable when one needs to
understand the actual difference.

I decided to test dependencies of each source file separately. This way
when assertion exception is thrown we get a stack trace that points
us at the line which tested dependencies of a specific source file.
Also, all files are relative to baseDirectory of the project.
2013-11-25 00:15:41 +01:00
Mark Harrah 1d9b44d5d7 Expand aliases instead of evaluating directly.
This avoids an additional cause of recursion via the semicolon/multiple command, which fixes #933.
It also provides error messages on the expanded command.  This fixes #598.
2013-11-24 15:19:24 -05:00
Mark Harrah e268db3f80 Propagate argument to 'reload' to load failure handling command. Fixes #973. 2013-11-24 15:19:24 -05:00
Mark Harrah 7f8c056452 Docs: reorganize table of contents. 2013-11-23 20:51:52 -05:00
Mark Harrah 5a0d27356e Docs: Document warn, --warn, etc... in howto 2013-11-23 19:36:57 -05:00
Mark Harrah 1a3c88ef20 Log at the debug level the commands being executed by the command engine. 2013-11-23 19:18:54 -05:00
Mark Harrah 4d7dccb02e Remove the need for resetLocalAttrs. Fixes #994, #952.
The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.
2013-11-22 13:08:10 -05:00
Mark Harrah 4ec88dba43 Remove the need for resetLocalAttrs. Fixes #994, #952.
The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.
2013-11-22 13:08:10 -05:00
Mark Harrah 3b213e59ca Remove the need for resetLocalAttrs. Fixes #994, #952.
The fix was made possible by the very helpful information provided by @retronym.

This commit does two key things:
 1. changes the owner when splicing original trees into new trees
 2. ensures the synthetic trees that get spliced into original trees do not need typechecking

Given this original source (from Defaults.scala):

  ...
  lazy val sourceConfigPaths = Seq(
    ...
    unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
    ...
  )
  ...

After expansion of .value, this looks something like:

    unmanagedSourceDirectories := Seq(
      InputWrapper.wrapInit[File](scalaSource),
      InputWrapper.wrapInit[File](javaSource)
    )

where wrapInit is something like:

    def wrapInit[T](a: Any): T

After expansion of := we have (approximately):

    unmanagedSourceDirectories <<=
      Instance.app( (scalaSource, javaSource) ) {
        $p1: (File, File) =>
          val $q4: File = $p1._1
          val $q3: File = $p1._2
          Seq($q3, $q4)
      }

So,

 a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
 b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
 c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
 d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
 e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions

changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.

An example of the compiler crash seen when the symbol of the references is not that of the vals:

symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
	at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
	at scala.tools.nsc.Global.abort(Global.scala:254)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
	at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
   ...

Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.

If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:

java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
   ...
     while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
        during phase: global=lambdalift, atPhase=constructors
   ...
  last tree to typer: term $outer
              symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
   symbol definition: private[this] val $outer: sbt.BuildCommon
                 tpe: <notype>
       symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
      context owners: value dir -> value globalCore -> object Defaults -> package sbt
   ...

The problem here is the difference between context owners and the proxy search chain.
2013-11-22 13:08:10 -05:00
Bruno Bieth e9a54600f5 made fork-parallel scripted test a little more robust 2013-11-21 08:20:43 -05:00
Bruno Bieth 5a88bd2302 Third draft to execute the forked tests in parallel.
This feature is not activated by default. To enable it set `testForkedParallel` to `true`.

The test-agent then executes the tests in a thread pool.
For now it has a fixed size set to the number of available processors.
The concurrent restrictions configuration should be used.
2013-11-21 08:20:43 -05:00
Heikki Vesalainen 9989e5631f Completion command: support quoted strings 2013-11-21 08:20:37 -05:00
Heikki Vesalainen b2980b913f completions command
The completions command is meant for dump terminals that cannot use
the default tab completion. It has been built for use by the emacs
sbt-mode (see https://github.com/hvesalai/sbt-mode), but is equally
useful for other code editors that can integrate with sbt.
2013-11-21 08:20:37 -05:00
Anthony Whitford e7d5cd1bd2 Revised based on pull request feedback. 2013-11-19 07:52:35 -05:00
Anthony Whitford 86b0098078 Resolved bugs reported in Issue 947. Also added SNAPSHOT suffix to the version by default. 2013-11-19 07:52:35 -05:00
Mark Harrah f3c050921a Fixes #989, #990. TrapExit jvm-independent and awt handling is only done when awt is used.
SecurityManager.checkAccess(ThreadGroup) is specified to be called for every Thread creation
and every ThreadGroup creation and is therefore jvm-independent.  This can be used to get all
Threads associated with an application with good enough accuracy.

An application will be marked as using AWT if it gets associated with the AWT event queue thread.
To avoid unwanted side effects of accidental AWT initialization, TrapExit only tries to dispose
frames when an application is so marked.  Only one AWT application is supported due to a lack of
a way to associate displayed windows with an application.
2013-11-18 17:35:06 -05:00
Grzegorz Kossakowski c16d3e53cc Merge pull request #988 from benjyw/text_analysis_file2
Replace binary Analysis format with a text-based one.
2013-11-18 03:10:53 -08:00
Benjy e6bf85a30b Replace binary Analysis format with a text-based one.
Reads/writes are a little faster with the text format,
and it's far more useful. E.g., it allows external manipulation
and inspection of the analysis.

We don't gzip the output. It does greatly shrink the files,
however it makes reads and writes 1.5x-2x slower, and we're
optimizing for speed over compactness.
2013-11-16 13:59:02 -08:00
Bruno Bieth e4dd71c08c execute scripted test without publishing SBT first
Saves you some time when only your test has changed.
2013-11-13 15:51:21 -05:00
drdamour 54075c750d resubmit of #983 2013-11-13 15:49:29 -05:00
Grzegorz Kossakowski bf0ef95356 Merge pull request #976 from gkossakowski/macro-handling
Make recompilation on macro definition optional
2013-11-12 12:40:51 -08:00
Grzegorz Kossakowski 698e24da11 Mark fields storing keys in IncOptions as private.
It was an omission in the original commit that introduced them and didn't
mark them as private. They are purely an implementation detail and should
be hidden. We hiding them now.
2013-11-12 21:39:18 +01:00
Grzegorz Kossakowski 39036e7c20 Make recompilation on macro definition optional.
Introduce a new incremental compiler option that controls
incremental compiler's treatment of macro definitions and their clients.
The current strategy is that whenever a source file containing a macro
definition is touched it will cause recompilation of all direct
dependencies of that file.

That strategy has proven to be too conservative for some projects like
Scala compiler of specs2 leading to too many source files being recompiled.
We make this behavior optional by introducing a new option
`recompileOnMacroDef` in `IncOptions` class. The default value is set to
`true` which preserves the previous behavior.
2013-11-12 21:33:19 +01:00
Grzegorz Kossakowski a6f04cf53b Add specialized copy methods to IncOptions class.
Add methods that allow one to set a new value to one of the fields of
IncOptions class. These methods are meant to be an alternative to
copy method that is hard to keep binary compatible when new fields are
added to the class.

Each copying method is related to one field of the class so when new
fields are added existing methods (and their signatures) are unaffected.
2013-11-12 21:32:14 +01:00
Grzegorz Kossakowski b77b0e161e Desugar case class IncOptions in binary compatible way.
Expand case class `IncOptions` in binary compatible way so we can have
better control of methods like `unapply` when new fields are added.

Great precaution has been taken to ensure that this commit doesn't break
binary compatibility. I took a dump of javap output before and after
this change for both the class and it's companion object.
The diff is presented below:

diff -u ~/inc-options-before ~/inc-options-after
--- /Users/grek/inc-options-before	2013-11-03 14:48:45.000000000 +0100
+++ /Users/grek/inc-options-after	2013-11-03 15:53:10.000000000 +0100
@@ -9,7 +9,11 @@
     public static java.lang.String transitiveStepKey();
     public static sbt.inc.IncOptions setTransactional(sbt.inc.IncOptions, java.io.File);
     public static sbt.inc.IncOptions defaultTransactional(java.io.File);
+    public static scala.Option unapply(sbt.inc.IncOptions);
+    public static sbt.inc.IncOptions apply(int, double, boolean, boolean, int, scala.Option, scala.Function0);
     public static sbt.inc.IncOptions Default();
+    public static scala.Function1 tupled();
+    public static scala.Function1 curried();
     public int transitiveStep();
     public double recompileAllFraction();
     public boolean relationsDebug();

diff -u inc-options-module-before inc-options-module-after
--- inc-options-module-before	2013-11-03 14:48:55.000000000 +0100
+++ inc-options-module-after	2013-11-12 21:00:41.000000000 +0100
@@ -3,6 +3,9 @@
     public static final sbt.inc.IncOptions$ MODULE$;
     public static {};
     public sbt.inc.IncOptions Default();
+    public final java.lang.String toString();
+    public sbt.inc.IncOptions apply(int, double, boolean, boolean, int, scala.Option, scala.Function0);
+    public scala.Option unapply(sbt.inc.IncOptions);
     public sbt.inc.IncOptions defaultTransactional(java.io.File);
     public sbt.inc.IncOptions setTransactional(sbt.inc.IncOptions, java.io.File);
     public java.lang.String transitiveStepKey();
@@ -13,7 +16,5 @@
     public java.lang.String apiDiffContextSize();
     public sbt.inc.IncOptions fromStringMap(java.util.Map);
     public java.util.Map toStringMap(sbt.inc.IncOptions);
-    public sbt.inc.IncOptions apply(int, double, boolean, boolean, int, scala.Option, scala.Function0);
-    public scala.Option unapply(sbt.inc.IncOptions);
 }

The first diff shows that there are just more static forwarders defined
for top-level companion object and that is binary compatible change.

The second diff shows that there are just a few minor differences in
order in which `unapply`, `apply` and bridge method for `apply` are
defined. Also, there's a new `toString` declaration. All those changes are
binary compatible.

All methods that are generated for a case class are marked as deprecated
and will be removed in the future.
2013-11-12 21:31:28 +01:00
Grzegorz Kossakowski f2bdeff486 Merge pull request #941 from gkossakowski/apichanges-refactor
More detailed logging and APIChanges refactoring
2013-11-11 06:51:51 -08:00
Grzegorz Kossakowski 4b43110a2c Represent api changes as values and cleanup APIChanges class.
The main motivation behind this commit is to reify information about
api changes that incremental compiler considers. We introduce a new
sealed class `APIChange` that has (at the moment) two subtypes:

  * APIChangeDueToMacroDefinition - as the name explains, this represents
    the case where incremental compiler considers an api to be changed
    just because given source file contains a macro definition
  * SourceAPIChange - this represents the case of regular api change;
    at the moment it's just a simple wrapper around value representing
    source file but in the future it will get expanded to contain more
    detailed information about API changes (e.g. collection of changed
    name hashes)

The APIChanges becomes just a collection of APIChange instances.
In particular, I removed `names` field that seems to be a dead code in
incremental compiler. The `NameChanges` class and methods that refer to
it in `SameAPI` has been deprecated.

The Incremental.scala has been adapted to changed signature of APIChanges
class. The `sameSource` method returns representation of APIChange
(if there's one) instead of just simple boolean. One notable change is
that information about APIChanges is pushed deeper into invalidation logic.
This will allow us to treat the APIChangeDueToMacroDefinition case properly
once name hashing scheme arrives.

This commit shouldn't change any behavior and is purely a refactoring.
2013-11-11 15:43:28 +01:00
Mark Harrah 4f81512109 Fix source-dependencies/relative-source-error to use explicit reloads. Ref #923. 2013-11-11 09:41:08 -05:00
Mark Harrah 7d10c7e103 Docs: correct substitution syntax when preceding character is symbol. Fixes #970. 2013-11-11 09:30:31 -05:00
Grzegorz Kossakowski 4ed8abd4fb More detailed logging of incremental compiler's invalidation logic.
The following events are logged:

  * invalidation of source file due to macro definition
  * inclusion of dependency invalidated by inheritance; we log both
    nodes of dependency edge (dependent and dependency)

The second bullet helps to understand what's going on in case of
complex inheritance hierarchies like in Scala compiler.
2013-11-11 15:27:18 +01:00