Tracking of used names is a component needed by the name hashing
algorithm. The extraction and storage of used names is active only when
`AnalysisCallback.nameHashing` flag is enabled and it's disabled by
default.
This change constists of two parts:
1. Modification of Relations to include a new `names` relation
that allows us to track used names in Scala source files
2. Implementation of logic that extracts used names from Scala
compilation units (that correspond to Scala source files)
The first part is straightforward: add standard set of methods in
Relations (along with their implementation) and update the logic which
serializes and deserializes Relations.
The second part is implemented as tree walk that collects all symbols
associated with trees. For each symbol we extract a simple, decoded name
and add it to a set of extracted names. Check documentation of
`ExtractUsedNames` for discussion of implementation details.
The `ExtractUsedNames` comes with unit tests grouped in
`ExtractUsedNamesSpecification`. Check that class for details.
Given the fact that we fork while running tests in `compiler-interface`
subproject and tests are ran in parallel which involves allocating
multiple Scala compiler instances we had to bump the default memory limit.
This commit contains fixes for gkossakowski/sbt#3, gkossakowski/sbt#5 and
gkossakowski/sbt#6 issues.
The previous name of the flag was rather specific: it indicated
whether the new source dependency tracking is supported by given Relations
object. However, there will be more functionality added to Relations that
is specific to name hashing algorithm. Therefore it makes sense to name
the flag as just `nameHashing`.
I decided to rename Relations implementation classes to be more
consistent with the name of the flag and with the purpose they serve.
The flag in AnalysisCallback (and classes implementing it) has been
renamed as well.
Previously incremental compiler was extracting source code
dependencies by inspecting `CompilationUnit.depends` set. This set is
constructed by Scala compiler and it contains all symbols that given
compilation unit refers or even saw (in case of implicit search).
There are a few problems with this approach:
* The contract for `CompilationUnit.depend` is not clearly defined
in Scala compiler and there are no tests around it. Read: it's
not an official, maintained API.
* Improvements to incremental compiler require more context
information about given dependency. For example, we want to
distinguish between dependency on a class when you just select
members from it or inherit from it. The other example is that
we might want to know dependencies of a given class instead of
the whole compilation unit to make the invalidation logic more
precise.
That led to the idea of pushing dependency extracting logic to
incremental compiler side so it can evolve indepedently from Scala
compiler releases and can be refined as needed. We extract
dependencies of a compilation unit by walking a type-checked tree
and gathering symbols attached to them.
Specifically, the tree walk is implemented as a separate phase that
runs after pickler and extracts symbols from following tree nodes:
* `Import` so we can track dependencies on unused imports
* `Select` which is used for selecting all terms
* `Ident` used for referring to local terms, package-local terms
and top-level packages
* `TypeTree` which is used for referring to all types
Note that we do not extract just a single symbol assigned to `TypeTree`
node because it might represent a complex type that mentions
several symbols. We collect all those symbols by traversing the type
with CollectTypeTraverser. The implementation of the traverser is inspired
by `CollectTypeCollector` from Scala 2.10. The
`source-dependencies/typeref-only` test covers a scenario where the
dependency is introduced through a TypeRef only.
As pointed out by @harrah in #705, both beginSource and endSource are
not used in sbt internally for anything meaningful.
We've discussed an option of deprecating those methods but since they
are not doing anything meaningful Mark prefers to have compile-time
error in case somebody implements or calls those methods. I agree with
that hence removal.
Incremental compiler didn't have any explicit logic to handle
cancelled compilation so it would go into inconsistent state.
Specifically, what would happen is that it would treat cancelled
compilation as a compilation that finished normally and try to
produce a new Analysis object out of partial information collected
in AnalysisCallback. The most obvious outcome would be that the
new Analysis would contain latest hashes for source files. The
next time incremental compiler was asked to recompile the same files
that it didn't recompile due to cancelled compilation it would think
they were already successfully compiled and would do nothing.
We fix that problem by following the same logic that handles compilation
errors, cleans up partial results (produced class files) and makes sure
that no Analysis is created out of broken state.
We do that by introducing a new exception `CompileCancelled`
and throwing it at the same spot as an exception signalizing compilation
errors is being thrown. We also modify `IncrementalCompile` to
catch that exception and gracefully return as there was no compilation
invoked.
NOTE: In case there were compilation errors reported _before_
compilation cancellations was requested we'll still report them
using an old mechanism so partial errors are not lost in case
of cancelled compilation.
Specifically, when the Scala version for sbt is the same as that for the project being built,
the jars in UpdateReport should be the same as those in ScalaProvider. This is because the
loader will come from the ScalaProvider, which uses jars in the boot directory instead of the
cache. The first part of the fix for #661 checks that loaded classes come from the classpath
and so they need to line up.
Introduce a way to configure incremental compiler itself instead
of underlying Java/Scala compiler.
Specific list of changes in this commit:
* Add a method to `xsbti.compile.Setup` that returns incremental
compiler options as a `java.util.Map<String, String>`. We considered
statis interface instead of a `Map` but based on mailing
list feedback we decided that it's not the best way to go because
static interface is hard to evolve it by adding new options.
* Since passing `java.util.Map<String, String>` not very convenient
we convert it immediately to `sbt.inc.IncOptions`
* Add options argument to various methods/classes that implement
incremental compilation so in the end options reach
`sbt.inc.IncOptions` object
* Add `incOptions` task that allows users to configure incremental
compiler options in their build files. Default implementation of
that tasks returns just `IncOptions.DEFAULT`
* Both system property `xsbt.inc.debug` and `IncOptions.relationsDebug`
trigger debugging of relations now. In the near future, we should
deprecate use of `xsbt.inc.debug`.
- Read macro modifier from method definition.
- Always recompile downstream files after a file containing macro defs is recompiled.
- Source is extended with a hasMacro attribute. Mark suggests that this might be better
tracked in Relations, but I'm not sure how to make that change.
the custom scalac Reporter now delegates to an instance of
an sbt interface called xsbti.Reporter
handling compilation logging is now mainly done on the sbt-side of the
compiler interface
the xsbti.Reporter interface provides access to richer information
about errors and warnings, including source file, line, and offset
xsbti.Reporter can be implemented by users to get access to
detailed information without needing to parse the logging output
the CompileFailed exception that is thrown when compilation fails now
includes an array of the problems, providing detailed
error and warning information that can, for example, be consumed
by doing a mapFailure on 'compile' and using 'Compile.allProblems'
support lazy arguments in data type generator
SafeLazy implementation that explicitly clears the reference to the thunk
in API representation, drop synthetic modifier and merge deferred into abstract
handle cyclic structures in API generation, display, comparison, persistence
gzip compile cache file
bump to 2.8.1.RC3, project definition cleanup
fix main method detection to check for the right name
properly view inherited definitions
exclude constructors of ancestors
Reduce AnalysisCallback interface:
remove discovery
simplify dependency notification methods
Use map of classpath entry to Analysis for locating
source API for external dependencies
Handle classpath changes by locating class
on classpath and either locating Analysis/Source
as above or comparing Stamp. This requires storing
the class name of a binary dependency now.
Make this process aware of full classpath, including
boot classpath
* Added the top-level interface project for communicating across scala versions within a jvm.
* Added plugin project containing analysis compiler plugin
* Added component compiler to build xsbt components against required version of Scala on the fly
* Added interface to compiler that runs in the same version of Scala
* Added frontend that compiles against a given version of Scala with or without analysis.
Grzegorz Kossakowski
Benjy
Mark Harrah
Paolo G. Giarrusso
Eugene Vigdorchik
Jason Zaugg