For example, when the `--sourcepath` option is provided
and the refchecks phase compiles an annotation found
on a referenced symbol from the sourcepath.
`compileLate` assumes that all non-sentinel compiler
phases can be down cast to `GlobalPhase`.
This commit changes the two phases in SBT to extend
this instead of `Phase`. This has the knock on benefit
of simplifying the phases by letting the `GlobalPhase.run`
iterator over the list of compilation units and feed them
to us one by one.
I checked that the test case failed before making each
change.
If a method's type contains a non-primitive value class then it has two
signatures: one before erasure and one after erasure. Before this
commit, we checked if this was the case using `isAnyValSubtype`, but
this is too crude since primitive value classes are also subtypes of
`AnyVal` but do not change signature after erasure.
This commit replaces `isAnyValSubtype` by `isDerivedValueClass` which
excludes primitive value classes.
In practice, for an empty class, this reduces the size of the output of
`DefaultShowAPI` from 65 lines to 25 lines.
Before:
https://gist.github.com/smarter/cf1d6fe58efda88d6ee6#file-old-api
After:
https://gist.github.com/smarter/cf1d6fe58efda88d6ee6#file-new-api
Before this commit, we did not do the invalidation for methods with
multiple parameter list, the comment above `hasValueClassAsReturnType`
said:
Note: We only inspect the "outermost type" (i.e. no recursion) because
we don't need to inspect after erasure a function that would, for
instance, return a function that returns a subtype of AnyVal.
But this is wrong: a method with signature:
def foo(a: A)(b: B): C
is erased to:
def foo(a: A, b: B): C
and not, as the comment in the code suggest, to:
def foo(a: A): B => C
so we do need to inspect the final result type of methods, because they
can be value classes that will be erased to their underlying value.
Since pickled annotated types and symbols only mention
static annotations, whereas compilation from source
sees all annotations, we must explicitly filter annotations
in the API representation using the same criteria as the pickler,
so that we generate the same API when compiling from source
as when we're loading classfiles.
Also a bit more complete: handle SelectFromTypeTree,
consider the self type an inheritance dependency,
and flatten any refinement types in inherited types,
to get to the symbols of their parents, instead of the
useless symbol of the refinement class.
Include inheritance dependencies in regular ones
Also, update test to reflect the self type is now seen as an inheritance dependency.
self types are local, so don't treat them like inherited types
note inheritanceSymbols dealiases, where allSymbols is constructed differently
fix NPE in source-dependencies/macro-annotation
Specialize two implementations for each value of the `inherit` boolean argument.
Also use a more direct way of distinguishing declared and inherited members.
backwards compat for source-dependencies/inherited-dependencies
For refinement types, the Structure was already restricted
to declarations (and not inherited members), but all base types
were still included for a refinement's parents, which would
create unwieldy, and even erroneous (cyclic) types by expanding
all constituents of an intersection type to add all base types.
Since the logic already disregarded inherited members, it seems
logical to only include direct parents, and not all ancestor types.
```
class Dep {
def bla(c: Boolean) = if (c) new Value else "bla"
}
class Value extends java.lang.Comparable[Value] { def compareTo(that: Value): Int = 1 }
```
Motivated because we want to make it more robust & configurable.
Original motivation was to diagnose a cyclic type representation,
likely due to an f-bounded existential type, as illustrated by the following:
```
class Dep {
// The API representation for `bla`'s result type contains a cycle
// (an existential's type variable's bound is the existential type itself)
// This results in a stack overflow while showing the API diff.
// Note that the actual result type in the compiler is not cyclic
// (the f-bounded existential for Comparable is truncated)
def bla(c: Boolean) = if (c) new Value else "bla"
}
class Value extends java.lang.Comparable[Value] { def compareTo(that: Value): Int = 1 }
```
Limit nesting (`-Dsbt.inc.apidiff.depth=N`, where N defaults to `2`),
and number of declarations shown for a class/structural type
(via `sbt.inc.apidiff.decls`, which defaults to `0` -- no limit).
Limiting nesting is crucial in keeping the size of api diffs of large programs
within a reasonable amount of RAM...
For example, compiling the Scala library, the API diff with nesting at `4`
exhausts 4G of RAM...
The only aspect of the self variable that's relevant for
incremental compilation is its explicitly declared type,
and only when it's different from the type of the class that declares it.
Technically, any self type that's a super type of the class could be ignored,
as it cannot affect external use (instantiation/subclassing) of the class.
Call `initialize` in case symbol's `info` hadn't been completed
during normal compilation.
Also, normalize to the class symbol immediately.
Add a TODO regarding only looking at class symbols,
and thus ignoring the term symbol for objects,
as the corresponding class symbol has all the relevant info.
Mention that private members are being extracted and included in the api
structures but ignored in many other parts of incremental compiler. I've
made a mistake of assuming that private members are ignored at api
extraction time. This manifested itself as bug #2324.
The signatures of methods that have value classes as arguments or return
type change during the erasure phase. Because we only registered
signatures before the erasure, we missed some API changes when a class
was changed to a value class (or a value class changed to a class).
This commit fixes this problem by recording the signatures of method
before and after erasure.
Fixessbt/sbt#1171
In some cases the dependency extraction may encounter a null `TypeTree`
(eg. arguments of macro annotations that are untyped). In such cases,
we simply ignore the node.
Fixes#1593, #1655.
* Create a new sbt.compiler.javac package
* Create new interfaces to control running `javac` and `javadoc` whether forked or local.
* Ensure new interfaces make use of `xsbti.Reporter`.
* Create new method on `xsbti.compiler.JavaCompiler` which takes a `xsbti.Reporter`
* Create a new mechanism to parse (more accurately) Warnings + Errors, to distinguish the two.
* Ensure older xsbti.Compiler implementations still succeed via catcing NoSuchMethodError.
* Feed new toolchain through sbt.actions.Compiler API via dirty hackery until we can break things in sbt 1.0
* Added a set of unit tests for parsing errors from Javac/Javadoc
* Added a new integration test for hidden compilerReporter key, including testing threading of javac reports.
Fixes#875, Fixes#1542, Related #1178 could be looked into/cleaned up.
The fix for sbt/sbt#1237 was unfortunately not completely correct,
and infinite loops could still occur during the extraction of used
names.
In sbt/sbt#1544, a fix that was robuster and easier to understand
was applied to `/compile/interface/src/main/scala/xsbt/Dependency.scala`
in a similar situation (cyclic chains of original trees in macro
expansions).
This commit ports this fix to `ExtractUsedNames.scala`.
Closessbt/sbt#1640, sbt/sbt#1610.
In some cases, expanded macros report that their original tree and
its expansion are the same, thus creating a cyclic chain. This chain
may then produce a SOE during dependencies or used names extraction.
This kind of problem was already reported in sbt/sbt#1237 and
sbt/sbt#1408. Unfortunately, the fix that was applied to the
dependencies extraction part was not sufficient.
Mark test 'source-dependencies/macro' as passing
Fixes#1544
In Scala 2.10.4, this macro can produce a stack overflow :
def foo(a: Any): Any = macro impl
def impl(c: Context)(a: c.Expr[Any]): c.Expr[Any] = a
Here, an application such as `foo(someVal)` will produce the expansion
`someVal`. As expected, `someVal` has `original` tree `foo(someVal)`,
but if we inspect this tree, we will find that `someVal` has an
original tree, but it shouldn't.
Moreover, in Scala 2.11, some macros have their own application as
`original` trees.
See sbt/sbt#1237 for a description of these problems.
This commit fixes these two problems.
Fixessbt/sbt#1237
It has been reported in sbt/sbt#1237 that stack overflows may occur during the
extraction of used names (and later of dependencies between files). This
problem has been introduced by sbt/sbt#1163, which was about recording the
dependencies of macro arguments.
When a macro is expanded, the compiler attaches the tree before expansion to
the tree representing the expanded macro. As of Scala 2.11-RC3, some macros
have themselves attached as original tree, which caused the same macro to be
inspected over and over until a stack overflow.
This commit solves this problem by making sure that the original of a macro
expansion will be inspected if and only if it is different from the expanded
tree.
Fixessbt/sbt#1237
Macros take arguments as trees and return some other trees; both of
them have dependencies but we see trees only after expansion and
recorded only those dependencies.
This commit solves this problem by looking into the attachments of the
trees that are supposed to contain originals of macro expansions and
recording dependencies of the macro before its expansion.
Tracking of used names is a component needed by the name hashing
algorithm. The extraction and storage of used names is active only when
`AnalysisCallback.nameHashing` flag is enabled and it's disabled by
default.
This change constists of two parts:
1. Modification of Relations to include a new `names` relation
that allows us to track used names in Scala source files
2. Implementation of logic that extracts used names from Scala
compilation units (that correspond to Scala source files)
The first part is straightforward: add standard set of methods in
Relations (along with their implementation) and update the logic which
serializes and deserializes Relations.
The second part is implemented as tree walk that collects all symbols
associated with trees. For each symbol we extract a simple, decoded name
and add it to a set of extracted names. Check documentation of
`ExtractUsedNames` for discussion of implementation details.
The `ExtractUsedNames` comes with unit tests grouped in
`ExtractUsedNamesSpecification`. Check that class for details.
Given the fact that we fork while running tests in `compiler-interface`
subproject and tests are ran in parallel which involves allocating
multiple Scala compiler instances we had to bump the default memory limit.
This commit contains fixes for gkossakowski/sbt#3, gkossakowski/sbt#5 and
gkossakowski/sbt#6 issues.
Add documentation which explains how a general technique using implicits
conversions is employed in Compat class. Previously, it was hidden inside
of Compat class.
Also, I changed `toplevelClass` implementation to call
`sourceCompatibilityOnly` method that is designed for the purpose
of being a compatibility stub.
The scala/scala@2d4f0f1859 removes the
`toplevelClass` method. The recent change from
aac19fd02b introduces dependency on that
method. Combination of both changes makes incremental compiler incompatible
with Scala 2.11.
This change introduces a compatibility hack that brings back source
compatibility of incremental compiler with Scala 2.8, 2.9, 2.10 and 2.11.
The compatibility hack is making clever use implicit conversions that
can provide dummy method definitions for methods removed from Scala
compiler.
Also, the code that depends on `enclosingTopLevelClass` has been refactored
so the dependency is more centralized.
The previous name of the flag was rather specific: it indicated
whether the new source dependency tracking is supported by given Relations
object. However, there will be more functionality added to Relations that
is specific to name hashing algorithm. Therefore it makes sense to name
the flag as just `nameHashing`.
I decided to rename Relations implementation classes to be more
consistent with the name of the flag and with the purpose they serve.
The flag in AnalysisCallback (and classes implementing it) has been
renamed as well.
Adding source dependency on itself doesn't really bring any value so
there's no reason to do it. We avoided recording that kind of dependencies
by performing a check in `AnalysisCallback` implementation. However, if we
have another implementation like `TestCallback` used for testing we do
not benefit from that check.
Therefore, the check has been moved to dependency phase were dependencies
are collected.
Previously incremental compiler was extracting source code
dependencies by inspecting `CompilationUnit.depends` set. This set is
constructed by Scala compiler and it contains all symbols that given
compilation unit refers or even saw (in case of implicit search).
There are a few problems with this approach:
* The contract for `CompilationUnit.depend` is not clearly defined
in Scala compiler and there are no tests around it. Read: it's
not an official, maintained API.
* Improvements to incremental compiler require more context
information about given dependency. For example, we want to
distinguish between dependency on a class when you just select
members from it or inherit from it. The other example is that
we might want to know dependencies of a given class instead of
the whole compilation unit to make the invalidation logic more
precise.
That led to the idea of pushing dependency extracting logic to
incremental compiler side so it can evolve indepedently from Scala
compiler releases and can be refined as needed. We extract
dependencies of a compilation unit by walking a type-checked tree
and gathering symbols attached to them.
Specifically, the tree walk is implemented as a separate phase that
runs after pickler and extracts symbols from following tree nodes:
* `Import` so we can track dependencies on unused imports
* `Select` which is used for selecting all terms
* `Ident` used for referring to local terms, package-local terms
and top-level packages
* `TypeTree` which is used for referring to all types
Note that we do not extract just a single symbol assigned to `TypeTree`
node because it might represent a complex type that mentions
several symbols. We collect all those symbols by traversing the type
with CollectTypeTraverser. The implementation of the traverser is inspired
by `CollectTypeCollector` from Scala 2.10. The
`source-dependencies/typeref-only` test covers a scenario where the
dependency is introduced through a TypeRef only.
Jason Zaugg
Martin Duhem
Guillaume Martres
Adriaan Moors
Krzysztof Romanowski
Grzegorz Kossakowski
Pierre DAL-PRA
eugene yokota
Josh Suereth
Jean-Rémi Desjardins
Martin Duhem
Brian McKenna