Add `apiDiffContextSize` option to `IncOptions` which allows one
to control the size (in lines) of a context used when printing
diffs for textual API representation.
The default value for `apiDiffContextSize` is 5 which seems to be
enough for most situations. This is verified by many debugging
sessions I performed when using api diffing functionality.
Implement displaying API changes by using textual representation
of an API (ShowAPI) and good, old textual diff algorithm. We are
using java-diff-utils library that is distributed under Apache 2.0
license.
Notice that we have only soft dependency on java-diff-utils. It means
that we'll try to lookup java-diff-utils class through reflection
and fail gracefully if none is found on the classpath. This way
sbt is not getting any new dependency. If user needs to debug
api diffs then it's matter of starting sbt with
`-Dsbt.extraClasspath=path/to/diffutils.jar` option passed
to sbt launcher.
1. All parents of public/exported classes/modules/packages are tracked as
'publicInherited' dependencies. These are dealiased and normalized so
that the dependency is on the actual underlying template and not the
source enclosing the alias.
2. All CompilationUnit.depends dependencies are direct dependencies. These
include inherited dependencies.
3. When invalidating changed internal sources,
a. Invalidate all inherited dependencies, transitively and include the
originally modified sources,
b. Invalidate all direct dependencies of these sources,
c. Exclude any sources that were compiled in the previous step unless they
depend on a newly invalidated source.
4. Invalidate changed external sources in the same way as #3 but remove the
external sources from the final set.
Only public inheritance dependencies need to be considered because a template
that is not accessible outside its source file and that inherits from another
file can be handled as a normal, direct dependency. Because the template
isn't public, changes to its API will not propagate outside of the source
file.
Several existing tests cover the correctness, especially:
1. transitive-a covers direct, transitive dependencies with inferred return
types
2. transitive-b covers inherited, transitive dependencies with inferred return
types
There are two new tests, one that tests that public inherited dependencies are
tracked and one that verifies the basic invalidation progression.
More tests are needed to verify the improvements that this algorithm brings:
1. Inheritance-related dependencies are processed in one step to avoid the
otherwise unavoidable several steps.
2. Only immediate direct dependencies are ever processed, which should in many
typical cases avoid large invalidation sets.
In summary this commit:
* drops type normalization in api phase but keeps dealiasing
* fixes#736 and marks corresponding test as passing
I discussed type normalization with @adriaanm and according to him
sbt shouldn't call that method. The purpose of this method to
convert to a form that subtyping algorithm expects. Sbt doesn't need
to call it and it's fairly expensive in some cases.
Dropping type normalization also fixes#726 by not running into
stale cache in Scala compiler problem described in SI-7361.
The infrastructure for resident compilation still exists,
but the actual scalac-side code that was backported is removed.
Future work on using a resident scalac will use that invalidation
code directly from scalac anyway.
Ideally, we wouldn't need to construct the classpath ourselves and instead
reuse the classpath construction code from a compiler (scalac or javac).
However, we need to ensure that we only call the compiler when needed.
Because we need to construct the classpath even when compilation might
not happen, we have to duplicate classpath construction.
This reverts commit 9dd5f076ea which was a revert
itself so we are back to the state before those two reverts.
The problem that caused revert that happened in 9dd5f076ea
has been fixed in 88061dd262.
In `ClassToAPI` both `Private` and `Protected` vals had forward
reference to `Unqualified` so they would get `null` as a result.
Fixed that by rearranging the order of vals being declared.
This fixes a problem described in 9dd5f076ea.
Showing inherited members of a structure was disabled so we would not
run into cycles. To best of my knowledge, we can run into cycles only
if inherited member is `ClassLike`. We filter out those and let
anything else to be shown.
As described in sbt/sbt#676, arguments to `mergeMap` and `merge` methods
were swapped causing wrong structures being created for Java compiled
class files.
This commit fixessbt/sbt#676 and makes pending test to pass now.
That's foundation for API dumping and tests based on API representation contents.
Specific list of changes introduced by this commit:
* `AnalysisCallback` class takes `IncOptions` as argument so it
can determine if API should be minimized in `api` callback.
* Introduce `Incremental.apiDebug` method that determines if api debugging
is enabled. There are two ways to enable api debugging: through system
property and through incremental compiler options. The system property
method will be soon deprecated. We introduce it to make it easier to enable
API debugging until tools (like zinc and ide) catch up with making incremental
compiler configurable.
NOTE: The `apiDebug` method has been introduced in Incremental for two reasons:
1. It's analogous to `incDebug` method that's already there.
2. In other branch I need `apiDebug` to be defined in Incremental.
Once we deprecate and remove enabling debugging options through system properties
the code will be cleaned up.
Introduce a way to configure incremental compiler itself instead
of underlying Java/Scala compiler.
Specific list of changes in this commit:
* Add a method to `xsbti.compile.Setup` that returns incremental
compiler options as a `java.util.Map<String, String>`. We considered
statis interface instead of a `Map` but based on mailing
list feedback we decided that it's not the best way to go because
static interface is hard to evolve it by adding new options.
* Since passing `java.util.Map<String, String>` not very convenient
we convert it immediately to `sbt.inc.IncOptions`
* Add options argument to various methods/classes that implement
incremental compilation so in the end options reach
`sbt.inc.IncOptions` object
* Add `incOptions` task that allows users to configure incremental
compiler options in their build files. Default implementation of
that tasks returns just `IncOptions.DEFAULT`
* Both system property `xsbt.inc.debug` and `IncOptions.relationsDebug`
trigger debugging of relations now. In the near future, we should
deprecate use of `xsbt.inc.debug`.
Some methods take a lot of arguments and I'm about to add one more
which will make them too long for easy reading.
This change is changes code formatting only. That's done on purpose
to make it easier to review other changes.
Let's consider compile/inc/src/main/scala/sbt/CompileSetup.scala.
There are multiple Output types, and according to Eclipse importing
xsbti.compile.Output within the package sbt does not work because the
import is shadowed by sbt.Output.
However, compilation proceeds just fine within SBT. Reproducing the
example however gives the same warning, if the files are in the same
project. The problem here is probably that the shadowing Output
is declared in the same package but in another project, and that seems
to give different results in Eclipse and SBT, but relying on
that looks fragile.
Reading the spec is inconclusive since it doesn't match with Scalac's
behavior — see
https://groups.google.com/d/topic/scala-internals/-Rquc2HBYLk/discussion .
ForkTests has the same behavior as CompileSetup.
Resolution of https://issues.scala-lang.org/browse/SI-4695 seems to be to deprecate
inheriting from a class in a subpackage. This commit is an alternative solution,
possibly to be reverted or restricted if resolution of SI-4695 changes or if this
proves to be too conservative in practice.
Review by @gkossakowski. With separate inheritance/function call dependency tracking,
this probably should only pull in package objects with inheritance dependencies on
invalidated files.
While trying to determine binary dependencies sbt lookups class files
corresponding to symbols. It tried to do that for packages and most of the
time would fail because packages don't have corresponding class file
generated. However, in case of case insensitive file system, combined
with special nesting structure you could get spurious dependency.
See added test case for an example of such structure.
The remedy is to never even try to locate class files corresponding to
packages.
Fixes#620.
We store `Seq[xsbt.api.Compilation]` in `Analysis`. Compilation are
being appended to sequence for every iteration of the incremental
compiler.
Note that `Compilation`s are never removed from the sequence unless
you start from scratch with empty `Analysis`. You can do that by using
sbt's `clean` command.
The main use-case for using `compilations` field is to determine how
many iterations it took to compilen give code. The `Compilation` object
are also stored in `Source` objects so there's an indirect way to recover
information about files being recompiled in every iteration.
Since `Analysis` is persisted you can use this mechanism to track entire
sessions spanning multiple `compile` commands.
Extract the api after picklers, since that way we see the same symbol
information/structure irrespective of whether we were typechecking
from source / unpickling previously compiled classes.
Previously, the apiExtractor phase ran after typer.
Since this fix is hard to verify with a test (it's based on the
conceptual argument above, and anecdotal evidence of incremental
compilation of a big codebase), we're providing a way to restore the
old behaviour: run sbt with -Dsbt.api.phase=typer.
This fixes#609.
goal:
a representation of a type reference to a refinement class that's stable
across compilation runs (and thus insensitive to typing from source or
unpickling from bytecode)
problem:
the current representation, which corresponds to the owner chain of the
refinement:
1. is affected by pickling, so typing from source or using unpickled
symbols give different results (because the unpickler "localizes"
owners -- this could be fixed in the compiler in the long term)
2. can't distinguish multiple refinements in the same owner (this is
a limitation of SBT's internal representation and cannot be fixed in
the compiler)
solution:
expand the reference to the corresponding refinement type: doing that
recursively may not terminate, but we can deal with that by
approximating recursive references (all we care about is being sound for
recompilation: recompile iff a dependency changes, and this will happen
as long as we have one unrolling of the reference to the refinement)
* must start with val, lazy val, or def (no modifiers currently)
* visible only within the same .sbt file
* multiple definitions allowed without being separated by blank lines
* no blank lines allowed within a definition
Problems:
1. Without a message, users don't find 'last'
2. Showing a message for every error clutters output.
This tries to address these issues by:
1. Only showing the message when other feedback has not been provided and
'last' would not usually be helpful. This will require ongoing tweaking.
For now, all commands except 'compile' display the message. 'update' could
omit the message as well, but perhaps knowing about 'last' might be
useful there.
2. Including the exact command to show the output:
last test:compile
and not just
last <task>
3. Highlighting the command in blue for visibility as an experiment.
Review by @ijuma and @retronym, please.
Currently, only immediate parents classes are picked up for java classes.
This could be problematic, for example, in detecting Fingerprint for test frameworks.
So far, Scala types are not affected –– all the ancestors are available for them.
- Read macro modifier from method definition.
- Always recompile downstream files after a file containing macro defs is recompiled.
- Source is extended with a hasMacro attribute. Mark suggests that this might be better
tracked in Relations, but I'm not sure how to make that change.
While trying to build sbt without sbt, I got compile errors claiming that
util.Random would not be a member of util. The added import statements fixed
this.
The inserted spaces in the comments in project/Sbt.scala are a work around in a
bug in emacs scala syntax highlighting.
Scala compiler's way of handling empty classpath argument is problematic.
This workaround appends a dummy classpath argument when the classpath is
actually empty. Fixes#269 (also see #82, #85).
`docSetting` has been updated to do both Scaladoc and Javadoc. In
Scala/Java hybrid projects, the output docs are rebased to `scala`
or `java` sub-directory accordingly. But for pure scala or pure java
projects the subdirectories aren't added to becompliant with user
expectation as much as possible. We do hybrid mode iff both *.scala
and *.java files exist; other doc resources (package.html, *.jpg etc.)
don't influence the decision.
previously, for a directory on the classpath,
all files and directories under it would be checked
this caused unnecessary setting recompilation when
non-classfiles changed in the directory
One example was the sbt.plugins file, which doesn't affect
compilation, and caused projects with plugins to take longer
than necessary to start up
Mark Harrah
Grzegorz Kossakowski
xuwei-k
Jason Zaugg
Paolo G. Giarrusso
Adriaan Moors
Adriaan Moors
Lex Spoon
Benjy
nau
James Roper
Eugene Vigdorchik
Peter Vlugter
Paul Phillips
Indrajit Raychaudhuri
soc
Thomas Koch