The qualifier of the `.value` call may contain `DefTree`s (e.g.
vals, defs) or `Function` trees. When we snip them out of the
tree and graft them into a new context, we must also call
`changeOwner`, so that the symbol owner structure and the tree
structure are coherent.
Failure to do so resulted in a crash in the compiler backend.
Fixes#1150
This commit makes the code source compatible across Scala 2.10.3
and https://github.com/scala/scala/pull/3452, which is proposed
for inclusion in Scala 2.11.0-RC1.
We only strictly need the incremental compiler to build on Scala
2.11, as that is integrated into the IDE. But we gain valuable
insight into compiler regressions by building *all* of SBT with
2.11.
We only got there recently (the 0.13 branch of SBT now fully cross
compiles with 2.10.3 and 2.11.0-SNAPSHOT), and this aims to keep
things that way.
Once 2.10 support is dropped, SBT macros will be able to exploit
the new reflection APIs in 2.11 to avoid the need for casting
to compiler internals, which aren't governed by binary compatibility.
This has been prototyped by @xeno-by: https://github.com/sbt/sbt/pull/1121
Since the fix for SI-2066, Scala 2.11 calls logicallyEnclosingMember on the
`x` in the expansion of the task macro:
InitializeInstance.app[[T0[x]](T0[java.io.File], T0[java.io.File]), Seq[java.io.File]]
This exposed the fact that SBT has created `T0` with `NoSymbol` as
the owner. This led to the a SOE.
I will also change the compiler to be more tolerant of this, but we
can observe good discipline in the macro and pick a sensible owner.
Still TODO for auto-plugins/logic:
* property-based tests for logic system
* user documentation
* (optional) 'about plugins' or similar to show more information about the auto-plugins for a project
* (deferred) allow AutoPlugin to inject Commands directly?
* (deferred) provide AutoPlugin functionality to arbitrary scopes instead of just at the Project level?
This is an attempt to provide a decent error message in some cases. However, paths that include
the Java path separator character are just fundamentally problematic and aren't always going to
be cleanly detected.
This requires a Format[T] to be implicitly available at the call site and requires the task
to be referenced statically (not in a settingDyn call). References to previous task values
in the form of a ScopedKey[Task[T]] + Format[T] are collected at setting load time in the
'references' setting. These are used to know which tasks should be persisted (the ScopedKey)
and how to persist them (the Format).
When checking/delegating previous references, rules are slightly different.
A normal reference from a task t in scope s cannot refer to t in s unless
there is an earlier definition of t in s. However, a previous reference
does not have this restriction. This commit modifies validateReferenced
to allow this.
TODO: user documentation
TODO: stable selection of the Format when there are multiple .previous calls on the same task
TODO: make it usable in InputTasks, specifically Parsers
The fix was made possible by the very helpful information provided by @retronym.
This commit does two key things:
1. changes the owner when splicing original trees into new trees
2. ensures the synthetic trees that get spliced into original trees do not need typechecking
Given this original source (from Defaults.scala):
...
lazy val sourceConfigPaths = Seq(
...
unmanagedSourceDirectories := Seq(scalaSource.value, javaSource.value),
...
)
...
After expansion of .value, this looks something like:
unmanagedSourceDirectories := Seq(
InputWrapper.wrapInit[File](scalaSource),
InputWrapper.wrapInit[File](javaSource)
)
where wrapInit is something like:
def wrapInit[T](a: Any): T
After expansion of := we have (approximately):
unmanagedSourceDirectories <<=
Instance.app( (scalaSource, javaSource) ) {
$p1: (File, File) =>
val $q4: File = $p1._1
val $q3: File = $p1._2
Seq($q3, $q4)
}
So,
a) `scalaSource` and `javaSource` are user trees that are spliced into a tuple constructor after being temporarily held in `InputWrapper.wrapInit`
b) the constructed tuple `(scalaSource, javaSource)` is passed as an argument to another method call (without going through a val or anything) and shouldn't need owner changing
c) the synthetic vals $q3 and $q4 need their owner properly set to the anonymous function
d) the references (Idents) $q3 and $q4 are spliced into the user tree `Seq(..., ...)` and their symbols need to be the Symbol for the referenced vals
e) generally, treeCopy needs to be used when substituting Trees in order to preserve attributes, like Types and Positions
changeOwner is called on the body `Seq($q3, $q4)` with the original owner sourceConfigPaths to be changed to the new anonymous function.
In this example, no owners are actually changed, but when the body contains vals or anonymous functions, they will.
An example of the compiler crash seen when the symbol of the references is not that of the vals:
symbol value $q3 does not exist in sbt.Defaults.sourceConfigPaths$lzycompute
at scala.reflect.internal.SymbolTable.abort(SymbolTable.scala:49)
at scala.tools.nsc.Global.abort(Global.scala:254)
at scala.tools.nsc.backend.icode.GenICode$ICodePhase.genLoadIdent$1(GenICode.scala:1038)
at scala.tools.nsc.backend.icode.GenICode$ICodePhase.scala$tools$nsc$backend$icode$GenICode$ICodePhase$$genLoad(GenICode.scala:1044)
at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1246)
at scala.tools.nsc.backend.icode.GenICode$ICodePhase$$anonfun$genLoadArguments$1.apply(GenICode.scala:1244)
...
Other problems with the synthetic tree when it is spliced under the original tree often result in type mismatches or some other compiler error that doesn't result in a crash.
If the owner is not changed correctly on the original tree that gets spliced under a synthetic tree, one way it can crash the compiler is:
java.lang.IllegalArgumentException: Could not find proxy for val $q23: java.io.File in List(value $q23, method apply, anonymous class $anonfun$globalCore$5, value globalCore, object Defaults, package sbt, package <root>) (currentOwner= value dir )
...
while compiling: /home/mark/code/sbt/main/src/main/scala/sbt/Defaults.scala
during phase: global=lambdalift, atPhase=constructors
...
last tree to typer: term $outer
symbol: value $outer (flags: <synthetic> <paramaccessor> <triedcooking> private[this])
symbol definition: private[this] val $outer: sbt.BuildCommon
tpe: <notype>
symbol owners: value $outer -> anonymous class $anonfun$87 -> value x$298 -> method derive -> class BuildCommon$class -> package sbt
context owners: value dir -> value globalCore -> object Defaults -> package sbt
...
The problem here is the difference between context owners and the proxy search chain.
As pointed out by @harrah in #705, both beginSource and endSource are
not used in sbt internally for anything meaningful.
We've discussed an option of deprecating those methods but since they
are not doing anything meaningful Mark prefers to have compile-time
error in case somebody implements or calls those methods. I agree with
that hence removal.
Equality is reference equality for classes that have lazy members
(currently Structure and ClassLike) and member equality for everything else.
This avoids the circularity issue due to lazy members.
Forces each class to be either abstract or final, to ensure that
the equals implementation is always correct.
Fixes toString to avoid infinite recursion.
* when mvn does a local 'install', it doesn't update the pom.xml last modified time if the pom.xml content hasn't changed
* an ivy.xml includes publicationDate, so an ivy.xml will always be touched even if the other content hasn't changed
* when Ivy checks if a snapshot is uptodate
+ it sees a SNAPSHOT, so it knows the module metadata and artifacts might change
+ it then checks the lastModified time of the metadata
+ if unchanged, it uses the cached information
+ if useOrigin is effectively false (either it is explicitly false or a resource is remote/isLocal=false),
this means that a new artifact won't be retrieved
* the Ivy IBiblioResolver
+ must be used for Maven repositories for proper behavior (no FileResolver, for example)
+ only returns URLResources, even for file: URLs
+ a FileResource is needed in combination with useOrigin to avoid copying artifacts from .m2/repository/
This commit fixes the above by setting a custom URLRepository on a constructed IBiblioResolver.
This URLRepository returns FileResources for file: URLs and standard URLResources for others.
The returned FileResource has isLocal=true and sbt sets useOrigin=true by default, so the artifacts
are used from the origin.
If it turns out a similar situation happens when mvn publishes to remote repositories, it is likely the fix for
that would be to figure out how to disable the lastModified check on the metadata and always download the metadata.
This would be slower, however.
The startup script should set sbt.cygwin=true if running from cygwin.
This will set the terminal type properly for JLine if not already set.
If sbt.cygwin=false or unset and os.name includes "windows", JAnsi is
downloaded by the launcher and installed on standard out/err.
The value for jline.terminal is transformed from explicit jline.X to
the basic types "windows", "unix", or "none". Now that sbt uses JLine
2.0, these types are understood by both sbt's JLine and Scala's.
Older Scala versions shaded the classes but not the terminal property
so both couldn't be configured with a class name at the same time.
Set sbt.task.timings=true to print timings for tasks.
This sample progress handler shows how to get names for tasks and
deal with flatMapped tasks. There are still some tasks that make
it through as anonymous, which needs to be investigated.
A setting to provide a custom handler should come in a subsequent commit.
Construction of Scala providers was already properly synchronized jvm and machine-wide.
The cache on top of construction was not and neither was the newer ClassLoaderCache.
This could cause the same Scala version to be loaded in multiple class loaders, taking
up more permgen space and possibly decreasing performance due to less effective jit.
The issue is very rare in practice for 0.13 because of the low probability of contention
on ClassLoaderCache. This is because the work for a cache miss is mainly the construction
of a URLClassLoader. In 0.12, however, the work potentially involved network access and
class loading (not just class loader construction), thus greatly increasing the probability
of contention and thus duplicate work (i.e. class loader construction).
When there is contention, multiple class loaders are constructed and then preserved by the
scalaInstance task in each project throughout the first task execution. Only when multiple
scalaInstance tasks execute simultaneously and only during the first execution does this occur.
(Technically, it could still happen later, but it doesn't in practice.)
This means that the number of duplicate class loaders should quickly saturate instead of growing
linearly with the number of projects. It also means that the impact depends on the exact
tree structure of projects. A linear chain of dependencies will be unaffected, but a build with
independent leaves may be limited by the number of cores. The number of cores affects
the number of threads typically used by the task engine, which limits the number of concurrently
executing scalaInstance tasks.
In summary, this might affect the first, cold compilation of a multi-module project with
independent leaves on a multi-core machine with Scala version different from the version used
for sbt. It might increase the maximum permgen requirements as well as slow the jit compilation
by up to one task execution. Subsequent compilations should be unaffected and the permgen
utilization return to be as expected.
Support a definitive flag for Failure that ignores later failures
instead of appending them. This is useful to override the default
behavior of listing the failures of alternative parsers.
Specifically, when the Scala version for sbt is the same as that for the project being built,
the jars in UpdateReport should be the same as those in ScalaProvider. This is because the
loader will come from the ScalaProvider, which uses jars in the boot directory instead of the
cache. The first part of the fix for #661 checks that loaded classes come from the classpath
and so they need to line up.
It is now used for consoleProject, run, and test. This loader verifies
that all classes loaded through it came from a particular classpath or
from the "root" loader. Root loader here is the launcher loader so that
those classes with native bindings come from that shared loader.
Needed an explicit type in PMap to workaround an error.
Need to drop tuple assignment of parser.parsed in input task macro as a workaround
for macro/resetAllAttrs/pattern matching/annotation issue in RC1.
1. Hyphenated labels are still accepted when parsing scoped keys (so 'sbt test-only' still works)
There is currently no timeline for removing this support for hyphenated keys.
2. Only camelCase is shown for tab completion.
3. AttributeKey.rawLabel provides the unnormalized label.
This should only be used to implement support for accepting hyphenated keys as input for compatibility.
4. AttributeKey.normLabel provides the normalized label (hyphenated converted to camelCase)
Similar to task macros, the parsed value is accessed by calling `parsed`
on a Parser[T], Initialize[Parser[T]], or Initialize[State => Parser[T]].
Values of tasks and settings may be accessed as usual via `value`.
1. Scala jars won't be copied to the boot directory, except for those needed to run sbt.
2. Scala SNAPSHOTs behave like normal SNAPSHOTs. In particular, running `update` will properly re-resolve the dynamic revision.
3. Scala jars are resolved using the same repositories and configuration as other dependencies.
4. Classloaders (currently, Scala classloaders) are cached by the timestamps of entries instead of Scala class loaders being cached by version.
TODO: Support external dependency configuration
That is, implement Initialize[Task[T]].flatten correctly.
This requires preserving the transformations applied in a scope so that
they can be applied to an Initialize value after static settings have been
evaluated.
* use normal TypeTree constructor
* remove unnecessary 'with Singleton' in macro utility
* integrate changes suggested by @xeno-by
* add refVar back and call asTypeConstructor instead of asType to refer to a type variable
1. KList[M[_]] now instead of KList[HL <: HList, M[_]]
a. head, tail work properly in this variant
b. disadvantage is that full type not easily transformed to new type constructor
2. AList abstracts on K[L[x]], a higher order type constructor.
A. Instances written for:
a. KList
b. Seq[M[T]] for a fixed T
c. TupleN
d. single values
e. operate on one type constructor when nested
B. Main disadvantage is type inference. It just doesn't happen for K[L[x]].
This is mitigated by AList being used internally and rarely needing to construct a K.
* split several source files
* move base settings sources (Scope, Structure, ...) into main/settings/
* breaks cycles. In particular, setting system moved from Project to Def
alias only parses the right hand side for tab completion help.
The assignment should happen whether or not the parse is successful because the
context may change by the time the alias is actually evaluated.
In particular, the 'set' command uses the loaded project for tab completion in 0.12.1.
When a .sbtrc file is processed, the project has not been loaded yet, so aliases
involving set fail. Wrapping the rhs in failOnException addresses this.
In the unsupported terminal mode, JLine treats a broken
stdin as an endless stream of empty lines. This is problematic
for idea-sbt-plugin: if the IntelliJ process is forcibly killed
and leaves the child SBT process running, it consumes considerable
CPU processing these.
Patching JLine itself would be the cleanest solution (the change
has already been applied to JLine 2), but I've shied away from that
and instead wrapped the InputStream that is read by JLine to
intercept the result of -1 from read(). When this happens, the
flat `inputEof` is set to true.
Josh Suereth
Jason Zaugg
Mark Harrah
Jacek Laskowski
Dan Sanduleac
Benjy
Bruno Bieth
Grzegorz Kossakowski
James Roper
cheeseng
Alex Dupre
Eugene Yokota
Paolo G. Giarrusso
Anthony
Eugene Vigdorchik