Construction of Scala providers was already properly synchronized jvm and machine-wide.
The cache on top of construction was not and neither was the newer ClassLoaderCache.
This could cause the same Scala version to be loaded in multiple class loaders, taking
up more permgen space and possibly decreasing performance due to less effective jit.
The issue is very rare in practice for 0.13 because of the low probability of contention
on ClassLoaderCache. This is because the work for a cache miss is mainly the construction
of a URLClassLoader. In 0.12, however, the work potentially involved network access and
class loading (not just class loader construction), thus greatly increasing the probability
of contention and thus duplicate work (i.e. class loader construction).
When there is contention, multiple class loaders are constructed and then preserved by the
scalaInstance task in each project throughout the first task execution. Only when multiple
scalaInstance tasks execute simultaneously and only during the first execution does this occur.
(Technically, it could still happen later, but it doesn't in practice.)
This means that the number of duplicate class loaders should quickly saturate instead of growing
linearly with the number of projects. It also means that the impact depends on the exact
tree structure of projects. A linear chain of dependencies will be unaffected, but a build with
independent leaves may be limited by the number of cores. The number of cores affects
the number of threads typically used by the task engine, which limits the number of concurrently
executing scalaInstance tasks.
In summary, this might affect the first, cold compilation of a multi-module project with
independent leaves on a multi-core machine with Scala version different from the version used
for sbt. It might increase the maximum permgen requirements as well as slow the jit compilation
by up to one task execution. Subsequent compilations should be unaffected and the permgen
utilization return to be as expected.
Specifically, when the Scala version for sbt is the same as that for the project being built,
the jars in UpdateReport should be the same as those in ScalaProvider. This is because the
loader will come from the ScalaProvider, which uses jars in the boot directory instead of the
cache. The first part of the fix for #661 checks that loaded classes come from the classpath
and so they need to line up.
It is now used for consoleProject, run, and test. This loader verifies
that all classes loaded through it came from a particular classpath or
from the "root" loader. Root loader here is the launcher loader so that
those classes with native bindings come from that shared loader.
Mark Harrah
Grzegorz Kossakowski