When sbt was entered through xMain.run and the classloaders do not have
the expected format, sbt recreates the classloaders for itself.
Unfortunately the extra classpath was not added to the classloader. This
caused project/extra to fail if it was entered from RunFromSourceMain
rather than with the launcher.
The repo overrides scripted test relies on using the launcher to modify
the default resolvers. To support this, I extended the scripted launcher
to use the bundled sbt launcher if it is passed in via the
`-Dsbt.launch.jar` system property.
This commit makes it so that the scalaVersion, sbtVersion and classpath
are always passed in as parameters to any method that creates an sbt
server -- either for scripted or for the sbt server tests. By making
that change, I was able to change the implementation of scripted in the
sbt project to use publishLocalBin instead of publishLocal. This makes
the scripted tests start much faster (doc alone can easily take 30
second) with messing with the build to exclude slow tasks from
publishLocal.
As part of this change, I removed the test dependency on scriptedSbtRedux for
sbtProj and instead had scriptedSbtRedux depend on sbtProj. This allowed
me to remove some messy LocalProject logic in the resourceGenerators for
scriptedSbtReduxProj. I also had to remove a number of imports in the
scriptedSbtReduxProj because the definitions available in the sbt
package object became available.
I also removed the dependency on sbt-buildinfo and instead pass the
values from the build into test classes using scalatest properties. I
ran into a number of minor issues with the build info plugin, namely
that I couldn't get fullClasspathAsJars to reliably run as a BuildInfo
key. It also is somewhat more clear to me to just rely on the built in
scalatest functionality. The big drawback is that the scalatest
properties can only be strings, but that restriction isn't really a
problem here (strangely the TestData structure has a field configMap
which is effectively Map[String, Any] but Any is actually always String
given how the TestData is created as part of framework initialization.
Since scripted no longer publishes, scriptedUnpublished is now
effectively an alias for scripted.
To get publishLocalBin working, I had to copy private code from
IvyXml.scala into PublishBinPlugin. Once we publish a new version of
sbt, we can remove the copied code and invoke IvyXml.makeIvyXmlBefore
directly.
After the changes to `RunFromSourceMain` and the addition of
`Compile / exportJars` to the build, all of the tests pass with
RunFromSourceMain so I decided to remove the launcher based ones. The
RunFromSourceMain based tests are faster because they don't recompile
the various compiler bridges every time.
The main reason for having both the RunFromSourceMain and LauncherBased
scripted tests was that RunFromSourceMain would fail for any test that
ended up accessing the sbt.Package$ object. This commit fixes this bug
by reworking the classloader generated by RunFromSourceMain to invoke
sbt, switching from the classpath to jar classpath (by setting exportJars =
true) and entering sbt by calling `new xMain().run` rather than
`xMain.run`.
The reason for switching to the jar classpath is that the jvm seems to
have issues when there are two classes provided in different directories
that have the same case insensitive name, e.g. `sbt.package$` and
`sbt.Package$`. If those classes are instead provided in different
jars, the jvm seems to be able to handle it.
Exporting the jars is not enough though, I had to rework the
ClassLoader created in the launch method to have a layout that was
recognized by xMainConfiguration. I reimplemented the AppConfiguration
in java so that it could bootstrap itself in a single jar classloader
(the only needed jar is the Scripted.
If we export the jars in the build, then the NoClassDefErrors for
`sbt.Package$` go away during scripted tests using RunSourceFromMain.
This might make running tests in subprojects slightly slower but I think
its a worthy tradeoff.
In order for the sbt launcher to be able to resolve a local version of
sbt, we must publish the main jar, the sources jar, the doc jar, the pom
and an ivy.xml file. The publish and publishLocal tasks are wired in
IvyXml.scala to create an ivy.xml file before running publish. This
wasn't done with publishLocalBin which made it not work when no ivy.xml
file was already present (which was the case after running clean).
These started failing recently. I think it is because they may be using
insecure resolvers. Since these tests are unlikely to be testing widely
used functionality, we can disable them for now. If we start seeing
related issues, we can consider actually fixing them.
The swoval javafmt plugin uses the google java formatter (which I
believe is the only widely used java formatter) to format source files.
It does not provide an automatic javafmtOnCompile method like the
scalafmt plugin so I had to manually implement that functionality. In
general the java formatter is much faster than scalafmt so the impact of
having javafmtOnCompile set to true is very low.
I noticed that sbt does a _lot_ of no-op docs in the sbt project.
Through some debugging, I determined that this was because the target
directory last modified time of some of the dependent projects would
change between runs. I'm not really sure why that was happening but
instead of computing the last modified time of the directory, we should
be checking the last modified time of the directory contents.
After this change a no-op `doc` in the sbt project returns in less than
one second on my mac. Before, it was more like 25-60 seconds (the upper
bound is one runs `doc` because `sbtRoot/doc` takes about a minute).
Intellij has problems with older versions of scalafmt (see
https://github.com/scalameta/scalafmt/issues/1630). Not sure if this a
scalafmt issue or an intellij issue. I pinned the edition to October
2019 to avoid reformatting tons of files. Once development stabilizes,
we may wish to drop the edition but, for now, it's disruptive to change
the formatting.
Scripted automatically adding quotation marks when an argument contains
spaces breaks arguments that already were wrapped in quotations. The
scripted test should be responsible for its own escaping. Since this is
part of scripted-sbt-redux, we don't have to worry about downstream
builds relying on the old escaping behavior.
Stripping quotation marks makes it impossible to cleanly test certain
sbt features without resorting to weird hacks. For example:
> set Compile / scalacOptions += "-Xfatal-warnings"
did not work while
> set Compile / scalacOptions += '"-Xfatal-warnings"'
did.
I leave the single quote parser unchanged since single quotes are not
really used in sbt and so there is utility in leaving them as a way to
group arguments that should not be split apart.
This change should only affect the scripted tests in the sbt repo. We
can consider making stripQuotes = false the default for the plugin as
well.
either create test reports with legacy file names (legacyreport=true) or with standard file names (legacyreport=false or omitted) but not both as suggested in #4451
Ethan Atkins
eugene yokota
Dale Wijnand
Jannik Theiß