mirror of https://github.com/sbt/sbt.git
cad89d17a9
It can be quite slow to read and parse a large json file. Often, we are reading and writing the same file over and over even though it isn't actually changing. This is particularly noticeable with the UpdateReport*. To speed this up, I introduce a global cache that can be used to read values from a CacheStore. When using the cache, I've seen the time for the update task drop from about 200ms to about 1ms. This ends up being a 400ms time savings for test because update is called for both Compile / compile and Test / compile. The way that this works is that I add a new abstraction CacheStoreFactoryFactory, which is the most enterprise java thing I've ever written. We store a CacheStoreFactoryFactory in the sbt State. When we make Streams for the task, we make the Stream's cacheStoreFactory field using the CacheStoreFactoryFactory. The generated CacheStoreFactory may or may not refer to a global cache. The CacheStoreFactoryFactory may produce CacheStoreFactory instances that delegate to a Caffeine cache with a max size parameter that is specified in bytes by the fileCacheSize setting (which can also be set with -Dsbt.file.cache.size). The size of the cache entry is estimated by the size of the contents on disk. Since we are generally storing things in the cache that are serialized as json, I figure that this should be a reasonable estimate. I set the default max cache size to 128MB, which is plenty of space for the previous cache entries for most projects. If the size is set to 0, the CacheStoreFactoryFactory generates a regular DirectoryStoreFactory. To ensure that the cache accurately reflects the disk state of the previous cache (or other cache's using a CacheStore), the Caffeine cache stores the last modified time of the file whose contents it should represent. If there is a discrepancy in the last modified times (which would happen if, say, clean has been run), then the value is read from disk even if the value hasn't changed. * With the following build.sbt file, it takes roughly 200ms to read and parse the update report on my compute: libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.3" libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.1" This is because spark-sql has an enormous number of dependencies and the update report ends up being 3MB. |
||
|---|---|---|
| .github/ISSUE_TEMPLATE | ||
| core-macros/src/main/scala/sbt/internal/util/appmacro | ||
| internal | ||
| launch | ||
| licenses | ||
| main | ||
| main-actions/src | ||
| main-command/src | ||
| main-settings/src | ||
| notes | ||
| project | ||
| protocol/src/main | ||
| run | ||
| sbt/src | ||
| scripted-plugin/src/main/scala/sbt | ||
| scripted-sbt-old/src/main/scala/sbt/test | ||
| scripted-sbt-redux | ||
| src/main/conscript | ||
| tasks | ||
| tasks-standard | ||
| testing | ||
| vscode-sbt-scala | ||
| zinc-lm-integration/src | ||
| .appveyor.yml | ||
| .gitattributes | ||
| .gitignore | ||
| .java-version | ||
| .mailmap | ||
| .sbtopts | ||
| .scalafmt.conf | ||
| .travis.yml | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| NOTICE | ||
| PROFILING.md | ||
| README.md | ||
| SUPPORT.md | ||
| build.sbt | ||
| reset.sh | ||
| sbt-allsources.sh | ||
| server.md | ||
README.md
sbt
sbt is a build tool for Scala, Java, and more.
For general documentation, see http://www.scala-sbt.org/.
sbt 1.x
This is the 1.x series of sbt. The source code of sbt is split across several GitHub repositories, including this one.
- sbt/io hosts
sbt.iomodule. - sbt/util hosts a collection of internally used modules.
- sbt/librarymanagement hosts
sbt.librarymanagementmodule that wraps Ivy. - sbt/zinc hosts Zinc, an incremental compiler for Scala.
- sbt/sbt, this repository hosts modules that implements the build tool.
Other links
- Setup: Describes getting started with the latest binary release.
- FAQ: Explains how to get help and more.
- sbt/sbt-zero-seven: hosts sbt 0.7.7 and earlier versions
Issues and Pull Requests
Please read CONTRIBUTING carefully before opening a GitHub Issue.
The short version: try searching or asking on StackOverflow.
license
See LICENSE.