**Problem**
When the remote cache server (e.g. bazel-remote using S3 for storage) reports an AC (Action Cache)
hit but the underlying CAS (Content Addressable Storage) blob is missing or
corrupt, ActionCache.cache propagates the resulting exception (typically
java.io.FileNotFoundException) directly to the SBT task engine process with no interception of the propogated error.
This causes a build failure instead of a graceful cache miss.
The three unguarded call sites are:
1. organicTask - syncBlobs after a successful put only caught NoSuchFileException,
missing FileNotFoundException and other IO errors.
2. getWithFailure / readFromSymlink fast-path - syncBlobs inside flatMap with no
exception handling.
3. getWithFailure main branch - both syncBlobs calls and the subsequent IO.read
were completely unguarded.
**Solution**
Guard all three call sites with NonFatal catches:
- Cache read failures (getWithFailure) return Left(None) which the caller
interprets as a cache miss, triggering organic recomputation.
- Cache write failures (organicTask) are demoted to a debug-level log; the task
result that was already computed is returned successfully.
Two regression tests are added to ActionCacheTest:
1. Tests the main getWithFailure branch using the default relative-path converter.
2. Tests the readFromSymlink fast-path using an absolute-path converter so the
symlink created on the first run is found by Files.isSymbolicLink on the second.
Catch parse errors when reading cached values and return Left(None)
so the task is re-run instead of failing with IncompleteParseException.
Fixes plugins/dotty-sandwich scripted test flake.
Fixes#8627
- Use ${OUT} placeholder and remove StringVirtualFile1 special-case handling
- Make dirZipExt check generic (use vf instead of svf: StringVirtualFile1)
- Let syncBlobs handle StringVirtualFile1 transparently via fileConverter.toPath()
Fixes#8631
**Changes:**
- Add `useIvy` setting key (defaults to `true`)
- Add `ivylessPublishLocalImpl` helper that publishes without Ivy
- Modify `publishLocal` to use ivyless publisher when `useIvy := false`
- Generate ivy.xml via `lmcoursier.IvyXml`
- Generate MD5/SHA-1 checksums for all files
- Add scripted test `dependency-management/ivyless-publish-local`
* test: Migrate FileInfoSpec.scala to verify.BasicTestSuite (#8542)
Migrate FileInfoSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'it should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Add explicit Unit return types for consistency
- Add 'end FileInfoSpec' marker
Fixes#8542
Migrate SingletonCacheSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Replace intercept[Exception] with scala.util.Try pattern
- Add 'end SingletonCacheSpec' and 'end ComplexType' markers
Related to the ongoing test migration effort.
Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
* always create symlinks to the cache in the target locations, even if the digest matches #8445
* create a test (currently failing even on #develop) that fails because if `zipPath` in `sbt.util.ActionCache.packageDirectory` is a symlink to the CAS, in later calls, this path in the CAS gets overridden by the new sources.
- in this test, after "run 1" in line 15, the produced file "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip" is a symlink to the CAS, let's call it SH1.
- when "run 3" is executed, `IO.zip` saves the new value to `zipPath`, which is "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip -> SH1", so SH1 gets overridden.
- when the last "run 1" is executed, the cache retrieves SH1, but it contains the data from "run 3" (the test fails with "actual A.x is 3").
* when packaging a directory into a zip, use a temp directory to avoid overwriting the cache #8461
When a compilation fails with CompileFailed, the failure is now cached
so that subsequent builds with the same inputs don't re-run the failed
compilation. This significantly improves the experience when using BSP
clients like Metals that may trigger many compilations in a row.
The implementation:
- Adds CachedCompileFailure, CachedProblem, and CachedPosition types
to serialize compilation failures
- Modifies ActionCache.cache to catch CompileFailed exceptions and
store them in the cache with exitCode=1
- On cache lookup, checks for cached failures first and re-throws
the cached exception if found
- Fixes DiskActionCacheStore.put to preserve exitCode from request
- Adds unit test to verify cached failure behavior
Fixes#7662
- Ensure output files are synced even when reading from symlink
- Call findActionResult to get ActionResult for side-effect files
- Maintains performance benefit while ensuring correctness
- Addresses @eed3si9n's concern about tasks generating files on the side
- Replace try-catch with Exception.nonFatalCatch.opt for cleaner code
- Follows Scala best practices for non-fatal exception handling
- More functional and idiomatic approach
- Avoids catching fatal exceptions like VirtualMachineError
- Use config.fileConverter.toPath() instead of string manipulation
- Avoid hardcoded '/' prefix removal
- More robust and maintainable approach
- Fix IO.read() to include UTF-8 charset parameter
- Check for symlinked value files before reading AC JSON
- When symlink exists and is valid, read directly from it
- Fallback to original AC file reading if symlink read fails
- Improves performance by avoiding unnecessary AC file reads
- All existing tests pass
**Problem**
sha256 is currently a bottleneck for no-op compilation.
**Solution**
This adds a local, in-memory cache of sha256 hashes of binary files
using their timestamp and file size.
The size of the digest cache can be configured using localDigestCacheByteSize,
which is set to 1MB by default.
This flips the default `:=` operation to the cached task.
To opt out of the cache, use `Def.uncached(...)` or
mark the key with `@cacheLevel(include = Array.empty)`
**Problem**
Disk cache currently uses symbolic links, which won't
work on Windows without the Administrator privileges or
Developer Mode.
**Solution**
This falls back to using file copy.
Idan Ben-Zvi
bitloi
eugene yokota
calm
NeedmeFordev
dive2tech
E.G
azdrojowa123
MkDev11
john0030710
Matt Dziuban
xuwei-k
Adrien Piquerez