**Problem**
Intermittent java.nio.file.FileAlreadyExistsException when publishing
classes.sbtdir.zip during action-cache packaging under parallel tasks
(#9043). Copying from a temp directory straight into the final path races
on the fixed destination name.
**Solution**
Stage the built zip next to the destination with a unique temp file, then
replace the final path via Files.move with REPLACE_EXISTING and ATOMIC_MOVE,
falling back to a non-atomic move when needed. Add a concurrent
packageDirectory test.
Closes#9043
Generated-by: Cursor (AI-assisted)
**Problem**
When a file referenced during cache serialization is deleted between task
execution and cache storage, sbt crashes with an uncaught NoSuchFileException.
The IOException catch added in #8699 only wraps store.put(), but
Converter.toJsonUnsafe(result) and mkInput() also call
hashedVirtualFileRefToStr which calls Files.readAttributes — and these are
outside the try-catch.
**Solution**
Move the try/catch IOException to wrap the entire post-action cache storage
section, so NoSuchFileException from result serialization or cache key
computation is caught and handled gracefully (skip caching, return result).
Fixes#9044
Co-authored-by: bittoby <218712309+bittoby@users.noreply.github.co>
* [2.x] feat: Add cacheVersion setting for global cache invalidation
**Problem**
There was no escape hatch to invalidate all task caches when needed.
**Solution**
Add `Global / cacheVersion` setting that incorporates into the cache key
hash. Changing it invalidates all caches. Defaults to reading system
property `sbt.cacheversion`, or else 0L. When 0L, the hash is identical
to the previous behavior (backward compatible).
Fixes#8992
* [2.x] refactor: Simplify BuildWideCacheConfiguration and add cacheVersion test
- Replace auxiliary constructors with default parameter values
- Add unit test verifying cacheVersion invalidates the cache
* [2.x] fix: Restore auxiliary constructors for binary compatibility
* [2.x] test: Improve cacheVersion scripted test and add release note
- Scripted test now verifies cache invalidation via a counter
that increments only when the task body actually executes
- Add release note documenting the cacheVersion setting
**Problem**
When the remote cache server (e.g. bazel-remote using S3 for storage) reports an AC (Action Cache)
hit but the underlying CAS (Content Addressable Storage) blob is missing or
corrupt, ActionCache.cache propagates the resulting exception (typically
java.io.FileNotFoundException) directly to the SBT task engine process with no interception of the propogated error.
This causes a build failure instead of a graceful cache miss.
The three unguarded call sites are:
1. organicTask - syncBlobs after a successful put only caught NoSuchFileException,
missing FileNotFoundException and other IO errors.
2. getWithFailure / readFromSymlink fast-path - syncBlobs inside flatMap with no
exception handling.
3. getWithFailure main branch - both syncBlobs calls and the subsequent IO.read
were completely unguarded.
**Solution**
Guard all three call sites with NonFatal catches:
- Cache read failures (getWithFailure) return Left(None) which the caller
interprets as a cache miss, triggering organic recomputation.
- Cache write failures (organicTask) are demoted to a debug-level log; the task
result that was already computed is returned successfully.
Two regression tests are added to ActionCacheTest:
1. Tests the main getWithFailure branch using the default relative-path converter.
2. Tests the readFromSymlink fast-path using an absolute-path converter so the
symlink created on the first run is found by Files.isSymbolicLink on the second.
Catch parse errors when reading cached values and return Left(None)
so the task is re-run instead of failing with IncompleteParseException.
Fixes plugins/dotty-sandwich scripted test flake.
Fixes#8627
- Use ${OUT} placeholder and remove StringVirtualFile1 special-case handling
- Make dirZipExt check generic (use vf instead of svf: StringVirtualFile1)
- Let syncBlobs handle StringVirtualFile1 transparently via fileConverter.toPath()
Fixes#8631
**Changes:**
- Add `useIvy` setting key (defaults to `true`)
- Add `ivylessPublishLocalImpl` helper that publishes without Ivy
- Modify `publishLocal` to use ivyless publisher when `useIvy := false`
- Generate ivy.xml via `lmcoursier.IvyXml`
- Generate MD5/SHA-1 checksums for all files
- Add scripted test `dependency-management/ivyless-publish-local`
* test: Migrate FileInfoSpec.scala to verify.BasicTestSuite (#8542)
Migrate FileInfoSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'it should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Add explicit Unit return types for consistency
- Add 'end FileInfoSpec' marker
Fixes#8542
Migrate SingletonCacheSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Replace intercept[Exception] with scala.util.Try pattern
- Add 'end SingletonCacheSpec' and 'end ComplexType' markers
Related to the ongoing test migration effort.
Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
* always create symlinks to the cache in the target locations, even if the digest matches #8445
* create a test (currently failing even on #develop) that fails because if `zipPath` in `sbt.util.ActionCache.packageDirectory` is a symlink to the CAS, in later calls, this path in the CAS gets overridden by the new sources.
- in this test, after "run 1" in line 15, the produced file "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip" is a symlink to the CAS, let's call it SH1.
- when "run 3" is executed, `IO.zip` saves the new value to `zipPath`, which is "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip -> SH1", so SH1 gets overridden.
- when the last "run 1" is executed, the cache retrieves SH1, but it contains the data from "run 3" (the test fails with "actual A.x is 3").
* when packaging a directory into a zip, use a temp directory to avoid overwriting the cache #8461
When a compilation fails with CompileFailed, the failure is now cached
so that subsequent builds with the same inputs don't re-run the failed
compilation. This significantly improves the experience when using BSP
clients like Metals that may trigger many compilations in a row.
The implementation:
- Adds CachedCompileFailure, CachedProblem, and CachedPosition types
to serialize compilation failures
- Modifies ActionCache.cache to catch CompileFailed exceptions and
store them in the cache with exitCode=1
- On cache lookup, checks for cached failures first and re-throws
the cached exception if found
- Fixes DiskActionCacheStore.put to preserve exitCode from request
- Adds unit test to verify cached failure behavior
Fixes#7662
- Ensure output files are synced even when reading from symlink
- Call findActionResult to get ActionResult for side-effect files
- Maintains performance benefit while ensuring correctness
- Addresses @eed3si9n's concern about tasks generating files on the side
- Replace try-catch with Exception.nonFatalCatch.opt for cleaner code
- Follows Scala best practices for non-fatal exception handling
- More functional and idiomatic approach
- Avoids catching fatal exceptions like VirtualMachineError
- Use config.fileConverter.toPath() instead of string manipulation
- Avoid hardcoded '/' prefix removal
- More robust and maintainable approach
- Fix IO.read() to include UTF-8 charset parameter
- Check for symlinked value files before reading AC JSON
- When symlink exists and is valid, read directly from it
- Fallback to original AC file reading if symlink read fails
- Improves performance by avoiding unnecessary AC file reads
- All existing tests pass
**Problem**
sha256 is currently a bottleneck for no-op compilation.
**Solution**
This adds a local, in-memory cache of sha256 hashes of binary files
using their timestamp and file size.
The size of the digest cache can be configured using localDigestCacheByteSize,
which is set to 1MB by default.
This flips the default `:=` operation to the cached task.
To opt out of the cache, use `Def.uncached(...)` or
mark the key with `@cacheLevel(include = Array.empty)`
bitloi
BitToby
Dream
Idan Ben-Zvi
bitloi
eugene yokota
calm
NeedmeFordev
dive2tech
E.G
azdrojowa123
MkDev11
john0030710
Matt Dziuban
xuwei-k
Adrien Piquerez