**Problem**
mkInput may fail for file locking reasons outside of the task engine
management.
**Solution**
This adds a simple retry. Note that we do not try to fall back
to organic task execution when mkInput fails.
**Problem**
The clean task no longer works as intended on sbt 2.x due to the disk cache.
**Solution**
This implements an in-memory state of invalidation for subproject and
configuration axes. So clean task would mark subprojects to be
invalidated regardless of the disk cache state.
The invalidation state would be lifted once any cached task executes,
so it's not technically same as sbt 1.x, but the UX should be similar.
* [2.x] test: Pin how a server writes its files
**Problem**
Nothing covers how the server writes the portfile and the token file.
The next commit changes both.
**Solution**
Assert what the server does now. The tests read the token file only.
The server rewrites it on every authentication, so a test can read it
while the server writes. The server writes the portfile once, as it
starts.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
* [2.x] fix: Write the portfile and zip by rename
**Problem**
A client reads the portfile as soon as it appears. The server wrote it
with IO.write, in place, so a client could read part of one and fail to
parse it. ActionCache staged and renamed its zip by hand.
**Solution**
The portfile calls IO.writeFileAtomically, and the cache zip calls
IO.copyFile, which stages and renames on its own. The token file needs
its staging file kept to the owner, which neither can do yet, so it
still writes its own way.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
* [2.x] sbt-io: upgrade to v1.13.1
* [2.x] fix: Write the token file by rename
**Problem**
writeTokenfile would first remove the existing token file, and then go
through a non-atomic sequence of steps to write a new one. If a client
attempts to read the file during that process, it is likely to find it
either missing or half-written.
**Solution**
Use newly released IO.writeFileAtomically with ownerOnly flag set.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Follow-up to #9643. The verification added there re-read every downloaded blob right after writing it, and it sat on top of an older cost: DiskActionCacheStore fully re-hashes CAS files on every findBlobs/syncBlobs/getBlobs call, even though the file name is the digest. On a cache hit that meant hashing a whole classpath worth of blobs just to conclude nothing changed.
- Remote blobs (grpc downloads and inline data) are hashed while the stream is written to a staged temp file, then moved into the CAS, so verification costs no extra I/O and tampered bytes never land under the CAS name. Local puts are copied without hashing and are not stamped, so a mismatched local file is still caught by the lookup hash.
- Verified entries are remembered per process by (digest, size, mtime, fileKey), so repeated lookups are a stat instead of a full hash. Any external change to a CAS file invalidates the stamp and forces a re-hash. This is the same trade-off CacheImplicits already makes for file stamps.
The cache now writes each value-distinct ModuleReport once into a "modules"
table and gives each configuration's details a list of indices into it. That is
~5x smaller and hands the reader instance sharing structurally, with nothing to
probe to rediscover it.
CacheStoreFactory.makeCompressed writes gzip-framed JSON and reads either
framing, sniffing the magic bytes so a cache written before a store was
switched to compression still loads. It defaults to make, so existing
CacheStoreFactory implementations are unaffected.
---------
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
A cache-restored tree is a farm of symlinks into the shared CAS. Writing it is
cheap, but walking it is not: resolving each link reads an inode out of a
directory far too large to stay in the OS metadata cache. Measured against a
57 GB / 952k-blob CAS, a cold stat of a CAS blob costs 165 us against 3.6 us for
a regular file, which makes a cold classpath walk of a restored tree roughly 38x
slower than one of real files. On an 81-module build with exportJars, a no-op
compile drops from 14.75s to 5.96s once the tree is real files.
DiskActionCacheStore now seeds the existing symlinkSupported latch from the
filesystem holding the CAS, and materializes entries with copyFile on APFS.
Files.copy reaches clonefile(2) there, so each entry gets its own inode and walks
at full speed while its data blocks stay shared with the CAS: the restored tree
measured 11 MB against 137 MB for the symlinked one. Copies also cannot corrupt
a blob when a task overwrites its output, which a symlink into the CAS can.
On every other filesystem the behavior is unchanged.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
sbt writes two archives per module per build: the packageBin /
packageInternal jar, and the zip the disk action cache stores an output
directory as. Both deflate one entry at a time on the calling thread.
io 1.13.0 adds IO.jarParallel / IO.zipParallel, which deflate entries
concurrently and produce byte-identical archives. Switch both call sites
over, taking the executor from IO.Implicits.zipContext -- io keeps it
behind that object so the choice is explicit rather than an ambient
ExecutionContext.
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
mkInput folded only a 32-bit murmur hash of the task input into the cache key, so distinct inputs collide at the birthday bound (~31 per 500k realistic inputs), silently resolving a task to the wrong cached output. A new DigestHasher hashes inputs into a full-width sha256 Merkle digest. Changes all cache keys, so the cache is repopulated once. Adds a regression test.
Blobs pulled from a gRPC remote cache were written to the CAS and synced into the build without re-hashing, so a compromised cache server or plaintext MITM could substitute bytes under the requested digest. Re-hash remote blobs on write and reject on mismatch, and warn when a plaintext grpc:// endpoint is configured.
Reuse the sibling dirzip when zinc reports it wrote nothing. The output
is still declared, so the stored ActionResult stays complete and a later hit
restores the class directory exactly as before -- it is the same
HashedVirtualFileRef the packaging path would have produced, read off disk
rather than rebuilt.
Co-authored-by: Claude Opus 5 <[email protected]>
syncFile treated an output as up to date only when its digest matched and the
path was already a symlink. packageDirectory installs the archive with
Files.move, so on the write path it is always a regular file and the second
conjunct never held. Every compile therefore fell through to IO.delete +
writeFileAndNotify -> afterFileWrite -> unpackageDirZip, which inflated the
archive it had just written into a temp directory, digest-compared every file in
the output tree, found all of them unchanged, and deleted the temp directory.
The cost is proportional to output size and is paid on every compile of every
module. Measured at ~450 ms for a 2557-file, 33 MB classes directory (11 MB
archive), against a 4.2 s warm single-file edit.
A matching digest already means the local content is the blob's, whether or not
the path is a symlink, so take the up-to-date path in that case. Where the path
is not yet a symlink, still relink it to the CAS to keep deduplication, but
reach afterFileUpToDate rather than afterFileWrite. The symlink-creation
fallback is factored into linkOrCopy and shared with writeFileAndNotify.
For non-dirzip outputs both hooks are no-ops, and that branch already did a
delete-and-relink, so their behaviour is unchanged. The behaviour given up is
that a file inside the extracted directory which diverged from the manifest is
no longer silently restored while the archive digest still matches; on a cache
hit with a differing digest the full unpackage runs as before.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
**Problem**
In theory, we shouldn't have to care about the timestamps,
but for local optimization it's still a useful key for caching.
Currently local Analysis cache gets invalidated after
syncBlob replaces the Analysis file with a symlink.
**Solution**
This preserves the original timestamp when copying files to
the local diskcache.
putBlobsIfNeeded reads each blob's hash and size once, up front, and
returns only plain HashedVirtualFileRef values, so serializing an ActionResult
(disk, in-memory, or remote store) performs no file I/O and nothing re-stats a
blob after its CAS entry is written: a file vanishing once stored no longer
prevents the write, and I/O errors on an output file surface upfront at blob
storage time rather than mid-serialization.
Fixes#9349
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
The cached-task macro allocated one mutable slot per syntactic
Def.declareOutput / Def.declareOutputDirectory call site and snapshotted the
slots into the task's outputs after the body ran. A call inside a loop or .map
over a runtime-determined list is a single syntactic site executed many times,
so each iteration overwrote the same slot and only the last file was cached and
restored on a cache hit. There was also no way for a conditional call site that
did not execute to stay out of the outputs: its slot remained null.
Declared outputs now accumulate in a per-task ListBuffer: the macro emits one
buffer at the top of the cached body and rewrites each call site to
ActionCache.registerOutput(vf, buffer), which appends and returns the value.
Every execution registers, an unexecuted site contributes nothing, and the
static multi-site shape is unchanged.
Refs #9462 (the declareOutput-in-a-loop half)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
A directory declared via Def.declareOutputDirectory is packaged as a sibling
<dir>.sbtdir.zip, so deleting the directory leaves the zip behind. On a cache
hit, syncFile's up-to-date short-circuit saw the zip in sync (same digest,
already a CAS symlink) and returned without the unpack side effect, which only
ran from the file-write path: the directory was never restored. For sbt's own
compile, whose classes directory is declared this way, rm -rf of the classes
directory with a warm cache meant run failed with ClassNotFoundException and no
recompile; only deleting the zip as well (or the whole cache) recovered.
The up-to-date branch now re-extracts when the extracted directory itself is
missing: a single stat on the warm path, per review preference over a
manifest-based per-file check. Partial deletions inside a still-existing
directory are not repaired, consistent with treating target/ contents as
sbt-managed.
Refs #9462 (the directory-restoration half; the declareOutput-in-a-loop half is
a separate macro-layer issue)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
A genuinely broken restore now degrades to the onsite task instead of a
silent cache hit with incomplete outputs. Related to #9349.
Co-authored-by: sshevchenko <[email protected]>
Failure caching assumes a CompileFailed is a function of
the sources, which is true for source errors. But zinc also surfaces I/O write
failures ("error writing X.class") as compiler problems, so an environmental
failure (a concurrent target/ deletion, a permission blip) was cached under the
same mechanism and replayed from the global action cache on every later build,
even after the cause was gone. When the poisoned task is the metabuild compile
this is self-sustaining and unrecoverable from inside sbt: project loading
fails, so no task -- including clean -- can run, and only deleting the global
cache by hand recovers. The replayed diagnostics also name files/permissions
that no longer exist.
Refs #9455
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
**Problem**
BatchUpdateBlobsRequest suffers from gRPC's message size limitation.
**Solution**
For larger files, we switch to using the ByteStream API, chunked to 1MB at a time.
When a file referenced by a task's inputs/outputs (e.g. Compile / resources +=
file("nope.txt")) does not exist, hashing the task's cache key threw a
NoSuchFileException deep inside sjsonnew serialization. It surfaced as an opaque
sjsonnew.SerializationException that dumped the entire input list, with the real
cause buried several `Caused by:` levels down, so users routinely mistook it for
a corrupt cache and reached for `clean`.
ActionCache.mkInput now catches the hashing failure, detects a NoSuchFileException
anywhere in the cause chain (ActionCache.findMissingFile), and throws a
MessageOnlyException naming the file:
[error] file referenced by the build does not exist: nope.txt
util-cache gains a dependency on util-control (a leaf module, no cycle) for
MessageOnlyException.
Fixes#9217.
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
**Problem**
We need some tooling to debug caching issues.
**Solution**
This adds an exeprimental execution log support,
which shows input and output of cached tasks.
Bumps io to released 1.12.0 which contains the atomic-write changes
to IO.copyFile, IO.transfer(InputStream, File), and IO.jar/zip.
ActionCacheStore.putBlob now uses IO.transfer instead of duplicating
staging logic.
**Problem**
Intermittent java.nio.file.FileAlreadyExistsException when publishing
classes.sbtdir.zip during action-cache packaging under parallel tasks
(#9043). Copying from a temp directory straight into the final path races
on the fixed destination name.
**Solution**
Stage the built zip next to the destination with a unique temp file, then
replace the final path via Files.move with REPLACE_EXISTING and ATOMIC_MOVE,
falling back to a non-atomic move when needed. Add a concurrent
packageDirectory test.
Closes#9043
Generated-by: Cursor (AI-assisted)
**Problem**
When a file referenced during cache serialization is deleted between task
execution and cache storage, sbt crashes with an uncaught NoSuchFileException.
The IOException catch added in #8699 only wraps store.put(), but
Converter.toJsonUnsafe(result) and mkInput() also call
hashedVirtualFileRefToStr which calls Files.readAttributes — and these are
outside the try-catch.
**Solution**
Move the try/catch IOException to wrap the entire post-action cache storage
section, so NoSuchFileException from result serialization or cache key
computation is caught and handled gracefully (skip caching, return result).
Fixes#9044
Co-authored-by: bittoby <[email protected]>
* [2.x] feat: Add cacheVersion setting for global cache invalidation
**Problem**
There was no escape hatch to invalidate all task caches when needed.
**Solution**
Add `Global / cacheVersion` setting that incorporates into the cache key
hash. Changing it invalidates all caches. Defaults to reading system
property `sbt.cacheversion`, or else 0L. When 0L, the hash is identical
to the previous behavior (backward compatible).
Fixes#8992
* [2.x] refactor: Simplify BuildWideCacheConfiguration and add cacheVersion test
- Replace auxiliary constructors with default parameter values
- Add unit test verifying cacheVersion invalidates the cache
* [2.x] fix: Restore auxiliary constructors for binary compatibility
* [2.x] test: Improve cacheVersion scripted test and add release note
- Scripted test now verifies cache invalidation via a counter
that increments only when the task body actually executes
- Add release note documenting the cacheVersion setting
**Problem**
When the remote cache server (e.g. bazel-remote using S3 for storage) reports an AC (Action Cache)
hit but the underlying CAS (Content Addressable Storage) blob is missing or
corrupt, ActionCache.cache propagates the resulting exception (typically
java.io.FileNotFoundException) directly to the SBT task engine process with no interception of the propogated error.
This causes a build failure instead of a graceful cache miss.
The three unguarded call sites are:
1. organicTask - syncBlobs after a successful put only caught NoSuchFileException,
missing FileNotFoundException and other IO errors.
2. getWithFailure / readFromSymlink fast-path - syncBlobs inside flatMap with no
exception handling.
3. getWithFailure main branch - both syncBlobs calls and the subsequent IO.read
were completely unguarded.
**Solution**
Guard all three call sites with NonFatal catches:
- Cache read failures (getWithFailure) return Left(None) which the caller
interprets as a cache miss, triggering organic recomputation.
- Cache write failures (organicTask) are demoted to a debug-level log; the task
result that was already computed is returned successfully.
Two regression tests are added to ActionCacheTest:
1. Tests the main getWithFailure branch using the default relative-path converter.
2. Tests the readFromSymlink fast-path using an absolute-path converter so the
symlink created on the first run is found by Files.isSymbolicLink on the second.
Catch parse errors when reading cached values and return Left(None)
so the task is re-run instead of failing with IncompleteParseException.
Fixes plugins/dotty-sandwich scripted test flake.
Fixes#8627
- Use ${OUT} placeholder and remove StringVirtualFile1 special-case handling
- Make dirZipExt check generic (use vf instead of svf: StringVirtualFile1)
- Let syncBlobs handle StringVirtualFile1 transparently via fileConverter.toPath()
Fixes#8631
**Changes:**
- Add `useIvy` setting key (defaults to `true`)
- Add `ivylessPublishLocalImpl` helper that publishes without Ivy
- Modify `publishLocal` to use ivyless publisher when `useIvy := false`
- Generate ivy.xml via `lmcoursier.IvyXml`
- Generate MD5/SHA-1 checksums for all files
- Add scripted test `dependency-management/ivyless-publish-local`
* test: Migrate FileInfoSpec.scala to verify.BasicTestSuite (#8542)
Migrate FileInfoSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'it should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Add explicit Unit return types for consistency
- Add 'end FileInfoSpec' marker
Fixes#8542
Migrate SingletonCacheSpec.scala from ScalaTest's AnyFlatSpec to
verify.BasicTestSuite, following the pattern established by other
test files in the util-cache module.
Changes:
- Replace AnyFlatSpec class with BasicTestSuite object
- Convert 'should ... in' syntax to 'test(...)' syntax
- Use Scala 3 syntax with colon indentation
- Change === to == for assertions (BasicTestSuite style)
- Replace intercept[Exception] with scala.util.Try pattern
- Add 'end SingletonCacheSpec' and 'end ComplexType' markers
Related to the ongoing test migration effort.
Co-authored-by: GlobalStar117 <[email protected]>
* always create symlinks to the cache in the target locations, even if the digest matches #8445
* create a test (currently failing even on #develop) that fails because if `zipPath` in `sbt.util.ActionCache.packageDirectory` is a symlink to the CAS, in later calls, this path in the CAS gets overridden by the new sources.
- in this test, after "run 1" in line 15, the produced file "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip" is a symlink to the CAS, let's call it SH1.
- when "run 3" is executed, `IO.zip` saves the new value to `zipPath`, which is "target/out/jvm/scala-3.7.4/a/classes.sbtdir.zip -> SH1", so SH1 gets overridden.
- when the last "run 1" is executed, the cache retrieves SH1, but it contains the data from "run 3" (the test fails with "actual A.x is 3").
* when packaging a directory into a zip, use a temp directory to avoid overwriting the cache #8461
When a compilation fails with CompileFailed, the failure is now cached
so that subsequent builds with the same inputs don't re-run the failed
compilation. This significantly improves the experience when using BSP
clients like Metals that may trigger many compilations in a row.
The implementation:
- Adds CachedCompileFailure, CachedProblem, and CachedPosition types
to serialize compilation failures
- Modifies ActionCache.cache to catch CompileFailed exceptions and
store them in the cache with exitCode=1
- On cache lookup, checks for cached failures first and re-throws
the cached exception if found
- Fixes DiskActionCacheStore.put to preserve exitCode from request
- Adds unit test to verify cached failure behavior
Fixes#7662
- Ensure output files are synced even when reading from symlink
- Call findActionResult to get ActionResult for side-effect files
- Maintains performance benefit while ensuring correctness
- Addresses @eed3si9n's concern about tasks generating files on the side
eugene yokota
Albert MeltzerandClaude Opus 5
Stas Shevchenko
Mai Huy HoàngandClaude Opus 5
kenji yoshida
BrianHotoppandClaude Opus 4.8
Anatolii Kmetiuk
bitloi
BitTobyandbittoby
Dream
Idan Ben-Zvi
bitloi
calm
NeedmeFordev
dive2tech
E.G
azdrojowa123
MkDev11
john0030710