2 independent, pre-existing retention bugs kept the classloader of a
finished in-process test run -- and the open jar handles it holds -- alive
for the rest of the server session.
- JUnitXmlTestsListener: testSuite is an InheritableThreadLocal, so
threads spawned during a run (e.g. async-framework pool workers) inherit
a copy of the suite reference that remove() cannot reach.
- ClassLoaderCache: loaders evicted from delegate by clearExpiredLoaders
are unreachable from the map and never enqueued on the ReferenceQueue, so
neither clear()/close() nor the cleanup thread could ever close them.
SuiteResult now documents the retention hazard on throwables. Adds
deterministic, cross-platform tests for all three severed chains.
**Problem**
Checksums are still generated for asc file.
1. localStaging is a file repo, which was not handled
2. It was checking Artifact name, not the file name
**Solution**
This fixes both.
putBlobsIfNeeded reads each blob's hash and size once, up front, and
returns only plain HashedVirtualFileRef values, so serializing an ActionResult
(disk, in-memory, or remote store) performs no file I/O and nothing re-stats a
blob after its CAS entry is written: a file vanishing once stored no longer
prevents the write, and I/O errors on an output file surface upfront at blob
storage time rather than mid-serialization.
Fixes#9349
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
The cached-task macro allocated one mutable slot per syntactic
Def.declareOutput / Def.declareOutputDirectory call site and snapshotted the
slots into the task's outputs after the body ran. A call inside a loop or .map
over a runtime-determined list is a single syntactic site executed many times,
so each iteration overwrote the same slot and only the last file was cached and
restored on a cache hit. There was also no way for a conditional call site that
did not execute to stay out of the outputs: its slot remained null.
Declared outputs now accumulate in a per-task ListBuffer: the macro emits one
buffer at the top of the cached body and rewrites each call site to
ActionCache.registerOutput(vf, buffer), which appends and returns the value.
Every execution registers, an unexecuted site contributes nothing, and the
static multi-site shape is unchanged.
Refs #9462 (the declareOutput-in-a-loop half)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
A directory declared via Def.declareOutputDirectory is packaged as a sibling
<dir>.sbtdir.zip, so deleting the directory leaves the zip behind. On a cache
hit, syncFile's up-to-date short-circuit saw the zip in sync (same digest,
already a CAS symlink) and returned without the unpack side effect, which only
ran from the file-write path: the directory was never restored. For sbt's own
compile, whose classes directory is declared this way, rm -rf of the classes
directory with a warm cache meant run failed with ClassNotFoundException and no
recompile; only deleting the zip as well (or the whole cache) recovered.
The up-to-date branch now re-extracts when the extracted directory itself is
missing: a single stat on the warm path, per review preference over a
manifest-based per-file check. Partial deletions inside a still-existing
directory are not repaired, consistent with treating target/ contents as
sbt-managed.
Refs #9462 (the directory-restoration half; the declareOutput-in-a-loop half is
a separate macro-layer issue)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
A genuinely broken restore now degrades to the onsite task instead of a
silent cache hit with incomplete outputs. Related to #9349.
Co-authored-by: sshevchenko <[email protected]>
The thin client (sbtn) parsed the launcher value flags (-java-home, -mem,
-jvm-debug, -sbt-dir, ...) but then dropped them. The space form was consumed
and discarded; the flag=value form fell through to the residual arguments and
was forwarded to the server verbatim, where --java-home=/path was rejected as a
command (`Not a valid command: --`).
When the client had to start a server (none running, no --server), the consumed
-java-home never reached the forked sbt launcher, so the server came up under
the default JVM and, in CI where the intended JDK is only reachable via
-java-home, failed to connect.
parseArgs now captures the consumed launcher value flags (both `flag value` and
`flag=value`) into Arguments.launcherValueArgs, and the cold-start fork re-passes
them to the sbt launcher so the server runs under the requested JVM. The client
tokenizes arguments by splitting on whitespace, which would otherwise fragment a
value that contains spaces (a Windows path like C:\Program Files\Java); parseArgs
tracks those split boundaries and rejoins a value flag's value. An empty flag=
value and a dangling flag with no value are consumed but not propagated, since
the launcher's require_arg would otherwise fail the fork.
The fork command construction is extracted into a pure, package-visible
serverCommand so a test can assert the propagated flag reaches the started
server. The sbt-launch-jar path is unchanged: it invokes java directly, with no
launcher to interpret the flag.
Fixes#9418
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Failure caching assumes a CompileFailed is a function of
the sources, which is true for source errors. But zinc also surfaces I/O write
failures ("error writing X.class") as compiler problems, so an environmental
failure (a concurrent target/ deletion, a permission blip) was cached under the
same mechanism and replayed from the global action cache on every later build,
even after the cause was gone. When the poisoned task is the metabuild compile
this is self-sustaining and unrecoverable from inside sbt: project loading
fails, so no task -- including clean -- can run, and only deleting the global
cache by hand recovers. The replayed diagnostics also name files/permissions
that no longer exist.
Refs #9455
Co-authored-by: BrianHotopp <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
**Problem**
sbt 2.x currently constructs the test classloader using symlinks
to the JARs in CAS (content-addressable storage).
This causes issues because JVM apparently caches opened JAR files by path,
and since the symlink itself doesn't change it could end up
serving stale resource files.
**Solution**
We can workaround this issue by resolving the symlinks to the real path.
For the purposes of matching resolved dependencies to reverse-dependency entries, do not use dependency overrides to check equality.
Co-authored-by: Anatolii Kmetiuk <[email protected]>
Problem:
When useIvy := false, ivyless publishing dropped the optional
scala_[scalaVersion]/ and sbt_[sbtVersion]/ path segments from Ivy-style
plugin publish patterns. This caused sbt 1 plugins published from sbt 2 builds
to land under non-plugin Ivy paths, so consumers looking under
scala_2.12/sbt_1.0/ could not resolve them.
Solution:
Read the plugin cross-version attributes from the CsrProject module and use
them when constructing ivyless Ivy-layout publish paths. Apply the same
substitution for local/file Ivy publishing and remote Ivy-style URL publishing,
while continuing to omit those optional segments when the attributes are absent.
Add scripted regressions for local and HTTP Ivy-style publishing of an sbt 1
plugin with useIvy := false.
A task that floods stdout could not be stopped promptly: cancelling it
(Ctrl+C from the thin client) stopped the task, but the server kept
draining the already-queued output to the client for several seconds,
because onCancellationRequest never discarded the pending backlog.
onCancellationRequest now sets a per-channel isCanceled flag and clears the
queued frames and buffered stdout (discardPending), and jsonRpcNotify
drops the cancelled task's systemOut/systemErr while the flag is set, so
output stops promptly.
Co-authored-by: Claude Fable 5 <[email protected]>
The byteStreamStub lazy val applied withDeadlineAfter once, baking in an
absolute deadline that was reused for the store's whole (session-long)
lifetime. remoteTimeoutInSec after the first blob transfer, every later
ByteStream read/write was rejected with DEADLINE_EXCEEDED, so the remote
cache could neither upload nor download blobs >chunkSizeBytes for the rest
of the sbt server's life.
Derive a fresh stub with the deadline per RPC so each call gets its own
relative timeout.
Co-authored-by: Yannick Heiber <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
NetworkChannel batches writes to the client at most once per 20ms to
reduce terminal flicker and byte volume. forceFlush(), used to order
buffered stdout ahead of a control-plane message, called
flushExecutor.shutdownNow() -- the only place the executor was ever
shut down. Once it ran during active output it tore the executor
down (and, if a coalesced flush was pending, cancelled its future
without resetting flushFuture, leaving the slot stuck), so the 20ms
coalescing was gone for the rest of the connection: stdout was then
flushed per write via the inline fallback instead of batched. Output
still reaches the client, so the effect is extra flushes rather than
lost output, but shutting the shared executor down on the first
forceFlush is clearly unintended.
Extract the flush state machine into CoalescingFlusher. forceFlush now
drains immediately and leaves the timer live (a pending coalesced
drain harmlessly drains the remainder when it fires); the executor is
shut down once, at channel teardown, so it no longer leaks. doFlush
now holds a lock across both the drain and the publish so an inline
forceFlush and the timer's drain can't deliver two stdout batches out
of order.
Fixes#9415
Co-authored-by: BrianHotopp <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
This applies a range of optimisations local to ClassStamper that bring the time needed for refinedTestDigests down to acceptable levels (see #9108):
* Cache the digests of transitive dependencies (big impact)
* Avoid sorting of digest subsets that would later get sorted again (small)
* Pre-compute which Analysis instances are required for each class to avoid repeated scanning of the whole list (medium)
* Merge two loops on relations.externalDeps into one (small)
* Compute the set of extra digests outside loop (small)
* Track the digest closure of each test via a BitSet (big)
**Problem**
sbt runner script defaults to sbtn, even on the operating system
where sbtn is not available.
**Solution**
This fallbacks to the jvm client.
**Problem**
Global plugin loading doesn't work.
**Solution**
1. Use ModuleID from to supply the location of global-plugin module.
2. Update pluginData with the global plugin classpath.
eugene yokota
Kevin Lee
BrianHotoppandClaude Opus 4.8
Stas Shevchenkoandsshevchenko
