A client disconnecting with a terminal control query outstanding could park a
server thread forever, wedging prompts and command dispatch for every client
(#6841, #6840):
- VirtualTerminal.cancelRequests drained only 2 of the 8 pending terminal
maps, so waiters on the set-echo, raw-mode, attributes, and size queues were
never woken. It now drains all of them, with offer instead of put so the
shutdown path itself cannot block on a full queue.
- Raw-mode requests were registered in the set-echo map, so their waiters were
invisible to any raw-mode-specific handling.
- Closing a channel terminal did not wake readers parked on its input stream;
close now delivers EOF so a prompt blocked on a dead client's input unwinds.
- The failed-load prompt read its answer byte from System.in, which under
non-virtual IO is the process's own stdin and never carries client input; it
now reads the active terminal's input stream.
- ServerSessionImpl.close() could not deliver EOF to the peer while its read
thread was parked in a native read (the native close is never delivered), so
the server never noticed orderly client disconnects at all. It now shuts
down socket input first, which wakes the reader and lets the close through.
Regression test: a raw-protocol client that attaches, triggers the failed-load
prompt without answering the raw-mode query, and disconnects; the server must
shut down cleanly (EOF at the prompt maps to 'q') instead of staying parked
forever. Fails on develop with the server still alive and the command loop
parked in setRawMode; passes with this change. VirtualTerminalSpec pins the
drain across all eight maps and that other channels are untouched.
Generated-by: kimi-code/k3 (Oh My Pi)
**Problem**
`remoteCacheHeaders` entries are parsed in GrpcActionCacheStore.AuthCallCredentials
with h.split("="), which splits on every =. Java's split discards trailing empty
strings, so a Basic auth header such as authorization=Basic dXNlcjpwdw== produces
exactly two elements and matches List(k, v) with the base64 padding silently removed.
The truncated credential is rejected by the cache server with UNAUTHENTICATED.
**Solution**
Split on the first = only, keeping the remainder of the string verbatim as the header
value. The error case narrows to a header containing no = at all.
Generated-by: Claude Opus 5
**Problem**
Forked run baseDirectory was changed to current directory in sbt 2.0.4,
which on its own is fine, but it doesn't respect
Compile / run / baseDirectory.
**Solution**
This fixes that.
Resolution.projectCache is not a field: it maps projectCache0 into a
version-string-keyed copy on every call. SbtUpdateReport read it once per
dependency, and again per parent POM while assembling inherited licence info, so
for N modules resolved that is N rebuilds of an N-entry immutable map -- turning
a Resolution into an UpdateReport was quadratic in the modules it names.
On a 301-project build this was 61.9% of the CPU update spends, 83% of it
entering through lookupProject.
Read it once per report and reuse it, at every call site including the eviction
loop, which read it three times per conflict. The report produced is unchanged;
only the number of times the same map is built.
Co-authored-by: Claude Opus 5 <[email protected]>
One attached client answering the sbt/terminalpropertiesquery request badly, or
slower than 5 seconds, could freeze the whole server for every client:
- The response handler dropped malformed responses (response.foreach(buffer.put))
instead of falling back to a default like every sibling handler, so the updater
thread waiting on the queue timed out with the properties reference still null.
- getProperties(block = true) waits while properties is null, but nothing
completes it after the updater's one-shot 5-second poll times out: waiters woke
from the notify, saw null, and waited again with no updater outstanding. The
1-second lastUpdate throttle also let a caller start waiting with no query in
flight at all, and the wait condition was checked outside the pending monitor,
losing wakeups that fired between the check and the wait.
- A response arriving after the poll timeout was delivered into a queue that was
never deregistered, so it neither set properties nor woke anyone.
The threads that block here include the command loop iterating channels and the
fast-track thread handling attach and cancel, so one bad or briefly-stalled
client wedged prompts, Ctrl-C, and command dispatch server-wide until that
client disconnected.
The properties response handler now falls back to a default like its siblings;
the updater expires its query on timeout, deregistering it (rescuing a response
that raced in) and completing properties with the empty default so waiters
always make progress; and both wait sites hold the pending monitor and gate on
the query in flight. waitForPending gets the same treatment, since it seeds
lazy vals whose initialization otherwise parks every thread touching them.
Regression test: a raw-protocol session that answers every server request with
a result of the wrong shape attaches interactively; a well-behaved batch client
must then still be served twice. Fails on develop with a three-minute timeout
(the server is frozen), passes with this change. A unit spec pins the expiry
semantics, including the late-response rescue.
Co-authored-by: Claude Fable 5 <[email protected]>
The projects of a build mostly depend on the same libraries, and each project's
report materializes its own copies of every coordinate it names, so one value
exists once per project that mentions it: on a 302-module monorepo the cached
reports hold 949,492 ModuleReports for 2,036 distinct values.
UpdateReportInterner adds weak pools for ConfigRef, InclExclRule, File,
Artifact, ModuleID, Caller and ModuleReport itself. Pooling the report is worth
more than pooling its parts alone, because sharing it also shares its licenses
vector, extraAttributes map, homepage string and artifacts vector. Reports
carrying a publicationDate are canonicalized but never pooled, since that
java.util.Calendar is mutable; everything else reachable from a ModuleReport is
immutable, so sharing is semantically invisible.
The pools are weak, so a value lives exactly as long as some report references
it and nothing accumulates across a reload.
Two sites cover a freshly resolved report. SbtUpdateReport interns each report
as it builds it rather than sweeping the finished one, so only the module under
construction is ever un-interned. That alone would not survive, though:
coursier memoizes moduleReport on a key that includes the dependees, so each
project builds its own instance of a shared coordinate, and transformDetails
then rebuilt every report to drop the callers -- discarding the sharing and
leaving a copy per module per configuration. Dropping the callers is what makes
those reports value-equal in the first place, so transformDetails now re-interns
what it rebuilds, and only rebuilds when there is something to drop. A scripted
test pins it: two projects resolving one coordinate must end up holding one
ModuleReport instance, on the fresh path and from the cache.
Co-Authored-By: Claude Opus 5 <[email protected]>
WeakInterner is a ConcurrentHashMap keyed by a WeakReference subclass that
hashes and compares by its referent, with dead entries expunged on each
operation. Ported from zinc's sbt.internal.inc.WeakInterner, so lm-core gains
no interning dependency.
internWith derives the instance to pool only when the value is not pooled yet.
Equality is structural, so a value finds its pooled twin whatever the identity
of the values inside it, which lets a caller whose canonical form is itself
expensive to build skip building it on a hit. intern is internWith with
identity.
Co-Authored-By: Claude Opus 5 <[email protected]>
sjsonnew serializes a File as a (uri, Long) pair whose Long is a SHA-256 of the
file's contents, and the read direction discards it. A report names each
artifact once per configuration it resolved in, and the projects of a build
largely share their dependencies, so writing the caches re-reads the whole
downloaded classpath many times over: on a 302-module monorepo, 755 GB of jars
and about 11 minutes of CPU for bytes no reader looks at.
Staleness comes from LibraryManagement.fileUptodate instead, which checks
File.exists and the modification time against UpdateReport.stamps.
UpdateReportPersistence.CacheCodec extends the LibraryManagementCodec trait and overrides
the inherited fileStringLongIso so the pair carries 0. That member is virtual,
so the override also reaches the Vector[(Artifact, File)] nested inside the
generated ModuleReportFormat, which a locally-scoped JsonFormat[File] could
not. The inputs store keeps the stock codec, so Tracked.inputChanged still
hashes contents for invalidation.
The JSON shape is unchanged, so caches stay readable in both directions.
Co-authored-by: Claude Opus 5 <[email protected]>
sbt.bat fails to start in client/server mode on Windows, falling back to running the full JVM in the foreground.
sbtn uses CreateProcess to spawn the server process. The original %%20 path encoding doesn't work in batch delayed expansion context, and the default install path C:\Program Files (x86) contains spaces that CreateProcess splits at.
**Problem**
There's a race condition between per-byte readSystemIn notification
and one-byte-read thread lifecycle.
Note: One-byte-read thread was introduced as a solution to the problem
that switching the terminal between raw and canonical mode cannot happen
if it's blocked by read.
**Solution**
This eliminates the thread lifecycle issue by keeping the thread alive
throughout the lifecycle of sbtn itself.
read is still called on demand by the server readSystemIn notification.
**Problem**
subproject build.sbt like a/build.sbt leaks to siblings.
**Solution**
Don't forward freshly computed common settings unless
the build.sbt is at root.
forked run used the project's baseDirectory as the working directory, while non-forked execution inherits sbt's own working directory — so toggling fork silently changed how relative paths resolved.
forked run (and forked console) now inherit sbt's working directory, consistent with non-forked execution and `sbtn` expectations.
Running reboot in the sbt shell dropped to the OS shell instead of rebooting.
The break was in teardown: Server.shutdown opened with
log.info, and during a client-initiated reboot the terminal in scope is that
client's already-closed virtual terminal, so the log write throws
ClosedChannelException through the terminal proxy. That aborted teardown
before the portfile was deleted and the server socket closed, and the
exception was swallowed by the shutdown hook (whose own error print goes to
the same dead terminal).
Server.shutdown now completes its state cleanup (portfile, tokenfile, running
flag, server socket) before logging, and CommandExchange.shutdown wraps each
channel shutdown and the server shutdown individually so one failing step
cannot skip the rest.
Fixes#9095
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
**Problem**
Checksums are still generated for asc file.
1. localStaging is a file repo, which was not handled
2. It was checking Artifact name, not the file name
**Solution**
This fixes both.
putBlobsIfNeeded reads each blob's hash and size once, up front, and
returns only plain HashedVirtualFileRef values, so serializing an ActionResult
(disk, in-memory, or remote store) performs no file I/O and nothing re-stats a
blob after its CAS entry is written: a file vanishing once stored no longer
prevents the write, and I/O errors on an output file surface upfront at blob
storage time rather than mid-serialization.
Fixes#9349
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
The cached-task macro allocated one mutable slot per syntactic
Def.declareOutput / Def.declareOutputDirectory call site and snapshotted the
slots into the task's outputs after the body ran. A call inside a loop or .map
over a runtime-determined list is a single syntactic site executed many times,
so each iteration overwrote the same slot and only the last file was cached and
restored on a cache hit. There was also no way for a conditional call site that
did not execute to stay out of the outputs: its slot remained null.
Declared outputs now accumulate in a per-task ListBuffer: the macro emits one
buffer at the top of the cached body and rewrites each call site to
ActionCache.registerOutput(vf, buffer), which appends and returns the value.
Every execution registers, an unexecuted site contributes nothing, and the
static multi-site shape is unchanged.
Refs #9462 (the declareOutput-in-a-loop half)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
A directory declared via Def.declareOutputDirectory is packaged as a sibling
<dir>.sbtdir.zip, so deleting the directory leaves the zip behind. On a cache
hit, syncFile's up-to-date short-circuit saw the zip in sync (same digest,
already a CAS symlink) and returned without the unpack side effect, which only
ran from the file-write path: the directory was never restored. For sbt's own
compile, whose classes directory is declared this way, rm -rf of the classes
directory with a warm cache meant run failed with ClassNotFoundException and no
recompile; only deleting the zip as well (or the whole cache) recovered.
The up-to-date branch now re-extracts when the extracted directory itself is
missing: a single stat on the warm path, per review preference over a
manifest-based per-file check. Partial deletions inside a still-existing
directory are not repaired, consistent with treating target/ contents as
sbt-managed.
Refs #9462 (the directory-restoration half; the declareOutput-in-a-loop half is
a separate macro-layer issue)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
Co-authored-by: eugene yokota <[email protected]>
Three independent, pre-existing retention bugs kept the classloader of a
finished in-process test run -- and the open jar handles it holds -- alive
for the rest of the server session.
- JUnitXmlTestsListener: testSuite is an InheritableThreadLocal, so
threads spawned during a run (e.g. async-framework pool workers) inherit
a copy of the suite reference that remove() cannot reach.
- TestRecap.collect: the recap is stashed on State.attributes and outlives
the command, so it must not retain live throwables.
- ClassLoaderCache: loaders evicted from delegate by clearExpiredLoaders
are unreachable from the map and never enqueued on the ReferenceQueue, so
neither clear()/close() nor the cleanup thread could ever close them.
SuiteResult now documents the retention hazard on throwables. Adds
deterministic, cross-platform tests for all three severed chains.
A genuinely broken restore now degrades to the onsite task instead of a
silent cache hit with incomplete outputs. Related to #9349.
Co-authored-by: sshevchenko <[email protected]>
Failure caching assumes a CompileFailed is a function of
the sources, which is true for source errors. But zinc also surfaces I/O write
failures ("error writing X.class") as compiler problems, so an environmental
failure (a concurrent target/ deletion, a permission blip) was cached under the
same mechanism and replayed from the global action cache on every later build,
even after the cause was gone. When the poisoned task is the metabuild compile
this is self-sustaining and unrecoverable from inside sbt: project loading
fails, so no task -- including clean -- can run, and only deleting the global
cache by hand recovers. The replayed diagnostics also name files/permissions
that no longer exist.
Refs #9455
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
**Problem**
sbt 2.x currently constructs the test classloader using symlinks
to the JARs in CAS (content-addressable storage).
This causes issues because JVM apparently caches opened JAR files by path,
and since the symlink itself doesn't change it could end up
serving stale resource files.
**Solution**
We can workaround this issue by resolving the symlinks to the real path.
The ivyless-publish-http and ivyless-publish-http-plugin scripted tests
defined publishToHttp as three unordered `.value` dependencies
(startPublishServer, publish, stopPublishServer). sbt evaluates task
dependencies before the task body in an unspecified order, so publish
could run before the HTTP server was listening and fail intermittently
with "Connection refused". It is reproducible under CPU contention (a
2-vCPU runner), which is why it began flaking on the JDK-25 shard and
turned develop red.
Use Def.sequential to guarantee start -> publish -> stop ordering, the
way the sibling ivyless-publish-maven-http test already sequences via
separate scripted commands.
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
kenji yoshida
BrianHotopp
KilianSwissborg
eugene yokota
Mai Huy HoàngandClaude Opus 5
Mai Huy HoàngandClaude Opus 5
Fabrizio Colonna
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Jozef Koval
Kevin Lee
Stas Shevchenkoandsshevchenko
Anatolii Kmetiuk