Add a resolvedScalacOptions task that resolves cache placeholders in scalacOptions to absolute machine paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
**Problem**
packageBin includes version into the file name, which ends up
invalidating the cache.
**Solution**
Define packageInternal, which does not include version in the file name, and used during Compile or Test/compile. Note that publishing and Runtime classpath would continue to use packageBin.
Both compileOptions sites mapped the whole classpath through
converter.toPath -> converter.toVirtualFile. Going through toPath
defeats FileConverter.toVirtualFile(VirtualFileRef), which already
returns the ref unchanged when it is a VirtualFile, so every entry was
rebuilt from scratch. For a class directory that means walking the
entire output tree, once per dependent.
backendOutput is converted explicitly because it is a settingKey
evaluated once at project load, so reusing it would pin the listing
taken before anything compiled.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
**Problem**
CrossVersion(module, scalaModuleInfo) is handed everything needed to name an
artifact, including ScalaModuleInfo.platform, and the name it returns is what
a caller publishes or resolves under. Nothing covers what it does with the
platform.
**Solution**
Name the artifact in full: platform suffix before cross suffix, matching the
coordinate (sbt/sbt#9117), through addPlatformSuffix, which already knows that
jvm contributes no suffix.
Generated-by: Claude Opus 5
**Problem/Solution**
This extends the idea started with TestRecap, and applies it to
both test success and failures.
1. Existing TestResultLogger trait is extended to handle the summary
rendering.
2. TestSummary enum is added to control the verbosity via
commandline option, system property, or a setting.
3. Script test captures the log.
Reuse the sibling dirzip when zinc reports it wrote nothing. The output
is still declared, so the stored ActionResult stays complete and a later hit
restores the class directory exactly as before -- it is the same
HashedVirtualFileRef the packaging path would have produced, read off disk
rather than rebuilt.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
**Problem**
1. build.sbt was compiled with console reporter,
so we couldn't capture the error.
2. Another problem was that reload deleted global log
so we couldn't get to the previous load failure.
**Solution**
1. This forwards build.sbt compilation errors to the logger.
2. This retains the failed loading log.
3. Using the facility above, this adds negative test for loading.
**Problem**
-java-home wasn't fully honored on Windows: sbtw rejected the single-dash form, silently ignored bad paths, and didn't propagate the JDK to the sbt/sbtn child; sbt.bat didn't fix up PATH/JAVACMD, and a project .java-version could override an explicit -java-home. Residual #963 gaps (the original javac-from-PATH bug was fixed long ago).
**Solution**
New SelectedJava seam in sbtw resolves the JDK once (--java-home > JAVACMD > JAVA_HOME > PATH) and applies one env overlay on every launch path; sbt.bat's handler now sets PATH/JAVACMD and a one-hop marker so .java-version defers to an explicit -java-home.
JDK_HOME > JAVA_HOME (the issue's "even more desired" ordering) is intentionally **not** implemented — it only mattered when JAVA_HOME pointed at a JRE without javac, and JDK 17+ (required by sbt 2.x) no longer ships standalone JREs, so JAVA_HOME always resolves to a full JDK.
The local analysis cache validates an entry against the analysis file's
timestamp and size, so a file re-created with the same content loses its
entry. That is exactly what happens to every analysis sbt writes: the
analysis file is a declared output of compileIncremental, so once the task
body has written it, syncFile deletes it and re-creates it as a symlink into
the CAS. Measured on a two-module build, the file came back four
milliseconds later at the same size under a new timestamp, and the read in
compileIncremental that follows deserialized the analysis it had just
written - one full deserialization per compiled module per compile, which is
the cost the cache exists to avoid. On this repo's own largest analysis
(519K) that read costs 19.5ms, against 0.098ms to hash the file and 0.0025ms
to stat it.
Populating the cache from set() is not enough on its own, and populating it
with the contents handed to set() is wrong: ConsistentAnalysisFormat does not
persist Compilations, so serving the in-memory analysis flips
CompileResult.hasModified from false to true and makes compileTask store the
analysis on every compile, including no-ops. set() now records what a read
of the file returns, and an entry whose size and content hash still match is
served under a new timestamp rather than discarded.
A read whose timestamp and size still match does not hash the file, so a warm
no-op compile costs one stat per read; a read that finds no entry to compare
against hashes the file it is about to deserialize, and set() hashes the file
it wrote, since the file it has to describe is replaced moments later.
Recording an entry cannot fail a compile that has already written its
analysis, so an IO error from stat-ing or hashing that file is dropped.
Across five compiles of two modules with two source edits, the
deserializations go from five to none: previousCompile, compileIncremental,
compileScalaBackend and the dependency analysis read in compileIncSetup are
all served from memory after the first write of each module's analysis.
Nothing a build can observe says whether a compile deserialized an analysis
it already had: an analysis served from the cache is indistinguishable from
one read back, so the unit tests can pin the store's behaviour but not the
compile's. The cache therefore counts what its reads cost - answered from
memory, hashed the file to answer, deserialized it - and a scripted test
reads those counts around a compile. The counts are sbt-private, so the test
reaches them from a helper declared in package sbt under project/.
Two invariants, one per failure mode. A compile with nothing to do must
answer every read from memory without even re-reading the file to hash it,
which pins the timestamp check; dropping it makes 7 of 7 reads hash. A
compile that recompiles must be served the analysis it just wrote, which pins
the fix; keeping develop's cache and adding only the counters deserializes 1
of 7.
Both depend on the compiles being real ones, because an analysis the action
cache served was never written and so there is nothing for the local cache to
have kept. The action cache is global, and its key does not depend on where
the build sits, so a second run of this test would otherwise recompile
nothing: the build points localCacheDirectory inside the sandbox - in Global,
which is the scope cacheStores resolves it in - and the test deletes target
first, so a run cannot inherit what an earlier one compiled.
Generated-by: claude-opus-5 (Claude Code)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncFile treated an output as up to date only when its digest matched and the
path was already a symlink. packageDirectory installs the archive with
Files.move, so on the write path it is always a regular file and the second
conjunct never held. Every compile therefore fell through to IO.delete +
writeFileAndNotify -> afterFileWrite -> unpackageDirZip, which inflated the
archive it had just written into a temp directory, digest-compared every file in
the output tree, found all of them unchanged, and deleted the temp directory.
The cost is proportional to output size and is paid on every compile of every
module. Measured at ~450 ms for a 2557-file, 33 MB classes directory (11 MB
archive), against a 4.2 s warm single-file edit.
A matching digest already means the local content is the blob's, whether or not
the path is a symlink, so take the up-to-date path in that case. Where the path
is not yet a symlink, still relink it to the CAS to keep deduplication, but
reach afterFileUpToDate rather than afterFileWrite. The symlink-creation
fallback is factored into linkOrCopy and shared with writeFileAndNotify.
For non-dirzip outputs both hooks are no-ops, and that branch already did a
delete-and-relink, so their behaviour is unchanged. The behaviour given up is
that a file inside the extracted directory which diverged from the manifest is
no longer silently restored while the archive digest still matches; on a cache
hit with a differing digest the full unpackage runs as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
**Problem**
Ivyless publishing doesn't work with sbt-pgp.
**Solution**
This refactors the ivyless publishing code and revives publishOrSkip, which is called by the plugin.
Instead of overriding publish task, implement proper PublisherInterface.
**Problem**
In theory, we shouldn't have to care about the timestamps,
but for local optimization it's still a useful key for caching.
Currently local Analysis cache gets invalidated after
syncBlob replaces the Analysis file with a symlink.
**Solution**
This preserves the original timestamp when copying files to
the local diskcache.
**Problem**
Serializing an UpdateReport re-reads every artifact through sjson-new's
File iso to recompute its sha256, so every ActionCache key computation
and Tracked write-back re-reads the whole dependency classpath on every
invocation, including no-ops.
**Solution**
Override the iso with a hash memoized by (path, size, lastModified),
with the same invalidation semantics as the digest cache in sbt/sbt#8363,
and mix it into the generated codec via contrabandCodecParents.
Generated-by: Claude Fable 5
**Problem**
Some custom LSP calls do not check initialize-handshake,
which over TCP includes token-based authentication.
**Solution**
This adds checkAuthenticated check around sbt/exec etc.
The thin client's blockUntilStart loop recursed while the portfile was
missing and the forked process looked valid, with no deadline: a forked
server that died before writing project/target/active.json, or stayed
alive but wedged, hung the client forever with no output. On Windows the
check ignored process death entirely (Properties.isWin short-circuited
the liveness test), making the hang unconditional there.
The wait is now bounded (default 5 minutes, tunable with
-Dsbt.client.boot.timeout.seconds). On expiry the client fails with a
"did not start within N seconds" message and then prints the server's
captured stderr, instead of hanging silently (#9484). The Windows
liveness workaround is preserved; the deadline is what bounds it.
Generated-by: kimi-code/k3 (Oh My Pi)
With usePipelining enabled, earlyOutputPing was completed only as a side
effect of zinc reporting progress mid-compile (CompileProgress.afterEarlyOutput
via writeEarlyOut / notifyNoEarlyOut in zinc-core). On an action-cache hit zinc
never runs, and on a failed compile it never reports, so the promise stayed
unfulfilled and every task waiting on it - compileEarly and makePickleProducts
across downstream pipelined projects - parked forever. A warm compile;compile
on a multi-module build hung indefinitely (#9486).
The compile task now completes the ping on every resolution path. Zinc's own
completion still wins when zinc ran (tryComplete is atomic, so it can never
pre-empt it); a cache hit completes it from the pickle jar's presence on disk
(absent jar means downstream falls back to a full compile, which is the correct
degradation); a failed compile completes it false so waiters take the full
compile path and propagate the failure instead of hanging.
Generated-by: kimi-code/k3 (Oh My Pi)
**Problem**
When test classloader holds on to the JAR file, Windows gets
AccessDefinedException on packageBin.
**Solution**
Flip the default to close the test class loader.
A client disconnecting with a terminal control query outstanding could park a
server thread forever, wedging prompts and command dispatch for every client
(#6841, #6840):
- VirtualTerminal.cancelRequests drained only 2 of the 8 pending terminal
maps, so waiters on the set-echo, raw-mode, attributes, and size queues were
never woken. It now drains all of them, with offer instead of put so the
shutdown path itself cannot block on a full queue.
- Raw-mode requests were registered in the set-echo map, so their waiters were
invisible to any raw-mode-specific handling.
- Closing a channel terminal did not wake readers parked on its input stream;
close now delivers EOF so a prompt blocked on a dead client's input unwinds.
- The failed-load prompt read its answer byte from System.in, which under
non-virtual IO is the process's own stdin and never carries client input; it
now reads the active terminal's input stream.
- ServerSessionImpl.close() could not deliver EOF to the peer while its read
thread was parked in a native read (the native close is never delivered), so
the server never noticed orderly client disconnects at all. It now shuts
down socket input first, which wakes the reader and lets the close through.
Regression test: a raw-protocol client that attaches, triggers the failed-load
prompt without answering the raw-mode query, and disconnects; the server must
shut down cleanly (EOF at the prompt maps to 'q') instead of staying parked
forever. Fails on develop with the server still alive and the command loop
parked in setRawMode; passes with this change. VirtualTerminalSpec pins the
drain across all eight maps and that other channels are untouched.
Generated-by: kimi-code/k3 (Oh My Pi)
**Problem**
`remoteCacheHeaders` entries are parsed in GrpcActionCacheStore.AuthCallCredentials
with h.split("="), which splits on every =. Java's split discards trailing empty
strings, so a Basic auth header such as authorization=Basic dXNlcjpwdw== produces
exactly two elements and matches List(k, v) with the base64 padding silently removed.
The truncated credential is rejected by the cache server with UNAUTHENTICATED.
**Solution**
Split on the first = only, keeping the remainder of the string verbatim as the header
value. The error case narrows to a header containing no = at all.
Generated-by: Claude Opus 5
**Problem**
Forked run baseDirectory was changed to current directory in sbt 2.0.4,
which on its own is fine, but it doesn't respect
Compile / run / baseDirectory.
**Solution**
This fixes that.
Resolution.projectCache is not a field: it maps projectCache0 into a
version-string-keyed copy on every call. SbtUpdateReport read it once per
dependency, and again per parent POM while assembling inherited licence info, so
for N modules resolved that is N rebuilds of an N-entry immutable map -- turning
a Resolution into an UpdateReport was quadratic in the modules it names.
On a 301-project build this was 61.9% of the CPU update spends, 83% of it
entering through lookupProject.
Read it once per report and reuse it, at every call site including the eviction
loop, which read it three times per conflict. The report produced is unchanged;
only the number of times the same map is built.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
One attached client answering the sbt/terminalpropertiesquery request badly, or
slower than 5 seconds, could freeze the whole server for every client:
- The response handler dropped malformed responses (response.foreach(buffer.put))
instead of falling back to a default like every sibling handler, so the updater
thread waiting on the queue timed out with the properties reference still null.
- getProperties(block = true) waits while properties is null, but nothing
completes it after the updater's one-shot 5-second poll times out: waiters woke
from the notify, saw null, and waited again with no updater outstanding. The
1-second lastUpdate throttle also let a caller start waiting with no query in
flight at all, and the wait condition was checked outside the pending monitor,
losing wakeups that fired between the check and the wait.
- A response arriving after the poll timeout was delivered into a queue that was
never deregistered, so it neither set properties nor woke anyone.
The threads that block here include the command loop iterating channels and the
fast-track thread handling attach and cancel, so one bad or briefly-stalled
client wedged prompts, Ctrl-C, and command dispatch server-wide until that
client disconnected.
The properties response handler now falls back to a default like its siblings;
the updater expires its query on timeout, deregistering it (rescuing a response
that raced in) and completing properties with the empty default so waiters
always make progress; and both wait sites hold the pending monitor and gate on
the query in flight. waitForPending gets the same treatment, since it seeds
lazy vals whose initialization otherwise parks every thread touching them.
Regression test: a raw-protocol session that answers every server request with
a result of the wrong shape attaches interactively; a well-behaved batch client
must then still be served twice. Fails on develop with a three-minute timeout
(the server is frozen), passes with this change. A unit spec pins the expiry
semantics, including the late-response rescue.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The projects of a build mostly depend on the same libraries, and each project's
report materializes its own copies of every coordinate it names, so one value
exists once per project that mentions it: on a 302-module monorepo the cached
reports hold 949,492 ModuleReports for 2,036 distinct values.
UpdateReportInterner adds weak pools for ConfigRef, InclExclRule, File,
Artifact, ModuleID, Caller and ModuleReport itself. Pooling the report is worth
more than pooling its parts alone, because sharing it also shares its licenses
vector, extraAttributes map, homepage string and artifacts vector. Reports
carrying a publicationDate are canonicalized but never pooled, since that
java.util.Calendar is mutable; everything else reachable from a ModuleReport is
immutable, so sharing is semantically invisible.
The pools are weak, so a value lives exactly as long as some report references
it and nothing accumulates across a reload.
Two sites cover a freshly resolved report. SbtUpdateReport interns each report
as it builds it rather than sweeping the finished one, so only the module under
construction is ever un-interned. That alone would not survive, though:
coursier memoizes moduleReport on a key that includes the dependees, so each
project builds its own instance of a shared coordinate, and transformDetails
then rebuilt every report to drop the callers -- discarding the sharing and
leaving a copy per module per configuration. Dropping the callers is what makes
those reports value-equal in the first place, so transformDetails now re-interns
what it rebuilds, and only rebuilds when there is something to drop. A scripted
test pins it: two projects resolving one coordinate must end up holding one
ModuleReport instance, on the fresh path and from the cache.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
WeakInterner is a ConcurrentHashMap keyed by a WeakReference subclass that
hashes and compares by its referent, with dead entries expunged on each
operation. Ported from zinc's sbt.internal.inc.WeakInterner, so lm-core gains
no interning dependency.
internWith derives the instance to pool only when the value is not pooled yet.
Equality is structural, so a value finds its pooled twin whatever the identity
of the values inside it, which lets a caller whose canonical form is itself
expensive to build skip building it on a hit. intern is internWith with
identity.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
azdrojowa123
eugene yokota
Mai Huy Hoàng
Albert Meltzer
BrianHotopp
kenji yoshida
Jozef Koval
Anatolii Kmetiuk
tanaka takaya
KilianSwissborg
Mai Huy Hoàng