Add a resolvedScalacOptions task that resolves cache placeholders in scalacOptions to absolute machine paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
**Problem**
packageBin includes version into the file name, which ends up
invalidating the cache.
**Solution**
Define packageInternal, which does not include version in the file name, and used during Compile or Test/compile. Note that publishing and Runtime classpath would continue to use packageBin.
Both compileOptions sites mapped the whole classpath through
converter.toPath -> converter.toVirtualFile. Going through toPath
defeats FileConverter.toVirtualFile(VirtualFileRef), which already
returns the ref unchanged when it is a VirtualFile, so every entry was
rebuilt from scratch. For a class directory that means walking the
entire output tree, once per dependent.
backendOutput is converted explicitly because it is a settingKey
evaluated once at project load, so reusing it would pin the listing
taken before anything compiled.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
**Problem**
CrossVersion(module, scalaModuleInfo) is handed everything needed to name an
artifact, including ScalaModuleInfo.platform, and the name it returns is what
a caller publishes or resolves under. Nothing covers what it does with the
platform.
**Solution**
Name the artifact in full: platform suffix before cross suffix, matching the
coordinate (sbt/sbt#9117), through addPlatformSuffix, which already knows that
jvm contributes no suffix.
Generated-by: Claude Opus 5
**Problem/Solution**
This extends the idea started with TestRecap, and applies it to
both test success and failures.
1. Existing TestResultLogger trait is extended to handle the summary
rendering.
2. TestSummary enum is added to control the verbosity via
commandline option, system property, or a setting.
3. Script test captures the log.
Reuse the sibling dirzip when zinc reports it wrote nothing. The output
is still declared, so the stored ActionResult stays complete and a later hit
restores the class directory exactly as before -- it is the same
HashedVirtualFileRef the packaging path would have produced, read off disk
rather than rebuilt.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
**Problem**
1. build.sbt was compiled with console reporter,
so we couldn't capture the error.
2. Another problem was that reload deleted global log
so we couldn't get to the previous load failure.
**Solution**
1. This forwards build.sbt compilation errors to the logger.
2. This retains the failed loading log.
3. Using the facility above, this adds negative test for loading.
The local analysis cache validates an entry against the analysis file's
timestamp and size, so a file re-created with the same content loses its
entry. That is exactly what happens to every analysis sbt writes: the
analysis file is a declared output of compileIncremental, so once the task
body has written it, syncFile deletes it and re-creates it as a symlink into
the CAS. Measured on a two-module build, the file came back four
milliseconds later at the same size under a new timestamp, and the read in
compileIncremental that follows deserialized the analysis it had just
written - one full deserialization per compiled module per compile, which is
the cost the cache exists to avoid. On this repo's own largest analysis
(519K) that read costs 19.5ms, against 0.098ms to hash the file and 0.0025ms
to stat it.
Populating the cache from set() is not enough on its own, and populating it
with the contents handed to set() is wrong: ConsistentAnalysisFormat does not
persist Compilations, so serving the in-memory analysis flips
CompileResult.hasModified from false to true and makes compileTask store the
analysis on every compile, including no-ops. set() now records what a read
of the file returns, and an entry whose size and content hash still match is
served under a new timestamp rather than discarded.
A read whose timestamp and size still match does not hash the file, so a warm
no-op compile costs one stat per read; a read that finds no entry to compare
against hashes the file it is about to deserialize, and set() hashes the file
it wrote, since the file it has to describe is replaced moments later.
Recording an entry cannot fail a compile that has already written its
analysis, so an IO error from stat-ing or hashing that file is dropped.
Across five compiles of two modules with two source edits, the
deserializations go from five to none: previousCompile, compileIncremental,
compileScalaBackend and the dependency analysis read in compileIncSetup are
all served from memory after the first write of each module's analysis.
Nothing a build can observe says whether a compile deserialized an analysis
it already had: an analysis served from the cache is indistinguishable from
one read back, so the unit tests can pin the store's behaviour but not the
compile's. The cache therefore counts what its reads cost - answered from
memory, hashed the file to answer, deserialized it - and a scripted test
reads those counts around a compile. The counts are sbt-private, so the test
reaches them from a helper declared in package sbt under project/.
Two invariants, one per failure mode. A compile with nothing to do must
answer every read from memory without even re-reading the file to hash it,
which pins the timestamp check; dropping it makes 7 of 7 reads hash. A
compile that recompiles must be served the analysis it just wrote, which pins
the fix; keeping develop's cache and adding only the counters deserializes 1
of 7.
Both depend on the compiles being real ones, because an analysis the action
cache served was never written and so there is nothing for the local cache to
have kept. The action cache is global, and its key does not depend on where
the build sits, so a second run of this test would otherwise recompile
nothing: the build points localCacheDirectory inside the sandbox - in Global,
which is the scope cacheStores resolves it in - and the test deletes target
first, so a run cannot inherit what an earlier one compiled.
Generated-by: claude-opus-5 (Claude Code)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
**Problem**
Ivyless publishing doesn't work with sbt-pgp.
**Solution**
This refactors the ivyless publishing code and revives publishOrSkip, which is called by the plugin.
Instead of overriding publish task, implement proper PublisherInterface.
**Problem**
Some custom LSP calls do not check initialize-handshake,
which over TCP includes token-based authentication.
**Solution**
This adds checkAuthenticated check around sbt/exec etc.
With usePipelining enabled, earlyOutputPing was completed only as a side
effect of zinc reporting progress mid-compile (CompileProgress.afterEarlyOutput
via writeEarlyOut / notifyNoEarlyOut in zinc-core). On an action-cache hit zinc
never runs, and on a failed compile it never reports, so the promise stayed
unfulfilled and every task waiting on it - compileEarly and makePickleProducts
across downstream pipelined projects - parked forever. A warm compile;compile
on a multi-module build hung indefinitely (#9486).
The compile task now completes the ping on every resolution path. Zinc's own
completion still wins when zinc ran (tryComplete is atomic, so it can never
pre-empt it); a cache hit completes it from the pickle jar's presence on disk
(absent jar means downstream falls back to a full compile, which is the correct
degradation); a failed compile completes it false so waiters take the full
compile path and propagate the failure instead of hanging.
Generated-by: kimi-code/k3 (Oh My Pi)
**Problem**
When test classloader holds on to the JAR file, Windows gets
AccessDefinedException on packageBin.
**Solution**
Flip the default to close the test class loader.
A client disconnecting with a terminal control query outstanding could park a
server thread forever, wedging prompts and command dispatch for every client
(#6841, #6840):
- VirtualTerminal.cancelRequests drained only 2 of the 8 pending terminal
maps, so waiters on the set-echo, raw-mode, attributes, and size queues were
never woken. It now drains all of them, with offer instead of put so the
shutdown path itself cannot block on a full queue.
- Raw-mode requests were registered in the set-echo map, so their waiters were
invisible to any raw-mode-specific handling.
- Closing a channel terminal did not wake readers parked on its input stream;
close now delivers EOF so a prompt blocked on a dead client's input unwinds.
- The failed-load prompt read its answer byte from System.in, which under
non-virtual IO is the process's own stdin and never carries client input; it
now reads the active terminal's input stream.
- ServerSessionImpl.close() could not deliver EOF to the peer while its read
thread was parked in a native read (the native close is never delivered), so
the server never noticed orderly client disconnects at all. It now shuts
down socket input first, which wakes the reader and lets the close through.
Regression test: a raw-protocol client that attaches, triggers the failed-load
prompt without answering the raw-mode query, and disconnects; the server must
shut down cleanly (EOF at the prompt maps to 'q') instead of staying parked
forever. Fails on develop with the server still alive and the command loop
parked in setRawMode; passes with this change. VirtualTerminalSpec pins the
drain across all eight maps and that other channels are untouched.
Generated-by: kimi-code/k3 (Oh My Pi)
**Problem**
Forked run baseDirectory was changed to current directory in sbt 2.0.4,
which on its own is fine, but it doesn't respect
Compile / run / baseDirectory.
**Solution**
This fixes that.
One attached client answering the sbt/terminalpropertiesquery request badly, or
slower than 5 seconds, could freeze the whole server for every client:
- The response handler dropped malformed responses (response.foreach(buffer.put))
instead of falling back to a default like every sibling handler, so the updater
thread waiting on the queue timed out with the properties reference still null.
- getProperties(block = true) waits while properties is null, but nothing
completes it after the updater's one-shot 5-second poll times out: waiters woke
from the notify, saw null, and waited again with no updater outstanding. The
1-second lastUpdate throttle also let a caller start waiting with no query in
flight at all, and the wait condition was checked outside the pending monitor,
losing wakeups that fired between the check and the wait.
- A response arriving after the poll timeout was delivered into a queue that was
never deregistered, so it neither set properties nor woke anyone.
The threads that block here include the command loop iterating channels and the
fast-track thread handling attach and cancel, so one bad or briefly-stalled
client wedged prompts, Ctrl-C, and command dispatch server-wide until that
client disconnected.
The properties response handler now falls back to a default like its siblings;
the updater expires its query on timeout, deregistering it (rescuing a response
that raced in) and completing properties with the empty default so waiters
always make progress; and both wait sites hold the pending monitor and gate on
the query in flight. waitForPending gets the same treatment, since it seeds
lazy vals whose initialization otherwise parks every thread touching them.
Regression test: a raw-protocol session that answers every server request with
a result of the wrong shape attaches interactively; a well-behaved batch client
must then still be served twice. Fails on develop with a three-minute timeout
(the server is frozen), passes with this change. A unit spec pins the expiry
semantics, including the late-response rescue.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The projects of a build mostly depend on the same libraries, and each project's
report materializes its own copies of every coordinate it names, so one value
exists once per project that mentions it: on a 302-module monorepo the cached
reports hold 949,492 ModuleReports for 2,036 distinct values.
UpdateReportInterner adds weak pools for ConfigRef, InclExclRule, File,
Artifact, ModuleID, Caller and ModuleReport itself. Pooling the report is worth
more than pooling its parts alone, because sharing it also shares its licenses
vector, extraAttributes map, homepage string and artifacts vector. Reports
carrying a publicationDate are canonicalized but never pooled, since that
java.util.Calendar is mutable; everything else reachable from a ModuleReport is
immutable, so sharing is semantically invisible.
The pools are weak, so a value lives exactly as long as some report references
it and nothing accumulates across a reload.
Two sites cover a freshly resolved report. SbtUpdateReport interns each report
as it builds it rather than sweeping the finished one, so only the module under
construction is ever un-interned. That alone would not survive, though:
coursier memoizes moduleReport on a key that includes the dependees, so each
project builds its own instance of a shared coordinate, and transformDetails
then rebuilt every report to drop the callers -- discarding the sharing and
leaving a copy per module per configuration. Dropping the callers is what makes
those reports value-equal in the first place, so transformDetails now re-interns
what it rebuilds, and only rebuilds when there is something to drop. A scripted
test pins it: two projects resolving one coordinate must end up holding one
ModuleReport instance, on the fresh path and from the cache.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
**Problem**
subproject build.sbt like a/build.sbt leaks to siblings.
**Solution**
Don't forward freshly computed common settings unless
the build.sbt is at root.
forked run used the project's baseDirectory as the working directory, while non-forked execution inherits sbt's own working directory — so toggling fork silently changed how relative paths resolved.
forked run (and forked console) now inherit sbt's working directory, consistent with non-forked execution and `sbtn` expectations.
Running reboot in the sbt shell dropped to the OS shell instead of rebooting.
The break was in teardown: Server.shutdown opened with
log.info, and during a client-initiated reboot the terminal in scope is that
client's already-closed virtual terminal, so the log write throws
ClosedChannelException through the terminal proxy. That aborted teardown
before the portfile was deleted and the server socket closed, and the
exception was swallowed by the shutdown hook (whose own error print goes to
the same dead terminal).
Server.shutdown now completes its state cleanup (portfile, tokenfile, running
flag, server socket) before logging, and CommandExchange.shutdown wraps each
channel shutdown and the server shutdown individually so one failing step
cannot skip the rest.
Fixes#9095
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
**Problem**
sbt 2.x currently constructs the test classloader using symlinks
to the JARs in CAS (content-addressable storage).
This causes issues because JVM apparently caches opened JAR files by path,
and since the symlink itself doesn't change it could end up
serving stale resource files.
**Solution**
We can workaround this issue by resolving the symlinks to the real path.
Problem:
When useIvy := false, ivyless publishing dropped the optional
scala_[scalaVersion]/ and sbt_[sbtVersion]/ path segments from Ivy-style
plugin publish patterns. This caused sbt 1 plugins published from sbt 2 builds
to land under non-plugin Ivy paths, so consumers looking under
scala_2.12/sbt_1.0/ could not resolve them.
Solution:
Read the plugin cross-version attributes from the CsrProject module and use
them when constructing ivyless Ivy-layout publish paths. Apply the same
substitution for local/file Ivy publishing and remote Ivy-style URL publishing,
while continuing to omit those optional segments when the attributes are absent.
Add scripted regressions for local and HTTP Ivy-style publishing of an sbt 1
plugin with useIvy := false.
A task that floods stdout could not be stopped promptly: cancelling it
(Ctrl+C from the thin client) stopped the task, but the server kept
draining the already-queued output to the client for several seconds,
because onCancellationRequest never discarded the pending backlog.
onCancellationRequest now sets a per-channel isCanceled flag and clears the
queued frames and buffered stdout (discardPending), and jsonRpcNotify
drops the cancelled task's systemOut/systemErr while the flag is set, so
output stops promptly.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
NetworkChannel batches writes to the client at most once per 20ms to
reduce terminal flicker and byte volume. forceFlush(), used to order
buffered stdout ahead of a control-plane message, called
flushExecutor.shutdownNow() -- the only place the executor was ever
shut down. Once it ran during active output it tore the executor
down (and, if a coalesced flush was pending, cancelled its future
without resetting flushFuture, leaving the slot stuck), so the 20ms
coalescing was gone for the rest of the connection: stdout was then
flushed per write via the inline fallback instead of batched. Output
still reaches the client, so the effect is extra flushes rather than
lost output, but shutting the shared executor down on the first
forceFlush is clearly unintended.
Extract the flush state machine into CoalescingFlusher. forceFlush now
drains immediately and leaves the timer live (a pending coalesced
drain harmlessly drains the remainder when it fires); the executor is
shut down once, at channel teardown, so it no longer leaks. doFlush
now holds a lock across both the drain and the publish so an inline
forceFlush and the timer's drain can't deliver two stdout batches out
of order.
Fixes#9415
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Any IOException while creating the boot io socket was wrapped in
ServerAlreadyBootingException and reported as "sbt thinks that server
is already booting" with a stack trace, and non-interactive
invocations exited with code 2. Permission or path-length problems
with XDG_RUNTIME_DIR or the temp directory and Windows named-pipe
access errors all hit this, blocking sbt entirely (#6777). Raw
IOExceptions from the constructor (socket directory creation) were
not caught at all and crashed startup.
getSocketOrExit now connects to the socket (BootServerSocketProbe,
shared with the test suite) to check for a live server before
believing the exception.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This applies a range of optimisations local to ClassStamper that bring the time needed for refinedTestDigests down to acceptable levels (see #9108):
* Cache the digests of transitive dependencies (big impact)
* Avoid sorting of digest subsets that would later get sorted again (small)
* Pre-compute which Analysis instances are required for each class to avoid repeated scanning of the whole list (medium)
* Merge two loops on relations.externalDeps into one (small)
* Compute the set of extra digests outside loop (small)
* Track the digest closure of each test via a BitSet (big)
**Problem**
Global plugin loading doesn't work.
**Solution**
1. Use ModuleID from to supply the location of global-plugin module.
2. Update pluginData with the global plugin classpath.
- Use Builders to avoid building intermediate collections
- Use a mutable.Set for alreadySeen
- Use plain Set instead of SortedSet
- Sorting only needs to happen at the end of the computation in transitiveStamp
Job ids are long. The job id parser currently uses NotSpace to parse the ids, which fails with a NumberFormatException for short non-numeric values and OOMs SBT for long non-numeric values.
**Problem**
Test is based on https://github.com/sbt/sbt/issues/9345#issuecomment-4718229113 which gives us the following sequence:
1. Metals sends buildTarget/compile.
2. sbt publishes real non-empty diagnostics.
3. Metals sends buildTarget/scalaMainClasses.
4. During that request, sbt emits build/publishDiagnostics with diagnostics: [] and reset: true.
5. The following build/taskFinish still reports errors: 1.
Previously, errors for diagnostics reporting via bsp were collected from a live compilation run. In the sequence above, that is triggered by buildTarget/compile. Then, buildTarget/scalaMainClasses does not trigger such a run for the second time, it uses the cached compilation result. Therefore, the diagnostics is not populated.
**Solution**
The proposed fix modifies sendFailureReport to accept an optional CompileFailed object that contains the diagnostics even in case the actual compiler did not run because the cached result was used. If no problems were found for a file via default means, this CompileFailed object is queried to see if it has any information about problems in a given file.
In a project with multiple main classes, the "multiple main classes detected" warning is unwanted noise when running tests. This message is already suppressed for explicit run commands, now also suppress it for test.
Building on #9270 (which parallelized resolution in non-interactive runs by
narrowing the lm-coursier lock to only fire while coursier renders its
interactive progress bar), this makes `update` resolve in parallel under the
interactive super shell as well.
Count distinct non-checksum urls instead and drop the module claim.
Report the elapsed time of the current burst so the super shell renders
a live counter rather than a frozen "0s". Tests now encode coursier's
per-session call pattern.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
azdrojowa123
eugene yokota
Mai Huy Hoàng
Albert Meltzer
kenji yoshida
Anatolii Kmetiuk
BrianHotopp
Mai Huy Hoàng
Jozef Koval
Yannick Heiber
Matt Dziuban
Merlin Hughes