The thin client's blockUntilStart loop recursed while the portfile was
missing and the forked process looked valid, with no deadline: a forked
server that died before writing project/target/active.json, or stayed
alive but wedged, hung the client forever with no output. On Windows the
check ignored process death entirely (Properties.isWin short-circuited
the liveness test), making the hang unconditional there.
The wait is now bounded (default 5 minutes, tunable with
-Dsbt.client.boot.timeout.seconds). On expiry the client fails with a
"did not start within N seconds" message and then prints the server's
captured stderr, instead of hanging silently (#9484). The Windows
liveness workaround is preserved; the deadline is what bounds it.
Generated-by: kimi-code/k3 (Oh My Pi)
With usePipelining enabled, earlyOutputPing was completed only as a side
effect of zinc reporting progress mid-compile (CompileProgress.afterEarlyOutput
via writeEarlyOut / notifyNoEarlyOut in zinc-core). On an action-cache hit zinc
never runs, and on a failed compile it never reports, so the promise stayed
unfulfilled and every task waiting on it - compileEarly and makePickleProducts
across downstream pipelined projects - parked forever. A warm compile;compile
on a multi-module build hung indefinitely (#9486).
The compile task now completes the ping on every resolution path. Zinc's own
completion still wins when zinc ran (tryComplete is atomic, so it can never
pre-empt it); a cache hit completes it from the pickle jar's presence on disk
(absent jar means downstream falls back to a full compile, which is the correct
degradation); a failed compile completes it false so waiters take the full
compile path and propagate the failure instead of hanging.
Generated-by: kimi-code/k3 (Oh My Pi)
A client disconnecting with a terminal control query outstanding could park a
server thread forever, wedging prompts and command dispatch for every client
(#6841, #6840):
- VirtualTerminal.cancelRequests drained only 2 of the 8 pending terminal
maps, so waiters on the set-echo, raw-mode, attributes, and size queues were
never woken. It now drains all of them, with offer instead of put so the
shutdown path itself cannot block on a full queue.
- Raw-mode requests were registered in the set-echo map, so their waiters were
invisible to any raw-mode-specific handling.
- Closing a channel terminal did not wake readers parked on its input stream;
close now delivers EOF so a prompt blocked on a dead client's input unwinds.
- The failed-load prompt read its answer byte from System.in, which under
non-virtual IO is the process's own stdin and never carries client input; it
now reads the active terminal's input stream.
- ServerSessionImpl.close() could not deliver EOF to the peer while its read
thread was parked in a native read (the native close is never delivered), so
the server never noticed orderly client disconnects at all. It now shuts
down socket input first, which wakes the reader and lets the close through.
Regression test: a raw-protocol client that attaches, triggers the failed-load
prompt without answering the raw-mode query, and disconnects; the server must
shut down cleanly (EOF at the prompt maps to 'q') instead of staying parked
forever. Fails on develop with the server still alive and the command loop
parked in setRawMode; passes with this change. VirtualTerminalSpec pins the
drain across all eight maps and that other channels are untouched.
Generated-by: kimi-code/k3 (Oh My Pi)
One attached client answering the sbt/terminalpropertiesquery request badly, or
slower than 5 seconds, could freeze the whole server for every client:
- The response handler dropped malformed responses (response.foreach(buffer.put))
instead of falling back to a default like every sibling handler, so the updater
thread waiting on the queue timed out with the properties reference still null.
- getProperties(block = true) waits while properties is null, but nothing
completes it after the updater's one-shot 5-second poll times out: waiters woke
from the notify, saw null, and waited again with no updater outstanding. The
1-second lastUpdate throttle also let a caller start waiting with no query in
flight at all, and the wait condition was checked outside the pending monitor,
losing wakeups that fired between the check and the wait.
- A response arriving after the poll timeout was delivered into a queue that was
never deregistered, so it neither set properties nor woke anyone.
The threads that block here include the command loop iterating channels and the
fast-track thread handling attach and cancel, so one bad or briefly-stalled
client wedged prompts, Ctrl-C, and command dispatch server-wide until that
client disconnected.
The properties response handler now falls back to a default like its siblings;
the updater expires its query on timeout, deregistering it (rescuing a response
that raced in) and completing properties with the empty default so waiters
always make progress; and both wait sites hold the pending monitor and gate on
the query in flight. waitForPending gets the same treatment, since it seeds
lazy vals whose initialization otherwise parks every thread touching them.
Regression test: a raw-protocol session that answers every server request with
a result of the wrong shape attaches interactively; a well-behaved batch client
must then still be served twice. Fails on develop with a three-minute timeout
(the server is frozen), passes with this change. A unit spec pins the expiry
semantics, including the late-response rescue.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
forked run used the project's baseDirectory as the working directory, while non-forked execution inherits sbt's own working directory — so toggling fork silently changed how relative paths resolved.
forked run (and forked console) now inherit sbt's working directory, consistent with non-forked execution and `sbtn` expectations.
Running reboot in the sbt shell dropped to the OS shell instead of rebooting.
The break was in teardown: Server.shutdown opened with
log.info, and during a client-initiated reboot the terminal in scope is that
client's already-closed virtual terminal, so the log write throws
ClosedChannelException through the terminal proxy. That aborted teardown
before the portfile was deleted and the server socket closed, and the
exception was swallowed by the shutdown hook (whose own error print goes to
the same dead terminal).
Server.shutdown now completes its state cleanup (portfile, tokenfile, running
flag, server socket) before logging, and CommandExchange.shutdown wraps each
channel shutdown and the server shutdown individually so one failing step
cannot skip the rest.
Fixes#9095
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
putBlobsIfNeeded reads each blob's hash and size once, up front, and
returns only plain HashedVirtualFileRef values, so serializing an ActionResult
(disk, in-memory, or remote store) performs no file I/O and nothing re-stats a
blob after its CAS entry is written: a file vanishing once stored no longer
prevents the write, and I/O errors on an output file surface upfront at blob
storage time rather than mid-serialization.
Fixes#9349
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cached-task macro allocated one mutable slot per syntactic
Def.declareOutput / Def.declareOutputDirectory call site and snapshotted the
slots into the task's outputs after the body ran. A call inside a loop or .map
over a runtime-determined list is a single syntactic site executed many times,
so each iteration overwrote the same slot and only the last file was cached and
restored on a cache hit. There was also no way for a conditional call site that
did not execute to stay out of the outputs: its slot remained null.
Declared outputs now accumulate in a per-task ListBuffer: the macro emits one
buffer at the top of the cached body and rewrites each call site to
ActionCache.registerOutput(vf, buffer), which appends and returns the value.
Every execution registers, an unexecuted site contributes nothing, and the
static multi-site shape is unchanged.
Refs #9462 (the declareOutput-in-a-loop half)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: eugene yokota <eed3si9n@gmail.com>
A directory declared via Def.declareOutputDirectory is packaged as a sibling
<dir>.sbtdir.zip, so deleting the directory leaves the zip behind. On a cache
hit, syncFile's up-to-date short-circuit saw the zip in sync (same digest,
already a CAS symlink) and returned without the unpack side effect, which only
ran from the file-write path: the directory was never restored. For sbt's own
compile, whose classes directory is declared this way, rm -rf of the classes
directory with a warm cache meant run failed with ClassNotFoundException and no
recompile; only deleting the zip as well (or the whole cache) recovered.
The up-to-date branch now re-extracts when the extracted directory itself is
missing: a single stat on the warm path, per review preference over a
manifest-based per-file check. Partial deletions inside a still-existing
directory are not repaired, consistent with treating target/ contents as
sbt-managed.
Refs #9462 (the directory-restoration half; the declareOutput-in-a-loop half is
a separate macro-layer issue)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: eugene yokota <eed3si9n@gmail.com>
Failure caching assumes a CompileFailed is a function of
the sources, which is true for source errors. But zinc also surfaces I/O write
failures ("error writing X.class") as compiler problems, so an environmental
failure (a concurrent target/ deletion, a permission blip) was cached under the
same mechanism and replayed from the global action cache on every later build,
even after the cause was gone. When the poisoned task is the metabuild compile
this is self-sustaining and unrecoverable from inside sbt: project loading
fails, so no task -- including clean -- can run, and only deleting the global
cache by hand recovers. The replayed diagnostics also name files/permissions
that no longer exist.
Refs #9455
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The thin client (sbtn) parsed the launcher value flags (-java-home, -mem,
-jvm-debug, -sbt-dir, ...) but then dropped them. The space form was consumed
and discarded; the flag=value form fell through to the residual arguments and
was forwarded to the server verbatim, where --java-home=/path was rejected as a
command (`Not a valid command: --`).
When the client had to start a server (none running, no --server), the consumed
-java-home never reached the forked sbt launcher, so the server came up under
the default JVM and, in CI where the intended JDK is only reachable via
-java-home, failed to connect.
parseArgs now captures the consumed launcher value flags (both `flag value` and
`flag=value`) into Arguments.launcherValueArgs, and the cold-start fork re-passes
them to the sbt launcher so the server runs under the requested JVM. The client
tokenizes arguments by splitting on whitespace, which would otherwise fragment a
value that contains spaces (a Windows path like C:\Program Files\Java); parseArgs
tracks those split boundaries and rejoins a value flag's value. An empty flag=
value and a dangling flag with no value are consumed but not propagated, since
the launcher's require_arg would otherwise fail the fork.
The fork command construction is extracted into a pure, package-visible
serverCommand so a test can assert the propagated flag reaches the started
server. The sbt-launch-jar path is unchanged: it invokes java directly, with no
launcher to interpret the flag.
Fixes#9418
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A task that floods stdout could not be stopped promptly: cancelling it
(Ctrl+C from the thin client) stopped the task, but the server kept
draining the already-queued output to the client for several seconds,
because onCancellationRequest never discarded the pending backlog.
onCancellationRequest now sets a per-channel isCanceled flag and clears the
queued frames and buffered stdout (discardPending), and jsonRpcNotify
drops the cancelled task's systemOut/systemErr while the flag is set, so
output stops promptly.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
NetworkChannel batches writes to the client at most once per 20ms to
reduce terminal flicker and byte volume. forceFlush(), used to order
buffered stdout ahead of a control-plane message, called
flushExecutor.shutdownNow() -- the only place the executor was ever
shut down. Once it ran during active output it tore the executor
down (and, if a coalesced flush was pending, cancelled its future
without resetting flushFuture, leaving the slot stuck), so the 20ms
coalescing was gone for the rest of the connection: stdout was then
flushed per write via the inline fallback instead of batched. Output
still reaches the client, so the effect is extra flushes rather than
lost output, but shutting the shared executor down on the first
forceFlush is clearly unintended.
Extract the flush state machine into CoalescingFlusher. forceFlush now
drains immediately and leaves the timer live (a pending coalesced
drain harmlessly drains the remainder when it fires); the executor is
shut down once, at channel teardown, so it no longer leaks. doFlush
now holds a lock across both the drain and the publish so an inline
forceFlush and the timer's drain can't deliver two stdout batches out
of order.
Fixes#9415
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Any IOException while creating the boot io socket was wrapped in
ServerAlreadyBootingException and reported as "sbt thinks that server
is already booting" with a stack trace, and non-interactive
invocations exited with code 2. Permission or path-length problems
with XDG_RUNTIME_DIR or the temp directory and Windows named-pipe
access errors all hit this, blocking sbt entirely (#6777). Raw
IOExceptions from the constructor (socket directory creation) were
not caught at all and crashed startup.
getSocketOrExit now connects to the socket (BootServerSocketProbe,
shared with the test suite) to check for a live server before
believing the exception.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
**Problem**
In an Ivy resolver pattern, the [organisation]/[organization] token has its dots rewritten to slashes (org.example → org/example), behaving like Ivy's [orgPath] token rather than being literal. Per the [Apache Ivy spec](http://ant.apache.org/ivy/history/latest-milestone/concept.html), [organisation] should be substituted literally and [orgPath] is the slash-separated form.
Root cause: Patterns.isMavenCompatible defaulted to true, which sbt forwards to Apache Ivy as setM2compatible(true); with m2-compatibility on, Ivy rewrites the [organisation] token to slash form. A user supplying a custom Ivy pattern (e.g. for an SFTP/SSH resolver) inherited that default and got the wrong paths, with no obvious indication why. The only workaround was the non-obvious withIsMavenCompatible(false).
**Solution**
Flip the default of Patterns.isMavenCompatible from true to false, so a hand-written Patterns keeps [organisation] literal by default — matching the Ivy spec.
Building on #9270 (which parallelized resolution in non-interactive runs by
narrowing the lm-coursier lock to only fire while coursier renders its
interactive progress bar), this makes `update` resolve in parallel under the
interactive super shell as well.
Count distinct non-checksum urls instead and drop the module claim.
Report the elapsed time of the current burst so the super shell renders
a live counter rather than a frozen "0s". Tests now encode coursier's
per-session call pattern.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a file referenced by a task's inputs/outputs (e.g. Compile / resources +=
file("nope.txt")) does not exist, hashing the task's cache key threw a
NoSuchFileException deep inside sjsonnew serialization. It surfaced as an opaque
sjsonnew.SerializationException that dumped the entire input list, with the real
cause buried several `Caused by:` levels down, so users routinely mistook it for
a corrupt cache and reached for `clean`.
ActionCache.mkInput now catches the hashing failure, detects a NoSuchFileException
anywhere in the cause chain (ActionCache.findMissingFile), and throws a
MessageOnlyException naming the file:
[error] file referenced by the build does not exist: nope.txt
util-cache gains a dependency on util-control (a leaf module, no cycle) for
MessageOnlyException.
Fixes#9217.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
update held a process-global lock around coursier resolution and artifact
fetching whenever a logger was set OR coursier was not in fallback mode. That
lock exists only to serialize coursier's interactive progress bar, which is
rendered solely when no custom logger is supplied and coursier is not in
fallback mode. The loggerOpt.nonEmpty clause therefore over-serialized the
common non-interactive case (IntelliJ re-imports, CI, any non-TTY run, where
sbt supplies a quiet debug-only logger), making update scale with the number
of modules rather than the number of distinct artifacts.
Fixes#5508.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes#6886.
dependencyTree, dependencyBrowseTree, and inspect tree re-explore
the same node once per incoming edge. In a DAG with N levels and M
children per node the rendered output is O(M^N) -- the OP needed
>16 GB heap, #7360 has a 6 GB heap dump, and Friendseeker's analysis
on the issue showed the exponential re-traversal directly.
Fix: track a visited set across the renderer's recursion. The first
time a node is encountered it is rendered in full; on subsequent
visits the entry collapses to a one-line +- <id> (*) (ASCII) or a
<id> (*) leaf (JSON), matching Maven's dependency:tree (*)
convention. Cycle detection (separate parents set, (cycle) marker)
is unchanged.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [2.x] feat: Add cacheVersion setting for global cache invalidation
**Problem**
There was no escape hatch to invalidate all task caches when needed.
**Solution**
Add `Global / cacheVersion` setting that incorporates into the cache key
hash. Changing it invalidates all caches. Defaults to reading system
property `sbt.cacheversion`, or else 0L. When 0L, the hash is identical
to the previous behavior (backward compatible).
Fixes#8992
* [2.x] refactor: Simplify BuildWideCacheConfiguration and add cacheVersion test
- Replace auxiliary constructors with default parameter values
- Add unit test verifying cacheVersion invalidates the cache
* [2.x] fix: Restore auxiliary constructors for binary compatibility
* [2.x] test: Improve cacheVersion scripted test and add release note
- Scripted test now verifies cache invalidation via a counter
that increments only when the task body actually executes
- Add release note documenting the cacheVersion setting
For the details about this PR, please see the blog post https://eed3si9n.com/sbt-remote-cache/.
* Add cache basics
* Refactor Attributed to use StringAttributeMap, which is Map[StringAttributeKey, String]
* Implement disk cache
* Rename Package to Pkg
* Virtualize packageBin
* Use HashedVirtualFileRef for packageBin
* Virtualize compile task
Normally scripted tests are forked using the JVM that is running sbt.
If set `scripted / javaHome`, forked using it.
```
scripted / javaHome := Some(file("/path/to/jdk-x.y.z"))
```
Or use `java++` command before scripted.
```
sbt> java++ 11!
sbt> scripted
```
Add `testReportsDirectory` setting to allow output directory for
JUnitXmlTestsListener to be configured.
Add `testReportSettings` which provides defaults values:
- by default this uses the build configuration name as a prefix so
`target/test-reports` for `Test` config, but `target/it-reports`
for `IntegrationTest` (previously this was hardcoded to always
use `target/test-reports`). To override this set e.g.
`Test / testReportsDirectory := target.value / "my-custom-dir"`
- the `JunitXmlTestsListener` is now only attached to the `Test`
and `IntegrationTest` configs by default (previously it was added
to the global configuration object). Any configs which inherit
from one of these will continue to have the listener attached;
but completely custom configurations will need to re-add with:
`project.settings(testReportSettings)`
Fixes#2853
Fixes https://github.com/sbt/sbt/issues/5047
When setting swoval.tmpdir via globalBase, changed to set globalBase as absolute path.
`com.swoval.runtime.NativeLoader.loadPackaged` uses `java.lang.System.load`.
It requires absolute path, so we should set `swoval.tmpdir` with absolute path.
There's also a special case for aliases that will try to resolve
the target of the alias to a task key if possible and display the
output of that key if found.
see https://github.com/sbt/sbt/issues/2881
Fixes https://github.com/sbt/sbt/issues/1502
This adds `--addPluginSbtFile=<file>` command, which adds the given .sbt file to the plugin build.
Using this mechanism editors or IDEs can start a build with required plugin.
```
$ cat /tmp/extra.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.7")
$ sbt --addPluginSbtFile=/tmp/extra.sbt
...
sbt:helloworld> plugins
In file:/xxxx/hellotest/
...
sbtassembly.AssemblyPlugin: enabled in root
```
BrianHotopp
eugene yokota
Jozef Koval
Dream
seroperson
Josh Soref
xuwei-k
Krzysztof Pado
Rui Gonçalves
kijuky
kxbmap
Ashley Mercer
Yusuke Yamada
Josh Soref
Yusuke Izawa
Nepomuk Seiler
Kazuhiro Sera
Jason Pickens
Guillaume Poirier
alodavi