diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..25f455fa6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,56 @@ +AGENTS instructions +=================== + +The main developer documentation is [Contributor's guide](contributing-docs/README.md). + +- [Development environment](contributing-docs/02_development_environment.md) + +Compiling with sbt +------------------ + +```bash +sbt compile +``` + +Pull reuqest guideline +---------------------- + +- Follow the PR guidance in . +- In the commit message, include "Generated-by" tag for Gen-AI tools. + +Coding style +------------ + +```bash +sbt scalafmtAll +``` + +- Follow [Coding style and best practices](contributing-docs/03_coding_style.md) +- Avoid inline comments! + +Tests +----- + +Always add tests. For changes with small scopes prefer HedgeHog for Scala. +For changes that require coordination with file changes and tasks, use scripted test. + +- +- +- + +Binary compatibility +-------------------- + +sbt MUST maintain backward binary compatibility across minor releases. +This means removing public method signature MUST be avoided. + +Use mima to check: + +```bash +sbt mimaReportBinaryIssues +``` + +Copyright +--------- + +- NEVER reproduce copyrighted material. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2912829be..c56200a9f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,6 +4,7 @@ [Discussions]: https://github.com/sbt/sbt/discussions [327]: https://github.com/sbt/sbt/issues/327 [documentation]: https://github.com/sbt/website + [Discord]: https://discord.com/invite/scala Contributing ============ @@ -13,17 +14,23 @@ Contributing There are lots of ways to contribute to sbt ecosystem depending on your interests and skill level. - Help someone at work or online fix their build problem. -- Answer StackOverflow questions. -- Ask StackOverflow questions. +- Answer/ask StackOverflow questions or on [Scala Discord][Discord]. - Create plugins that extend sbt's features. +- Migrate builds to sbt 2.x. - Maintain and update [documentation]. - Garden the issue tracker. - Report issues. - Patch the core (send pull requests to code). -- On-ramp other contributors. +- Review pull requests. -Issues and Pull Requests ------------------------- +Ideas and proposals +------------------- + +If you have an enhancement idea, or a general discussion, please follow the [sbt 2.0 RFC process](https://eed3si9n.com/sbt-2.0-rfc-process) or start a [Discussion][Discussions]. + + +Reporting Issues +---------------- When you find a bug in sbt we want to hear about it. Your bug reports play an important part in making sbt more reliable and usable. @@ -31,8 +38,6 @@ Effective bug reports are more likely to be fixed. These guidelines explain how Please open a GitHub issue when you are 90% sure it's an actual bug. -If you have an enhancement idea, or a general discussion, please follow the [sbt 2.0 RFC process](https://eed3si9n.com/sbt-2.0-rfc-process) or start a [Discussion][Discussions]. - ### Notes about Documentation Documentation fixes and contributions are as much welcome as to patching the core. Visit [sbt/website][documentation] to learn about how to contribute. @@ -111,7 +116,7 @@ Other notes ## Signing the CLA Contributing to sbt requires you or your employer to sign the -[Scala Contributor License Agreement](https://www.lightbend.com/contribute/cla/scala). +[Scala Contributor License Agreement](https://contribute.akka.io/contribute/cla/scala). To make it easier to respect our license agreements, we have added an sbt task that takes care of adding the LICENSE headers to new files. Run `headerCreate` diff --git a/DEVELOPING.md b/DEVELOPING.md deleted file mode 100644 index d06145bef..000000000 --- a/DEVELOPING.md +++ /dev/null @@ -1,181 +0,0 @@ -Developer guide -=============== - -### Getting started - -Create a [fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) of the repository and [clone](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository) it to create a local copy. - -### Branch to work against - -sbt uses two or three branches for development: -Use the **default** branch set on GitHub for bug fixes. - -- Next minor branch: `1.$MINOR.x`, where `$MINOR` is next minor version (e.g. `1.10.x` during 1.9.x series) -- Development branch: `develop` -- Stable branch: `1.$MINOR.x`, where `$MINOR` is current minor version (e.g. `1.9.x` during 1.9.x series) - -The `develop` branch represents sbt 2.x, the next major sbt series. -Next minor branch is where new features should be added as long as it is binary compatible with sbt 1.x. -The `stable` branch represents the current stable sbt release. Only bug fixes are back-ported to the stable branch. - -### Note on supported JDK version for the sbt build - -The sbt build itself currently doesn't support any JDK beyond version 21. You may run into deprecation warnings (which would become build errors due to build configuration) if you use any later JDK version to build sbt. - -If you're using Metals as IDE, also check the `Java Version` setting. The default at the time of writing this is `17`, but this may change in the future, or you may have set it to a later version yourself. (Be aware that Metals may download a JDK in the background if you haven't switch to a local JDK matching the version before changing this setting. Also, this setting is currently only available as a `User` setting, so you can't set it for a single workspace. Don't forget to switch back if you need to for other projects). - -### Instruction to build just sbt - -Sbt has a number of sub-modules. If the change you are making is just contained in sbt/sbt (not one of the sub-modules), -you can use the `publishLocal` command to publish the sbt project to your local machine. This is helpful for testing your changes. - -``` -$ sbt -sbt:sbtRoot> publishLocal -``` - -### Instruction to build sbtn - -```bash -$ sbt nativeImage -``` - -On macOS, the following can be used to target ARM64: - -```bash -$ ARCHS=arm64 sbt nativeImage -``` - -### Instruction to build all modules from source - -When working on a change that requires changing one or more sub modules, the source code for these modules can be pulled in by running the following script -(instead of building them from Maven for example and not being able to change the source code for these sub-modules). - -1. Install the current stable binary release of sbt (see [Setup]), which will be used to build sbt from source. -2. Get the source code. - - ``` - $ mkdir sbt-modules - $ cd sbt-modules - $ for i in sbt io zinc; do \ - git clone https://github.com/sbt/$i.git && (cd $i; git checkout -b develop origin/develop) - done - $ cd sbt - $ ./sbt-allsources.sh - ``` - -3. To build and publish all components locally, - - ``` - $ ./sbt-allsources.sh - sbt:sbtRoot> publishLocalAllModule - ``` - -### Using the locally built sbt - -The `publishLocal` command above will build and publish version `1.$MINOR.$PATCH-SNAPSHOT` (e.g. 1.1.2-SNAPSHOT) to your local ivy repository. - -To use the locally built sbt, set the version in `build.properties` file in your project to `1.$MINOR.$PATCH-SNAPSHOT` then launch `sbt` (this can be the `sbt` launcher installed in your machine). - -``` -$ cd $YOUR_OWN_PROJECT -$ sbt -> compile -``` - -### Clearing out boot and local cache - -sbt consists of lots of JAR files. When running sbt locally, these JAR artifacts are cached in the `boot` directory under `$HOME/.sbt/boot/scala-2.12.6/org.scala-sbt/sbt/1.$MINOR.$PATCH-SNAPSHOT` directory. - -In order to see a change you've made to sbt's source code, this cache should be cleared. To clear this out run: `reboot dev` command from sbt's session of your test application. - -By default sbt uses a snapshot version (this is a scala convention for quick local changes- it tells users that this version could change). -One drawback of `-SNAPSHOT` version is that it's slow to resolve as it tries to hit all the resolvers. -This is important when testing performance, so that the slowness of the resolution does not impact sbt. - -You can workaround that by using a version name like `1.$MINOR.$PATCH-LOCAL1`. -A non-SNAPSHOT artifacts will now be cached under `$HOME/.ivy/cache/` directory, so you need to clear that out using [sbt-dirty-money](https://github.com/sbt/sbt-dirty-money)'s `cleanCache` task. - -### Running sbt "from source" - `sbtOn` - -In addition to locally publishing a build of sbt, there is an alternative, experimental launcher within sbt/sbt -to be able to run sbt "from source", that is to compile sbt and run it from its resulting classfiles rather than -from published jar files. - -Such a launcher is available within sbt/sbt's build through a custom `sbtOn` command that takes as its first -argument the directory on which you want to run sbt, and the remaining arguments are passed _to_ that sbt -instance. For example: - -I have setup a minimal sbt build in the directory `/s/t`, to run sbt on that directory I call: - -```bash -> sbtOn /s/t -[info] Packaging /d/sbt/scripted/sbt/target/scala-2.12/scripted-sbt_2.12-1.2.0-SNAPSHOT.jar ... -[info] Done packaging. -[info] Running (fork) sbt.RunFromSourceMain /s/t -Listening for transport dt_socket at address: 5005 -[info] Loading settings from idea.sbt,global-plugins.sbt ... -[info] Loading global plugins from /Users/dnw/.dotfiles/.sbt/1.0/plugins -[info] Loading project definition from /s/t/project -[info] Set current project to t (in build file:/s/t/) -[info] sbt server started at local:///Users/dnw/.sbt/1.0/server/ce9baa494c7598e4d59b/sock -> show baseDirectory -[info] /s/t -> exit -[info] shutting down sbt server -[success] Total time: 19 s, completed 25-Apr-2018 15:04:58 -``` - -Please note that this alternative launcher does _not_ have feature parity with sbt/launcher. (Meta) -contributions welcome! :-D - -### Updating Scala version - -See https://github.com/sbt/sbt/pull/6522 for the list of files to change for Scala version upgrade. - -### Diagnosing build failures - -Globally included plugins can interfere building `sbt`; if you are getting errors building sbt, try disabling all globally included plugins and try again. - -### Running Tests - -sbt has a suite of unit tests and integration tests, also known as scripted tests. - -#### Unit / Functional tests - -Various functional and unit tests are defined throughout the -project. To run all of them, run `sbt test`. You can run a single test -suite with `sbt testOnly` - -#### Integration tests - -Scripted integration tests reside in `sbt-app/src/sbt-test` and are -written using the same testing infrastructure sbt plugin authors can -use to test their own plugins with sbt. You can read more about [Testing sbt plugins](https://www.scala-sbt.org/1.x/docs/Testing-sbt-plugins). - -You can run the integration tests with the `sbt scripted` sbt -command. To run a single test, such as the test in -`sbt-app/src/sbt-test/project/global-plugin`, simply run: - - sbt "scripted project/global-plugin" - -### Random tidbits - -### Clean history - -Make sure you document each commit and squash them appropriately. You can use the following guides as a reference: - -* Scala's documentation on [Git Hygiene](https://github.com/scala/scala/tree/v2.12.0-M3#git-hygiene) -* Play's documentation on [Working with Git](https://www.playframework.com/documentation/2.4.4/WorkingWithGit#Squashing-commits) - -#### Import statements - -You'd need alternative DSL import since you can't rely on sbt package object. - -```scala -// for slash syntax -import sbt.SlashSyntax0.given - -// for IO -import sbt.io.syntax._ -``` diff --git a/README.md b/README.md index 9dbe4d146..de7c3d568 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ [sbt/sbt-zero-seven]: https://github.com/sbt/sbt-zero-seven [CONTRIBUTING]: CONTRIBUTING.md - [DEVELOPING]: DEVELOPING.md + [contributing-docs]: contributing-docs/README.md [Setup]: https://www.scala-sbt.org/release/docs/Getting-Started/Setup [FAQ]: https://www.scala-sbt.org/release/docs/Faq.html [sbt-dev]: https://groups.google.com/d/forum/sbt-dev @@ -41,7 +41,7 @@ several GitHub repositories, including this one. Issues and Pull Requests ------------------------ -Please read [CONTRIBUTING] carefully before opening a GitHub Issue, and [DEVELOPING] before opening a pull request. +Please read [CONTRIBUTING] carefully before opening a GitHub Issue, and [Contributor's guide][contributing-docs] before opening a pull request. If you're looking for an idea for a contribution, issues labeled with [good first issue](https://github.com/sbt/sbt/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or diff --git a/contributing-docs/01_pull_request.md b/contributing-docs/01_pull_request.md new file mode 100644 index 000000000..3b0464686 --- /dev/null +++ b/contributing-docs/01_pull_request.md @@ -0,0 +1,160 @@ +Sending pull request +==================== + +This document describes how you can create Pull Requests (PRs) and describes coding standards we use when implementing them. + +### Getting started + +Create a [fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) of the repository and [clone](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository) it to create a local copy. + +### Protect your identity + +Once a PR is merged, the commit author's (and any co-author's) name and email address become permanently public as part of the commit history and cannot be changed or removed afterward. + +You can verify your current configuration by running the following commands in your terminal within your local sbt repository: + +```bash +git config --get user.name +git config --get user.email +``` + +### Branch to work against + +sbt uses two branches for development: +Use the **default** branch set on GitHub for bug fixes. For backports, use the latest stable branch. + +- Development branch: `develop` +- Stable branch: `1.$MINOR.x`, where `$MINOR` is current minor version (e.g. `1.12.x` during 1.12.x series) + +The `develop` branch represents sbt 2.x, the next major sbt series. +The `stable` branch represents the current stable sbt 1.x release. Once sbt 2.x is released mostly bug fixes are back-ported to the stable branch. + +### Pull Request guidelines + +Before you submit a Pull Request (PR) from your forked repo, check that it meets these guidelines: + +- Include tests, either as scripted test, unit tests, or both, to your pull request. +- Follow our project's [Commit message guideline](#commit). +- Follow our project's [Coding style and best practices](03_coding_style.md). +- Sign the [Scala Contributor License Agreement](https://contribute.akka.io/contribute/cla/scala). +- Make sure your PR is small and focused on one change only - avoid adding unrelated changes, mixing adding features and refactoring. Keeping to that rule will make it easier to review your PR and will make it easier for core devs if they decide that your change should be cherry-picked to release it in a stable release of sbt. +- Maintainers will not merge a PR that regresses linting or does not pass CI tests (unless you have good justification that it a transient error or something that is being fixed in other PR). +- Maintainers will not merge a PR that breaks binary compatibility ("bincompat"). Run `mimaReportBinaryIssues` from the sbt shell. +- When merging PRs, Maintainer may use **Squash and Merge** which means then your PR will be merged as **one commit**, regardless of the number of commits in your PR. During the review cycle, you can keep a commit history for easier review. +- You can use any supported JDK version to run the tests, but the best is to check if it works for the oldest supported version (JDK 8 currently). In rare cases tests might fail with the oldest version when you use features that are available in newer JDK versions. +- Add an Apache header to all new files. Run `headerCreate` and sbt will put a copyright notice into it. + + +### Gen-AI assisted contributions + +Generally, it's fine to use Gen-AI tools to help you create Pull Requests for sbt as long as you adhere to the following guidelines: + +- State in your PR description that you have used Gen-AI tools to assist in creating the PR. +- No third party materials are included in the output; or materials that are included in the output are in compliance with an open source license compatible with Apache License. +- Ensure that you review and understand all code generated by Gen-AI tools before including it in your PR - do not blindly trust the generated code. +- State in your PR description that you have used Gen-AI tools to assist in creating the PR. +- Remember that the final responsibility for the code in your PR lies with you, regardless of whether it was generated by a tool or written by you. +- Blindly copy-pasting code from Gen-AI tools is detrimental as it might introduce security and stability risks to the project. Maintainers that spot such behaviours MAY close the related PRs and/or block the user from making further contributions. + + +### Commit message guideline + +Follow the following template: + +``` +[2.x] fix: Fix consoleProject not starting + +**Problem** +consoleProject doesn't work. REPL doesn't even start. + +**Solution** +I made some progress into consoleProject. +At least Scala 3.7 repl session will now start. + +Generated-by: Claude Sonnet 4.5 +``` + +1. (Optional) Subject should start with `[2.x]` for develop branch, and `[1.x]` for sbt 1.x +2. Subject should start with `fix` (bug fix), `feat` (new feature), `refactor`, `test`, `ci`, or `deps` +3. Subject should use imperative mood, for example Fix foo, Add bar. +4. Body should include Problem section, which summarizes the current understanding of the issue. +5. Body should include Solution section, which summarizes your approach to fixing the issue. +6. Do not at-mention people in the commit message. +7. Include "Generated-by" tag for Gen-AI tools. + +### Testing + +sbt features a testing infrastructure encompassing multiple testing methodologies designed to ensure reliability and functionality across different integrations. The testing framework includes: + +- [Unit tests](04_unit_tests.md) +- [scripted tests](05_scripted_tests.md) +- [Manual tests](06_manual_tests.md), using the locally baked sbt + +### Instruction to build just sbt + +sbt has a number of sub-modules. If the change you are making is just contained in sbt/sbt (not one of the sub-modules), +you can use the `publishLocalBin` command to publish the sbt project to your local machine. This is helpful for testing your changes. + +```bash +$ sbt +> publishLocalBin +``` + +### Instruction to build sbtn + +```bash +$ sbt nativeImage +``` + +On macOS, the following can be used to target ARM64: + +```bash +$ ARCHS=arm64 sbt nativeImage +``` + +### Instruction to build all modules from source + +When working on a change that requires changing one or more sub modules, the source code for these modules can be pulled in by running the following script +(instead of building them from Maven for example and not being able to change the source code for these sub-modules). + +1. Install the current stable binary release of sbt (see [Setup]), which will be used to build sbt from source. +2. Get the source code. + + ```bash + $ mkdir sbt-modules + $ cd sbt-modules + $ for i in sbt io zinc; do \ + git clone https://github.com/sbt/$i.git && (cd $i; git checkout -b develop origin/develop) + done + $ cd sbt + $ ./sbt-allsources.sh + ``` + +3. To build and publish all components locally, + + ```bash + $ ./sbt-allsources.sh + sbt:sbtRoot> publishLocalAllModule + ``` + +### Updating Scala version + +See https://github.com/sbt/sbt/pull/6522 for the list of files to change for Scala version upgrade. + +### Diagnosing build failures + +Globally included plugins can interfere building `sbt`; if you are getting errors building sbt, try disabling all globally included plugins and try again. + +### Random tidbits + +#### Import statements + +You'd need alternative DSL import since you can't rely on sbt package object. + +```scala +// for slash syntax +import sbt.SlashSyntax0.given + +// for IO +import sbt.io.syntax.* +``` diff --git a/contributing-docs/02_development_environment.md b/contributing-docs/02_development_environment.md new file mode 100644 index 000000000..898ee870c --- /dev/null +++ b/contributing-docs/02_development_environment.md @@ -0,0 +1,21 @@ +Development environment +======================= + +Generally sbt can be developed on any environment that supports JDK and sbt. +This includes macOS, Linux, and modern Windows. + +1. Install JDK (We recommend Azul Zulu JDK 8, 11, or 17 on macOS, and Temurin elsewhere). +2. Install `sbt`, following . + +sbt acts as a package manager, so once you installed JDK and sbt, it will download necessary dependencies. + +IDE support +----------- + +Scala has two IDEs: IntelliJ (Scala plugin) and VS Code (Metals). Either should work. + +### Note on supported JDK version for the sbt build + +The sbt build itself currently doesn't support any JDK beyond version 25. You may run into deprecation warnings (which would become build errors due to build configuration) if you use any later JDK version to build sbt. + +If you're using Metals as IDE, also check the `Java Version` setting. The default at the time of writing this is `17`, but this may change in the future, or you may have set it to a later version yourself. (Be aware that Metals may download a JDK in the background if you haven't switch to a local JDK matching the version before changing this setting. Also, this setting is currently only available as a `User` setting, so you can't set it for a single workspace. Don't forget to switch back if you need to for other projects). diff --git a/contributing-docs/03_coding_style.md b/contributing-docs/03_coding_style.md new file mode 100644 index 000000000..f0bbe0b2b --- /dev/null +++ b/contributing-docs/03_coding_style.md @@ -0,0 +1,86 @@ +sbt coding style and best practices +=================================== + +Some of our coding style rules are enforced programmatically by Scalafmt, which are run automatically with static checks and on every Pull Request (PR), but there are some rules that are not yet automated and are more sbt specific. + +General style +------------- + +### Naming + +- Use short names for small scopes, for example `xs` and `x`. +- Use longer names for larger scopes. + +### Functional programming + +- Prefer succinct and pure functions. +- Prefer the use of `Option` and `Either`, rather than `null` and `Exception`. + +### Import + +- Put all imports at the top of the file + +### Braces + +- Prefer to omit braces in Scala 3.x, that is use SIP-44 Fewer Braces syntax, especially for fresh code. + +```scala +// BAD +xs.map { x => + val y = x - 1 + y * y +} + +// GOOD +xs.map: x => + val y = x - 1 + y * y +``` + +- Use `end` marker for a class, trait, and object definition, regardless of the length of the template. + +```scala +// BAD +object O1 { + def foo: Int = 1 +} + +// GOOD +object O1: + def foo: Int = 1 +end O1 +``` + +### Infix notation + +- Avoid the infix notation for non-symbolic methods `a foo 1`, and use the method call notation `a.foo(1)` instead. + +### Comments + +- Use ScalaDoc to provide API documentation. In general, however, document the intent and background behind the code, rather than transcribing code to English. +- Avoid exessive inline comments, especially the ones that repeats the same information as the code. + +### Returns + +- Avoid `return` statements. + +Modular design +-------------- + +Because sbt has 100+ plugins, we have to be careful not to break them when we make changes. + +- Maintain backward binary compatibility. That is if a plugin was published against sbt 2.0, it must work on sbt 2.1. +- Expose as few public methods as possible. +- Hide implementation details in `sbt.internal.x` packages. +- Avoid exposing types from external libraries. We often change the library uses over the course of years. + +sbt build.sbt DSL +----------------- + +- Set the default value in the widest scope, such as the global scope. + +Hermeticity / stability +----------------------- + +- Avoid capturing machine-specific details in the task, including the absolute path. +- Avoid capturing time details in the task, including timestamps. diff --git a/contributing-docs/04_unit_tests.md b/contributing-docs/04_unit_tests.md new file mode 100644 index 000000000..5d482b1aa --- /dev/null +++ b/contributing-docs/04_unit_tests.md @@ -0,0 +1,100 @@ +sbt Unit tests +============== + +Various functional and unit tests are defined throughout the +project. + +- To run all of them, run `test` in the sbt shell. +- You can run a single test suite with `testOnly **.FooTest`. +- You can run an incremental test as `testQuick`. +- You can run an incremental test scoped to a subproject as `commandProj/testQuick` + +Property-based testing with HedgeHog +------------------------------------ + +sbt contains various testing frameworks, but our prefered unit testing framework is HedgeHog. + +See [main-actions/src/test/scala/sbt/internal/WorkerExchangeTest.scala](../main-actions/src/test/scala/sbt/internal/WorkerExchangeTest.scala) for an example. + +```scala +package sbt +package internal + +import hedgehog.* +import hedgehog.runner.* +import hedgehog.core.{ ShrinkLimit, SuccessCount } +import hedgehog.core.Result +import scala.sys.process.Process + +object WorkerExchangeTest extends Properties: + given Gen[WorkerConnection] = + Gen.choice1(Gen.constant(WorkerConnection.Stdio), Gen.constant(WorkerConnection.Tcp)) + + def gen[A1: Gen]: Gen[A1] = summon[Gen[A1]] + + override lazy val tests: List[Test] = List( + propertyN("non-jsonrpc should return exit code 1", propBadInput, 10), + ) + + def propertyN(name: String, result: => Property, n: Int): Test = + Test(name, result) + .config(_.copy(testLimit = SuccessCount(n), shrinkLimit = ShrinkLimit(n * 10))) + + def propBadInput: Property = + for + ct <- gen[WorkerConnection].forAll + w = WorkerExchange.startWorker(ForkOptions(), Nil, ct) + yield + w.println("{}") + val exitCode = w.blockForExitCode() + Result.assert(exitCode == 1) +end WorkerExchangeTest +``` + +Unit testing with HedgeHog +-------------------------- + +HedgeHog can be used for simple unit testing as well using `example(...)`: + +```scala +package sbt.internal.util + +import hedgehog.* +import hedgehog.runner.* + +object SourcePositionTest extends Properties: + override def tests: List[Test] = List( + example( + "SourcePosition() should return a SourcePosition", { + val filename = "SourcePositionTest.scala" + val lineNumber = 19 + SourcePosition.fromEnclosing() match + case pos @ LinePosition(path, startLine) => + Result.assert(path == filename && startLine == lineNumber) + .log(pos.toString()) + } + ) + ) +end SourcePositionTest +``` + +Unit testing with Verify +------------------------ + +Verify can also be used for unit tests: + +```scala +package sbt.util + +import sjsonnew.BasicJsonProtocol +import sjsonnew.support.murmurhash.Hasher +import verify.BasicTestSuite + +object HasherTest extends BasicTestSuite: + test("The IntJsonFormat should convert an Int to an int hash") { + import BasicJsonProtocol.given + val actual = Hasher.hashUnsafe[Int](1) + assert(actual == 1527037976) + } +end HasherTest +``` diff --git a/contributing-docs/05_scripted_tests.md b/contributing-docs/05_scripted_tests.md new file mode 100644 index 000000000..a7b65efb1 --- /dev/null +++ b/contributing-docs/05_scripted_tests.md @@ -0,0 +1,183 @@ +Scripted tests +============== + +sbt has a suite of integration tests, also known as scripted tests. +Scripted integration tests reside in **`sbt-app/src/sbt-test`** and are +written using the same testing infrastructure sbt plugin authors can +use to test their own plugins with sbt. + +You can run the integration tests with the `sbt scripted` sbt +command. To run a single test, such as the test in +`sbt-app/src/sbt-test/project/global-plugin`, from the sbt shell run: + +```bash +> scripted project/global-plugin +``` + +The scripted test framework lets you script a build scenario. It was written to test sbt itself on complex scenarios -- such as change detection and partial compilation. + +How to create a scripted test +----------------------------- + +### step 1: sbt-app/src/sbt-test + +Make a directory structure `sbt-app/src/sbt-test//`. +For example, `sbt-app/src/sbt-test/project/something`. + +Create an initial build in `something`. Like a real build using sbt. I'm sure you already have several of them to test manually. Here's an example `build.sbt`: + +```scala +name := "foo" +scalaVersion := "3.7.4" +``` + +I also have `Hello.scala`: + +```scala +@main def hello(): Unit = println("hi") +``` + +### step 2: Write a script + +Now, write a script to describe your scenario in a file called `test` located at the root dir of your test project. + +```bash +# check if the file gets created +> packageBin +$ exists target/**/foo/foo_3-0.1.0-SNAPSHOT.jar +``` + +Here is the syntax for the script: + +1. **`#`** starts a one-line comment +2. **`>`** `name` sends a task to sbt (and tests if it succeeds) +3. **`$`** `name arg*` performs a file command (and tests if it succeeds) +4. **`->`** `name` sends a task to sbt, but expects it to fail +5. **`-$`** `name arg*` performs a file command, but expects it to fail + +File commands are: + +- **`touch`** `path+` creates or updates the timestamp on the files +- **`delete`** `path+` deletes the files +- **`exists`** `path+` checks if the files exist +- **`mkdir`** `path+` creates dirs +- **`absent`** `path+` checks if the files don't exist +- **`newer`** `source target` checks if `source` is newer +- **`must-mirror`** `source target` checks if `source` is identical +- **`pause`** pauses until enter is pressed +- **`sleep`** `time` sleeps (in milliseconds) +- **`exec`** `command args*` runs the command in another process +- **`copy-file`** `fromPath toPath` copies the file +- **`copy`** `fromPath+ toDir` copies the paths to `toDir` preserving relative structure +- **`copy-flat`** `fromPath+ toDir` copies the paths to `toDir` flat + +So my script will run `packageBin` task, and checks if a JAR file gets created. We'll cover more complex tests later. + +### step 5: run the script + +To run the scripts run the following from the sbt shell (in sbt/sbt): + +```bash +> scripted project/something +``` + +**Note**: `scripted` runs all your tests. + +This will copy your test build into a temporary dir, and executes the `test` script. If everything works out, you'd see `publishLocal` running, then: + +```bash +[info] Tests selected: +[info] * project/something +[info] Running 1 / 1 (100.00%) scripted tests with LauncherBased(/Users/xxx/work/sbt/launch/target/sbt-launch.jar) +[info] Running project/something +[success] Total time: 12 s +``` + +Custom assertion +---------------- + +The file commands are great, but not nearly enough because none of them test the actual contents. An easy way to test the contents is to implement a custom task in your test build. + +For my hello project, I'd like to check if the resulting jar prints out "hello". I can use `scala.sys.process.Process` to run the JAR. To express a failure, just throw an error. Here's `build.sbt`: + +```scala +import scala.sys.process.Process + +@transient +lazy val check = taskKey[Unit]("check") + +name := "foo" +scalaVersion := "3.7.4" +check := { + val pkg = (Compile / packageBin).value + val conv = fileConverter.value + val cp0 = (Compile / externalDependencyClasspath).value + .map(_.data) + .map(conv.toPath(_)) + .toList + val cp = (crossTarget.value / "foo_3-0.1.0-SNAPSHOT.jar") :: cp0 + val p = Process("java", Seq("-cp", cp.mkString(":"), "hello")) + val out = p.!! + if out.trim == "bye" then () + else sys.error("unexpected output: " + out) +} +``` + +I am intentionally testing if it matches "bye", to see how the test fails. + +Here's `test`: + +```bash +# check if the file gets created +> packageBin +$ exists target/**/foo/foo_3-0.1.0-SNAPSHOT.jar + +# check if it says hi +> check +``` + +Running `scripted project/something` fails the test as expected: + +```scala +[info] [error] java.lang.RuntimeException: unexpected output: hi +[info] [error] +[info] [error] at scala.sys.package$.error(package.scala:27) +[info] [error] at $Wrap2988201cb0$.$sbtdef$$anonfun$1(build.sbt:18) +.... +[info] [error] at java.lang.Thread.run(Thread.java:750) +[info] [error] (check) unexpected output: hi +``` + +Testing the test +---------------- + +There are a few tips on debugging the scripted tests. + +First is to increase the log level. Add the following line to your `test` script to get the debug log: + +```bash +> debug +``` + +To suspend the test until you hit the enter key, add the following line to your `test` script: + +```bash +$ pause +``` + +Testing changes +--------------- + +See `source-dependencies/abstract-type`'s script as an example. + +```bash +> compile + +# remove type arguments from S +$ copy-file changes/A.scala A.scala + +# Both A.scala and B.scala should be recompiled, producing a compile error +-> compile +``` + +The convention is to store files under `changes/`, and use `copy-file` command to emulate the file changes. diff --git a/contributing-docs/06_manual_tests.md b/contributing-docs/06_manual_tests.md new file mode 100644 index 000000000..aed374f95 --- /dev/null +++ b/contributing-docs/06_manual_tests.md @@ -0,0 +1,68 @@ +Manual tests +============ + +Use the `publishLocalBin` command to publish the sbt project to your local machine. This is helpful for testing your changes. + +```bash +$ sbt +> publishLocalBin +``` + +### Using the locally built sbt + +The `publishLocalBin` command above will build and publish version `2.$MINOR.$PATCH-SNAPSHOT` (e.g. 1.1.2-SNAPSHOT) to your local Ivy repository. + +To use the locally built sbt, set the version in `build.properties` file in your project to `2.$MINOR.$PATCH-SNAPSHOT` then launch `sbt` (this can be the `sbt` launcher installed in your machine). + +```bash +$ cd $YOUR_OWN_PROJECT +$ sbt --server +> compile +``` + +### Clearing out boot and local cache + +sbt consists of lots of JAR files. When running sbt locally, these JAR artifacts are cached in the `boot` directory under `$HOME/.sbt/boot/scala-3.7.4/org.scala-sbt/sbt/2.0.0-RC8-bin-SNAPSHOT` directory. + +In order to see a change you've made to sbt's source code, this cache MUST be cleared. To clear this out, from the sbt shell in your application run: + +```bash +> reboot dev +``` + +By default sbt uses a snapshot version (this is a scala convention for quick local changes- it tells users that this version could change). +One drawback of `-SNAPSHOT` version is that it's slow to resolve as it tries to hit all the resolvers. +This is important when testing performance, so that the slowness of the resolution does not impact sbt. + +### Running sbt "from source" - `sbtOn` + +In addition to locally publishing a build of sbt, there is an alternative, experimental launcher within sbt/sbt +to be able to run sbt "from source", that is to compile sbt and run it from its resulting classfiles rather than +from published jar files. + +Such a launcher is available within sbt/sbt's build through a custom `sbtOn` command that takes as its first +argument the directory on which you want to run sbt, and the remaining arguments are passed _to_ that sbt +instance. For example: + +I have setup a minimal sbt build in the directory `/s/t`, to run sbt on that directory I call: + +```bash +> sbtOn /s/t +[info] Packaging /d/sbt/scripted/sbt/target/scala-2.12/scripted-sbt_2.12-1.2.0-SNAPSHOT.jar ... +[info] Done packaging. +[info] Running (fork) sbt.RunFromSourceMain /s/t +Listening for transport dt_socket at address: 5005 +[info] Loading settings from idea.sbt,global-plugins.sbt ... +[info] Loading global plugins from /Users/dnw/.dotfiles/.sbt/1.0/plugins +[info] Loading project definition from /s/t/project +[info] Set current project to t (in build file:/s/t/) +[info] sbt server started at local:///Users/dnw/.sbt/1.0/server/ce9baa494c7598e4d59b/sock +> show baseDirectory +[info] /s/t +> exit +[info] shutting down sbt server +[success] Total time: 19 s, completed 25-Apr-2018 15:04:58 +``` + +Please note that this alternative launcher does _not_ have feature parity with sbt/launcher. (Meta) +contributions welcome! :-D diff --git a/contributing-docs/README.md b/contributing-docs/README.md new file mode 100644 index 000000000..2697a2232 --- /dev/null +++ b/contributing-docs/README.md @@ -0,0 +1,33 @@ +Contributor's guide +=================== + +Contributions are welcome and are greatly appreciated! There are lots of avenues to contribute to the sbt ecosystem, depending on your interests and skill level. + +- Help someone at work or online fix their build problem. +- Answer/ask StackOverflow questions or on [Scala Discord][Discord]. +- Create plugins that extend sbt's features. +- Migrate builds to sbt 2.x. +- Maintain and update [documentation]. +- Garden the issue tracker. +- Report issues. +- Patch the core (send pull requests to code). +- Review pull requests. + +Ideas and proposals +------------------- + +If you have an enhancement idea, or a general discussion, please follow the [sbt 2.0 RFC process](https://eed3si9n.com/sbt-2.0-rfc-process) or start a [Discussion][Discussions]. + +Reporting Issues +---------------- + +See [CONTRIBUTING](../CONTRIBUTINTG.md). + +Patching the core (sending pull request) +---------------------------------------- + +See [pull request](01_pull_request.md) + + [documentation]: https://github.com/sbt/website + [Discord]: https://discord.com/invite/scala + [Discussions]: https://github.com/sbt/sbt/discussions