Understanding SBT Architecture

Build Definition and Task Model

SBT uses a declarative and functional DSL in build.sbt and project/*.scala files to define settings, tasks, and project structure. Tasks are lazily evaluated and support dependency chaining via :=, +=, and .dependsOn().

Incremental Compilation and Zinc

SBT uses Zinc for incremental compilation. Dependency changes or unstable classpaths can cause cache invalidation, resulting in full recompilation instead of partial updates.

Common Symptoms

  • Builds stall or run out of memory on large projects
  • Dependency resolution fails with evicted or version conflict warnings
  • Incremental compilation behaves inconsistently
  • Plugins cause version incompatibility or resolution loops
  • Cross-building to Scala versions fails or produces wrong artifacts

Root Causes

1. Transitive Dependency Conflicts

Unresolved dependency trees cause version eviction or unexpected API breakages. Libraries like Akka or Play often pull in conflicting transitive dependencies.

2. Misconfigured Memory and GC Settings

Default JVM settings in sbt or sbt-launch.jar are insufficient for large projects. GC pauses or heap exhaustion can stall builds or fail Zinc compilation.

3. Plugin Version Incompatibility

Plugins must match both the SBT version and the Scala version of the build definition (usually 2.12). A plugin compiled for another version causes classpath errors or crashes.

4. Scope Leakage Across Aggregates

Settings or dependencies defined in the wrong scope (e.g., using Compile instead of Test) can bleed across aggregates, resulting in unexpected compilation behavior or test failures.

5. Incorrect or Incomplete Cross-Version Settings

Cross-building for multiple Scala versions without setting crossScalaVersions and ++ correctly leads to partial builds or artifact mismatches in CI/CD pipelines.

Diagnostics and Monitoring

1. Use evicted and dependencyTree Commands

Run sbt evicted and sbt dependencyTree (with sbt-dependency-graph plugin) to analyze conflicting versions and identify root causes of resolution failures.

2. Inspect Compilation Cache Logs

Enable verbose logging with -debug or -Dsbt.log.noformat=true to monitor Zinc behavior and determine when incremental compilation is bypassed.

3. Monitor JVM Memory Use

Use jvisualvm, jstat, or top to monitor heap allocation. Adjust SBT_OPTS to increase heap size and tune GC for longer builds.

4. Validate Plugin Compatibility

Cross-check plugin compatibility with your sbt version at https://index.scala-lang.org or examine plugin source for published SBT/Scala cross-builds.

5. Use CI Logs to Detect Cross-Version Failures

Parse CI logs for scalaVersion mismatches or missing artifact uploads in cross-building workflows.

Step-by-Step Fix Strategy

1. Resolve Dependency Conflicts

Use dependencyOverrides or excludeDependencies to force known-good versions. Avoid transitive resolution ambiguity by explicitly declaring key dependencies.

2. Tune JVM Options

SBT_OPTS="-Xmx4G -XX:+UseG1GC -Dsbt.supershell=false"

Apply these settings in shell profiles or CI scripts to stabilize large builds and improve responsiveness.

3. Align Plugin Versions with SBT Version

Ensure that plugins in project/plugins.sbt match your SBT version. Use addSbtPlugin("org.xyz" % "plugin" % "version") with correct cross-version suffix.

4. Audit and Refactor Build Scopes

Place settings inside appropriate configurations using inConfig(Test) or Test / libraryDependencies. Avoid global settings unless intentional.

5. Enable Full Cross-Version Support

Set crossScalaVersions explicitly in your build.sbt and use ++ 2.12.17 to trigger cross-building correctly.

Best Practices

  • Use Global / onChangedBuildSource := ReloadOnSourceChanges for dev efficiency
  • Always pin Scala and plugin versions explicitly
  • Separate test, compile, and integration scopes with unique configurations
  • Use Compile / compileIncremental only when required; avoid over-customizing incremental settings
  • Document sbt version and plugin versions using version.sbt and plugins.sbt

Conclusion

SBT is a flexible and powerful tool, but scaling it for large codebases requires fine-grained control over dependency resolution, build configuration, and JVM behavior. By enforcing strict version management, optimizing memory usage, and structuring modular builds, teams can avoid the most common pitfalls and ensure efficient CI/CD pipelines using SBT.

FAQs

1. Why is SBT re-compiling everything even after a small change?

Likely due to unstable classpaths, timestamp mismatches, or changing source generators. Check for macros or codegen that resets timestamps.

2. How do I debug dependency eviction issues?

Use the evicted and dependencyTree commands with the sbt-dependency-graph plugin to visualize dependency conflicts and resolution strategy.

3. Can I use different SBT versions per project?

Yes, by using a per-project sbt-launch.jar or setting sbt.version in project/build.properties.

4. How can I speed up SBT builds?

Use --client mode with sbt 1.4+, increase JVM memory, and cache dependencies. Avoid unnecessary clean between runs.

5. Why do my plugins break when upgrading SBT?

Plugins are compiled against specific SBT and Scala versions. Check plugin docs and upgrade using compatible versions via addSbtPlugin.