Understanding Broccoli's Build Architecture

Core Concepts and DAG-Based Pipelines

Broccoli uses a directed acyclic graph (DAG) to manage build pipelines. Each plugin transforms input trees to output trees. The build process is deterministic and optimized for caching, but:

  • Plugins operate in isolation and must not mutate inputs
  • Tree merging and funneling must be precise to avoid overwrites
  • Output is recomputed only when inputs change (via hashing)

Large enterprise builds may include dozens of plugins and intermediate trees, making it difficult to track where transformations go wrong.

Plugin Race Conditions

Some plugins (e.g., `broccoli-funnel`, `broccoli-merge-trees`) are prone to race conditions when:

  • Multiple plugins write to the same destination directory
  • Output trees are improperly merged, causing silent overwrites
  • Funnel configurations overlap or omit critical file types

Common Symptoms and Pitfalls

Symptoms of Asset Loss

  • Static assets (SVGs, fonts) randomly disappear from `dist/`
  • Incremental builds work, but full rebuilds fail silently
  • Files appear in `tmp/` but not in final output

Why These Errors Are Rarely Detected

Broccoli does not warn when output trees collide unless explicitly configured. Developers assume correctness if local builds succeed, but:

  • CI environments may flush caches, revealing missing assets
  • Builds with slightly different dependency versions may trigger plugin misbehavior

Root Cause Analysis

Improper Funnel Globs

Funnel plugins are often misconfigured with overly strict `include` or `exclude` globs. For example, omitting a trailing slash can change behavior:

module.exports = funnel('public', {
  include: ['images/**']
});

This may exclude `.svg` files in nested folders unintentionally.

Merge Tree Clobbering

Using `broccoli-merge-trees` without the `overwrite: true` flag may cause last-write-wins behavior without warnings:

module.exports = mergeTrees([treeA, treeB], { overwrite: true });

Without overwrite, duplicate filenames can silently disappear from output.

Step-by-Step Troubleshooting Guide

1. Reproduce in a Cold Environment

Start with a clean `tmp/` and `dist/` folder. Use:

rm -rf tmp dist
BROCCOLI_ENV=production ember build

This ensures that no cached trees mask the issue.

2. Inspect Intermediate Trees

Use the `broccoli-viz` tool to visualize build DAGs:

ember build --verbose
BROCCOLI_VIZ=1 broccoli-viz

Look for missing branches or nodes with 0 output files.

3. Trace Funnel Configurations

Audit each `funnel()` usage in `ember-cli-build.js` or custom Brocfile:

  • Ensure globs match expected files
  • Confirm `destDir` paths do not conflict

4. Test MergeTree Overwrite Behavior

Temporarily enable `overwrite: true` and log merged files:

const merged = mergeTrees([tree1, tree2], { overwrite: true });
console.log('Merging trees with overwrite:', merged);

5. Use Watchman to Detect FS-Level Conflicts

File watchers like Watchman can trace when files are removed or overridden during build cycles. Configure Watchman with:

watchman watch-project .
watchman -- trigger . build *.svg

Best Practices for Production-Grade Builds

Declare Unique Output Paths

Ensure all plugins output to isolated directories. Avoid writing to shared roots like `/assets` unless post-merge cleanup is enforced.

Centralize Funnel Logic

Consolidate funnel plugins early in the build pipeline to avoid redundant or conflicting globs.

Implement Build Tests

Write custom tests that assert presence of critical files in `dist/` before deploying:

const fs = require('fs');
if (!fs.existsSync('dist/assets/logo.svg')) {
  throw new Error('Missing logo asset');
}

Version Pinning of Plugins

Lock plugin versions in `package.json` to avoid breakages from upstream updates that alter default behaviors (e.g., `broccoli-funnel` v3+).

Conclusion

Broccoli's deterministic build architecture is powerful but unforgiving in large-scale applications. Silent asset drops and inconsistent builds typically stem from funnel misconfigurations or merge collisions. Tools like `broccoli-viz`, Watchman, and build validators are essential to detect and resolve such issues. Long-term stability requires architectural rigor—isolated output paths, clean funnel globs, and version-pinned plugins—to ensure that enterprise-grade applications remain reliable through scale and change.

FAQs

1. Why do assets sometimes appear locally but disappear in CI builds?

Local builds often benefit from cached trees and relaxed race conditions. CI environments start from scratch, exposing plugin order or output path conflicts not seen locally.

2. Can Broccoli detect duplicate output files automatically?

No. Unless `overwrite: true` is explicitly set in `mergeTrees`, duplicate files will result in silent conflict where only one version is preserved without warning.

3. What is the best way to debug Broccoli's DAG?

Use `broccoli-viz` or Ember CLI's verbose mode to generate a visual graph of the build tree. It helps identify missing nodes, circular dependencies, or unexpectedly empty trees.

4. Is there a linter for Broccoli funnel or merge usage?

Not natively, but you can write ESLint custom rules or build scripts to enforce best practices like unique `destDir` values and valid globs.

5. How do I ensure deterministic builds with Broccoli?

Pin plugin versions, avoid dynamic input trees, and enforce isolated output directories. Include post-build assertions to confirm expected files are present before packaging artifacts.