Skills build-perf-diagnostics

Performance Analysis Methodology

1. Generate a binlog:

   dotnet build /bl:{} -m

2. Replay to diagnostic log with performance summary:

   dotnet msbuild build.binlog -noconlog -fl -flp:v=diag;logfile=full.log;performancesummary
   

3. Read the performance summary (at the end of

   full.log

   ):

   grep "Target Performance Summary\|Task Performance Summary" -A 50 full.log
   

4. Find expensive targets and tasks: The PerformanceSummary section lists all targets/tasks sorted by cumulative time
5. Check for node utilization: grep for scheduling and node messages

   grep -i "node.*assigned\|building with\|scheduler" full.log | head -30
   

6. Check analyzers: grep for analyzer timing

   grep -i "analyzer.*elapsed\|Total analyzer execution time\|CompilerAnalyzerDriver" full.log
   

Key Metrics and Thresholds

• Build duration: what's "normal" — small project <10s, medium <60s, large <5min
• Node utilization: ideal is >80% active time across nodes. Low utilization = serialization bottleneck
• Single target domination: if one target is >50% of build time, investigate
• Analyzer time vs compile time: analyzers should be <30% of Csc task time. If higher, consider removing expensive analyzers
• RAR time: ResolveAssemblyReference >5s is concerning. >15s is pathological

Common Bottlenecks

1. ResolveAssemblyReference (RAR) Slowness

• Symptoms: RAR taking >5s per project
• Root causes: too many assembly references, network-based reference paths, large assembly search paths
• Fixes: reduce reference count, use

  <DesignTimeBuild>false</DesignTimeBuild>

  for RAR-heavy analysis, set

  <ResolveAssemblyReferencesSilent>true</ResolveAssemblyReferencesSilent>

  for diagnostic
• Advanced:

  <DesignTimeBuild>

  and

  <ResolveAssemblyWarnOrErrorOnTargetArchitectureMismatch>

• Key insight: RAR runs unconditionally even on incremental builds because users may have installed targeting packs or GACed assemblies (see dotnet/msbuild#2015). With .NET Core micro-assemblies, the reference count is often very high.
• Reduce transitive references: Set

  <DisableTransitiveProjectReferences>true</DisableTransitiveProjectReferences>

  to avoid pulling in the full transitive closure (note: projects may need to add direct references for any types they consume). Use

  ReferenceOutputAssembly="false"

  on ProjectReferences that are only needed at build time (not API surface). Trim unused PackageReferences.

2. Roslyn Analyzers and Source Generators

• Symptoms: Csc task takes much longer than expected for file count (>2× clean compile time)
• Diagnosis: Check the Task Performance Summary in the replayed log for Csc task time; grep for analyzer timing messages; compare Csc duration with and without analyzers (

  /p:RunAnalyzers=false

  )
• Fixes:
  • Conditionally disable in dev:

    <RunAnalyzers Condition="'$(ContinuousIntegrationBuild)' != 'true'">false</RunAnalyzers>

  • Per-configuration:

    <RunAnalyzers Condition="'$(Configuration)' == 'Debug'">false</RunAnalyzers>

  • Code-style only:

    <EnforceCodeStyleInBuild Condition="'$(ContinuousIntegrationBuild)' == 'true'">true</EnforceCodeStyleInBuild>

  • Remove genuinely redundant analyzers from inner loop
  • Severity config in .editorconfig for less critical rules
• Key principle: Preserve analyzer enforcement in CI. Never just "remove" analyzers — configure them conditionally.
• GlobalPackageReference: Analyzers added via

  GlobalPackageReference

  in

  Directory.Packages.props

  apply to ALL projects. Consider if test projects need the same analyzer set as production code.
• EnforceCodeStyleInBuild: When set to

  true

  in

  Directory.Build.props

  , forces code-style analysis on every build. Should be conditional on CI environment (

  ContinuousIntegrationBuild

  ) to avoid slowing dev inner loop.

3. Serialization Bottlenecks (Single-threaded targets)

• Symptoms: Performance summary shows most build time concentrated in a single project; diagnostic log shows idle nodes while one works
• Common culprits: targets without proper dependency declaration, single project on critical path
• Fixes: split large projects, optimize the critical path project, ensure proper

  BuildInParallel

4. Excessive File I/O (Copy tasks)

• Symptoms: Copy task shows high aggregate time
• Root causes: copying thousands of files, copying across network drives, Copy task unintentionally running once per item (per-file) instead of as a single batch (see dotnet/msbuild#12884)
• Fixes: use hardlinks (

  <CreateHardLinksForCopyFilesToOutputDirectoryIfPossible>true</CreateHardLinksForCopyFilesToOutputDirectoryIfPossible>

  ), reduce CopyToOutputDirectory items, use

  <UseCommonOutputDirectory>true</UseCommonOutputDirectory>

  when appropriate, set

  <SkipCopyUnchangedFiles>true</SkipCopyUnchangedFiles>

  , consider

  --artifacts-path

  (.NET 8+) for centralized output layout
• Dev Drive: On Windows, switching to a Dev Drive (ReFS with copy-on-write and reduced Defender scans) can significantly reduce file I/O overhead for Copy-heavy builds. Recommend for both dev machines and self-hosted CI agents.

5. Evaluation Overhead

• Symptoms: build starts slow before any compilation
• Root causes: complex Directory.Build.props, wildcard globs scanning large directories, NuGetSdkResolver overhead (adds 180-400ms per project evaluation even when restored — see dotnet/msbuild#4025)
• Fixes: reduce Directory.Build.props complexity, use

  <EnableDefaultItems>false</EnableDefaultItems>

  for legacy projects with explicit file lists, avoid NuGet-based SDK resolvers if possible
• See:

  eval-performance

  skill for detailed guidance

6. NuGet Restore in Build

• Symptoms: restore runs every build even when unnecessary
• Fixes:
  • Separate restore from build:

    dotnet restore

    then

    dotnet build --no-restore

  • Enable static graph evaluation:

    <RestoreUseStaticGraphEvaluation>true</RestoreUseStaticGraphEvaluation>

    in Directory.Build.props — can save significant time in large builds (results are workload-dependent)

7. Large Project Count and Graph Shape

• Symptoms: many small projects, each takes minimal time but overhead adds up; deep dependency chains serialize the build
• Consider: project consolidation, or use

  /graph

  mode for better scheduling
• Graph shape matters: a wide dependency graph (few levels, many parallel branches) builds faster than a deep one (many levels, serialized). Refactoring from deep to wide can yield significant improvements in both clean and incremental build times.
• Actions: look for unnecessary project dependencies, consider splitting a bottleneck project into two, or merging small leaf projects

Using Binlog Replay for Performance Analysis

Step-by-step workflow using text log replay:

1. Replay with performance summary:

   dotnet msbuild build.binlog -noconlog -fl -flp:v=diag;logfile=full.log;performancesummary
   

2. Read target/task performance summaries (at the end of

   full.log

   ):

   grep "Target Performance Summary\|Task Performance Summary" -A 50 full.log
   

   This shows all targets and tasks sorted by cumulative time — equivalent to finding expensive targets/tasks.
3. Find per-project build times:

   grep "done building project\|Project Performance Summary" full.log
   

4. Check parallelism (multi-node scheduling):

   grep -i "node.*assigned\|RequiresLeadingNewline\|Building with" full.log | head -30
   

5. Check analyzer overhead:

   grep -i "Total analyzer execution time\|analyzer.*elapsed\|CompilerAnalyzerDriver" full.log
   

6. Drill into a specific slow target:

   grep 'Target "CoreCompile"\|Target "ResolveAssemblyReferences"' full.log
   

Quick Wins Checklist

• Use

  /maxcpucount

  (or

  -m

  ) for parallel builds
• Separate restore from build (

  dotnet restore

  then

  dotnet build --no-restore

  )
• Enable static graph restore (

  <RestoreUseStaticGraphEvaluation>true</RestoreUseStaticGraphEvaluation>

  )
• Enable hardlinks for Copy (

  <CreateHardLinksForCopyFilesToOutputDirectoryIfPossible>true</CreateHardLinksForCopyFilesToOutputDirectoryIfPossible>

  )
• Disable analyzers conditionally in dev inner loop:

  <RunAnalyzers Condition="'$(ContinuousIntegrationBuild)' != 'true'">false</RunAnalyzers>

• Enable reference assemblies (

  <ProduceReferenceAssembly>true</ProduceReferenceAssembly>

  )
• Check for broken incremental builds (see

  incremental-build

  skill)
• Check for bin/obj clashes (see

  check-bin-obj-clash

  skill)
• Use graph build (

  /graph

  ) for multi-project solutions
• Use

  --artifacts-path

  (.NET 8+) for centralized output layout
• Enable Dev Drive (ReFS) on Windows dev machines and self-hosted CI

Impact Categorization

When reporting findings, categorize by impact to help prioritize fixes:

• 🔴 HIGH IMPACT (do first): Items consuming >10% of total build time, or a single target >50% of build time
• 🟡 MEDIUM IMPACT: Items consuming 2-10% of build time
• 🟢 QUICK WINS: Easy changes with modest impact (e.g., property flags in Directory.Build.props)