Upgrade to Pro — share decks privately, control downloads, hide ads and more …

.NET Day 2023: Beyond simple benchmarks—A pract...

dotnetday
September 01, 2023

.NET Day 2023: Beyond simple benchmarks—A practical guide to optimizing code with BenchmarkDotNet

It is vital for code executed at scale to perform well. It is crucial to ensure performance optimizations actually make the code faster. Luckily, we have powerful tools which help—BenchmarkDotNet is a .NET library for benchmarking optimizations, with plenty of simple examples to help get started. In most systems, the code we need to optimize is rarely simple. It contains assumptions we need to discover before we even know what to improve. The code is hard to isolate. It has dependencies, which may or may not be relevant to optimization. And even when we've decided what to optimize, it's hard to reliably benchmark the before and after. Only measurement can tell us if our changes actually make things faster. Without them, we could even make things slower, without even realizing. Understanding how to create benchmarks is the tip of the iceberg. In this talk, you'll also learn how to identify what to change, how to isolate code for benchmarking, and more. You'll leave with a toolkit of succinct techniques and the confidence to go ahead and optimize your code.

dotnetday

September 01, 2023
Tweet

More Decks by dotnetday

Other Decks in Technology

Transcript

  1. BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH

    BENCHMARK.NET  | ✉ |  danielmarbach [email protected] Daniel Marbach
  2. Microsoft Teams’ Infrastructure and Azure Communication Services’ Journey to .NET

    6 "We were able to see Azure Compute cost reduction of up to 50% per month, on average we observed 24% monthly cost reduction after migrating to .NET 6. The reduction in cores reduced Azure spend by 24%."
  3. BE CURIOUS.... UNDERSTAND THE CONTEXT How is this code going

    to be executed at scale, and what would the memory characteristics be (gut feeling) Are there simple low-hanging fruits I can apply to accelerate this code? Are there things I can move away from the hot path by simply restructuring a bit my code? What part is under my control and what isn't really? What optimizations can I apply, and when should I stop?
  4. THE PERFORMANCE LOOP Profile at least CPU and memory using

    a profiling harness Improve parts of the hot path Benchmark and compare Profile improvements again with the harness and make adjustments where necessary Ship and focus your attention to other parts
  5. ASP.NET CORE MIDDLEWARE public class RequestCultureMiddleware { _next = next;

    public async Task InvokeAsync(HttpContext context) { await _next(context); 1 private readonly RequestDelegate _next; 2 3 public RequestCultureMiddleware(RequestDelegate next) { 4 5 } 6 7 8 // Do work that does something before 9 10 // Do work that does something after 11 } 12 } 13
  6. public class Behavior : Behavior<IIncomingLogicalMessageContext> { public override Task Invoke(IIncomingLogicalMessageContext

    context, Func<Task> next) { await next(); 1 2 3 // Do work that does something before 4 5 // Do work that does something after 6 } 7 } 8 BEHAVIORS
  7. THE HARNESS Compiled and executed in Release mode Runs a

    few seconds and keeps overhead minimal Disabled Tiered JIT <TieredCompilation>false</TieredCompilation> Emits full symbols <DebugType>pdbonly</DebugType <DebugSymbols>true</DebugSymbols> var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization<JsonSerializer>(); var transport = endpointConfiguration.UseTransport<MsmqTransport>(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence<InMemoryPersistence>(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit <enter>."); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UseSerialization<JsonSerializer>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 endpointConfiguration.UsePersistence<InMemoryPersistence>(); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 var tasks = new List<Task>(1000); 14 for (int i = 0; i < 1000; i++) 15 { 16 tasks.Add(endpointInstance.Publish(new MyEvent())); 17 } 18 await Task.WhenAll(tasks); 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var tasks = new List<Task>(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); var endpointConfiguration = new EndpointConfiguration("PublishSample"); 1 endpointConfiguration.UseSerialization<JsonSerializer>(); 2 var transport = endpointConfiguration.UseTransport<MsmqTransport>(); 3 transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); 4 endpointConfiguration.UsePersistence<InMemoryPersistence>(); 5 endpointConfiguration.EnableInstallers(); 6 endpointConfiguration.SendFailedMessagesTo("error"); 7 8 var endpointInstance = await Endpoint.Start(endpointConfiguration); 9 10 Console.WriteLine("Attach the profiler and hit <enter>."); 11 Console.ReadLine(); 12 13 14 15 16 17 18 19 20 Console.WriteLine("Publish 1000 done. Get a snapshot"); 21 Console.ReadLine(); 22 var endpointConfiguration = new EndpointConfiguration("PublishSample"); endpointConfiguration.UseSerialization<JsonSerializer>(); var transport = endpointConfiguration.UseTransport<MsmqTransport>(); transport.Routing().RegisterPublisher(typeof(MyEvent), "PublishSample"); endpointConfiguration.UsePersistence<InMemoryPersistence>(); endpointConfiguration.EnableInstallers(); endpointConfiguration.SendFailedMessagesTo("error"); var endpointInstance = await Endpoint.Start(endpointConfiguration); Console.WriteLine("Attach the profiler and hit <enter>."); Console.ReadLine(); var tasks = new List<Task>(1000); for (int i = 0; i < 1000; i++) { tasks.Add(endpointInstance.Publish(new MyEvent())); } await Task.WhenAll(tasks); Console.WriteLine("Publish 1000 done. Get a snapshot"); Console.ReadLine(); 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public class MyEventHandler : IHandleMessages<MyEvent> { public Task Handle(MyEvent message, IMessageHandlerContext context) { Console.WriteLine("Event received"); return Task.CompletedTask; } }  Profiling the pipeline
  8.  10X faster execution with compiled expression trees  How

    we achieved 5X faster pipeline execution by removing closure allocations IMPROVING go.particular.net/dotnetday-2023-pipeline
  9.  Benchmarking the pipeline Copy and paste relevant code Adjust

    it to the bare essentials to create a controllable environment EXTRACT CODE
  10. Trim down to relevant behaviors Replaced dependency injection container with

    creating relevant classes Replaced IO-operations with completed tasks EXTRACT CODE  Benchmarking the pipeline
  11. Get started with small steps Culture change takes time Make

    changes gradually PERFORMANCE CULTURE
  12. [ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public

    int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline
  13. [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations

    = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42  Benchmarking the pipeline
  14. [Params(10, 20, 40)] public int PipelineDepth { get; set; }

    for (int i = 0; i < PipelineDepth; i++) for (int i = 0; i < PipelineDepth; i++) [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 5 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [ShortRunJob] [MemoryDiagnoser] 1 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 [Benchmark(Baseline = true)] 33 public async Task Before() { 34 await pipelineBeforeOptimizations.Invoke(behaviorContext); 35 } 36 37 [Benchmark] 38 public async Task After() { 39 await pipelineAfterOptimizations.Invoke(behaviorContext); 40 } 41 } 42 [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } [ShortRunJob] 1 [MemoryDiagnoser] 2 public class PipelineExecution { 3 4 [Params(10, 20, 40)] 5 public int PipelineDepth { get; set; } 6 7 8 [GlobalSetup] 9 public void SetUp() { 10 behaviorContext = new BehaviorContext(); 11 12 pipelineModificationsBeforeOptimizations = new PipelineModifications(); 13 for (int i = 0; i < PipelineDepth; i++) 14 { 15 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 16 typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); 17 } 18 19 pipelineModificationsAfterOptimizations = new PipelineModifications(); 20 for (int i = 0; i < PipelineDepth; i++) 21 { 22 pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), 23 typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); 24 } 25 26 pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), 27 pipelineModificationsBeforeOptimizations); 28 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 29 pipelineModificationsAfterOptimizations); 30 } 31 32 33 34 35 36 37 38 39 40 41 } 42 [ShortRunJob] [MemoryDiagnoser] public class PipelineExecution { [Params(10, 20, 40)] public int PipelineDepth { get; set; } [GlobalSetup] public void SetUp() { behaviorContext = new BehaviorContext(); pipelineModificationsBeforeOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BaseLineBehavior), i.ToString(), b => new BaseLineBehavior())); } pipelineModificationsAfterOptimizations = new PipelineModifications(); for (int i = 0; i < PipelineDepth; i++) { pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(i.ToString(), typeof(BehaviorOptimization), i.ToString(), b => new BehaviorOptimization())); } pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { await pipelineBeforeOptimizations.Invoke(behaviorContext); } [Benchmark] public async Task After() { await pipelineAfterOptimizations.Invoke(behaviorContext); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42  Benchmarking the pipeline
  15. Single Responsibility Principle No side effects Prevents dead code elimination

    Delegates heavy lifting to the framework Is explicit No implicit casting No var Avoid running any other resource- heavy processes while benchmarking PRACTICES  Benchmarking the pipeline
  16. Benchmarking is really hard BenchmarkDotNet will protect you from the

    common pitfalls because it does all the dirty work for you
  17. [ShortRunJob] [MemoryDiagnoser] public class Step1_PipelineWarmup { // rest almost the

    same [Benchmark(Baseline = true)] public BaseLinePipeline<IBehaviorContext> Before() { var pipelineBeforeOptimizations = new BaseLinePipeline<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); return pipelineBeforeOptimizations; } [Benchmark] public PipelineOptimization<IBehaviorContext> After() { var pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); return pipelineAfterOptimizations; } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  Benchmarking the pipeline
  18. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp()

    { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdId.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext).ConfigureAwait(false); } catch (InvalidOperationException) { } } class Throwing : Behavior<IBehaviorContext> { public override Task Invoke(IBehaviorContext context, Func<Task> next) { throw new InvalidOperationException(); } } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47  Benchmarking the pipeline
  19. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp()

    { ... var stepdId = PipelineDepth + 1; pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); ... pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), pipelineModificationsBeforeOptimizations); pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext> (null, new SettingsHolder(), pipelineModificationsAfterOptimizations); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 pipelineModificationsBeforeOptimizations.Additions.Add(RegisterStep.Create(stepd Id.ToString(), typeof(Throwing), "1", b => new Throwing())); pipelineModificationsAfterOptimizations.Additions.Add(RegisterStep.Create(stepdI d.ToString(), typeof(Throwing), "1", b => new Throwing())); [ShortRunJob] 1 [MemoryDiagnoser] 2 public class Step2_PipelineException { 3 [GlobalSetup] 4 public void SetUp() { 5 ... 6 var stepdId = PipelineDepth + 1; 7 8 9 ... 10 11 12 pipelineBeforeOptimizations = new Step1.PipelineOptimization<IBehaviorContext>(null, new SettingsHolder(), 13 pipelineModificationsBeforeOptimizations); 14 pipelineAfterOptimizations = new PipelineOptimization<IBehaviorContext> (null, new SettingsHolder(), 15 pipelineModificationsAfterOptimizations); 16 } 17 } 18  Benchmarking the pipeline
  20. [ShortRunJob] [MemoryDiagnoser] public class Step2_PipelineException { [GlobalSetup] public void SetUp()

    { ... } [Benchmark(Baseline = true)] public async Task Before() { try { await pipelineBeforeOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } [Benchmark] public async Task After() { try { await pipelineAfterOptimizations.Invoke(behaviorContext); } catch (InvalidOperationException) { } } ... } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31  Benchmarking the pipeline
  21. PREVENTING REGRESSIONS C:\Projects\performance\src\tools\ResultsComparer> dotnet run --base "C:\results\before" --diff "C:\results\after" --threshold

    2% 1 2 Guidance Tool Preventing Regressions ResultComparer C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\before" 1 2 C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net8.0 \ --artifacts "C:\results\after" 1 2
  22. "Two subsequent builds on the same revision can have ranges

    of 1.5..2 seconds and 12..36 seconds. CPU-bound benchmarks are much more stable than Memory/Disk-bound benchmarks, but the “average” performance levels still can be up to three times different across builds." Andrey Akinshin - Performance stability of GitHub Actions
  23. BEYOND SIMPLE BENCHMARKS A PRACTICAL GUIDE TO OPTIMIZING CODE WITH

    BENCHMARK.NET  | ✉ |  danielmarbach [email protected] Daniel Marbach github.com/danielmarbach/BeyondSimpleBenchmarks Use the performance loop to improve your code where it matters Combine it with profiling to observe how the small changes add up Optimize until you hit a diminishing point of return You'll learn a ton about potential improvements for a new design