Turbocharged: Writing High Performance C# and .NET Code (90 mins)

Transcript

www.stevejgordon.co.uk @stevejgordon @stevejgordon https://stevejgordon.co.uk https://www.meetup.com/dotnetsoutheast Slides: http://bit.ly/highperfdotnet

None

@stevejgordon www.stevejgordon.co.uk • What is performance? • Measuring application and

code performance • Span<T>, ReadOnlySpan<T> and Memory<T> • ArrayPool • System.IO.Pipelines and ReadOnlySequence<T> • .NET Core 3.0 JSON APIs

@stevejgordon www.stevejgordon.co.uk Execution Time Throughput Memory Allocations

@stevejgordon www.stevejgordon.co.uk “We should forget about small efficiencies, say about

97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” http://web.archive.org/web/20130731202547/http://pplab.snu.ac.kr/courses/adv_pl05/papers/p261-knuth.pdf Donald Knuth, 1974, Structured Programming with go to Statements

None

READABILITY PERFORMANCE

@stevejgordon Measure Optimise Measure Optimise OPTIMISATION CYCLE

@stevejgordon www.stevejgordon.co.uk • Visual Studio Diagnostic Tools (debugging) • Visual

Studio Profiling / PerfView / dotTrace / dotMemory • ILSpy / JustDecompile / dotPeek • Production metrics and monitoring

BENCHMARK.NET https://benchmarkdotnet.org

@stevejgordon www.stevejgordon.co.uk namespace BenchmarkExample { public class Program { public

static void Main(string[] args) => _ = BenchmarkRunner.Run<NameParserBenchmarks>(); } [MemoryDiagnoser] public class NameParserBenchmarks { private const string FullName = "Steve J Gordon"; private static readonly NameParser Parser = new NameParser(); [Benchmark] public void GetLastName() { Parser.GetLastName(FullName); } } }

@stevejgordon www.stevejgordon.co.uk namespace BenchmarkExample { public class Program { public

@stevejgordon www.stevejgordon.co.uk // * Summary * BenchmarkDotNet=v0.11.3, OS=Windows 10.0.17134.706 (1803/April2018Update/Redstone4)

Intel Core i7-6700 CPU 3.40GHz (Skylake), 1 CPU, 8 logical and 4 physical cores .NET Core SDK=3.0.100-preview3-010410 [Host] : .NET Core 2.2.3 (CoreCLR 4.6.27414.05, CoreFX 4.6.27414.05), 64bit RyuJIT DefaultJob : .NET Core 2.2.3 (CoreCLR 4.6.27414.05, CoreFX 4.6.27414.05), 64bit RyuJIT | Gen 0 | Gen 1 | Gen 2 | Allocated | Method | Mean | Error | StdDev | Median | /1k Op | /1k Op | /1k Op | Memory/Op | ------------ |-----------:|-----------:|-----------:|-----------:|-------:|-------:|-------:|----------:| GetLastName | 163.18 ns | 3.1903 ns | 4.2590 ns | 161.87 ns | 0.0379 | - | - | 160 B | (1 / 0.0379) x 1000 = 26,385.2 operations before Gen 0 collection.

@stevejgordon www.stevejgordon.co.uk • System.Memory package. Built into .NET Core 2.1.

• Read/write 'view’ over a contiguous region of memory • Heap (Managed objects) – e.g. Arrays, Strings • Stack (via stackalloc) • Native/Unmanaged (P/Invoke) • Index / Iterate to modify the memory within the Span • Almost no overhead

@stevejgordon www.stevejgordon.co.uk Pointer Length Span<T>

Span<T>.Slice Slicing a Span is a constant time/cost operation –

O(1) Int[] myArray = new int[9] Span<int> span1 = myArray.AsSpan() Span<int> span2 = span1.Slice(start: 2, length: 5) Int[9] 0 1 2 3 4 5 6 7 8 0 1 2 3 4

OPTIMISING SOME CODE

Requirement: We need a method, that takes an array and

returns ¼ of its elements, starting from the middle element.

myArray.Skip(Size / 2). Take(Size / 4).ToArray();

Requirement 2: Turbocharge it and prosper!!

@stevejgordon www.stevejgordon.co.uk [MemoryDiagnoser] public class ArrayBenchmarks { private int[] _myArray;

[Params(100, 1000, 10000)] public int Size { get; set; } [GlobalSetup] public void Setup() { _myArray = new int[Size]; for (var i = 0; i < Size; i++) _myArray[i] = i; } // MORE CODE COMING RIGHT UP!!...

@stevejgordon www.stevejgordon.co.uk [MemoryDiagnoser] public class ArrayBenchmarks { private int[] _myArray;

@stevejgordon www.stevejgordon.co.uk [MemoryDiagnoser] public class ArrayBenchmarks { // SETUP METHODS

UP HERE! ... [Benchmark(Baseline = true)] public int[] Original() => _myArray.Skip(Size / 2).Take(Size / 4).ToArray(); ... }

@stevejgordon www.stevejgordon.co.uk | Method | Size | Mean | Ratio

| Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |------ |---------------:|------:|-------:|-------:|------:|----------:| | Original | 100 | 154.9018 ns | 1.00 | 0.0534 | - | - | 224 B | | | | | | | | | | | Original | 1000 | 727.2669 ns | 1.00 | 0.2670 | - | - | 1120 B | | | | | | | | | | | Original | 10000 | 7,332.0136 ns | 1.00 | 2.4109 | - | - | 10120 B |

@stevejgordon www.stevejgordon.co.uk [MemoryDiagnoser] public class ArrayBenchmarks { ... [Benchmark] public

int[] ArrayCopy() { var newArray = new int[Size / 4]; Array.Copy(_myArray, Size / 2, newArray, 0, Size / 4); return newArray; } ... }

@stevejgordon www.stevejgordon.co.uk | Method | Size | Mean | Ratio

| Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |------ |---------------:|-------:|-------:|-------:|------:|----------:| | Original | 100 | 154.9018 ns | 1.000 | 0.0534 | - | - | 224 B | | ArrayCopy | 100 | 24.5267 ns | 0.159 | 0.0051 | - | - | 128 B | | | | | | | | | | | Original | 1000 | 727.2669 ns | 1.000 | 0.2670 | - | - | 1120 B | | ArrayCopy | 1000 | 104.7282 ns | 0.142 | 0.1627 | - | - | 1024 B | | | | | | | | | | | Original | 10000 | 7,332.0136 ns | 1.000 | 2.4109 | - | - | 10120 B | | ArrayCopy | 10000 | 801.1695 ns | 0.109 | 1.5917 | - | - | 10024 B |

@stevejgordon www.stevejgordon.co.uk [MemoryDiagnoser] public class ArrayBenchmarks { ... [Benchmark] public

Span<int> Span() => _myArray.AsSpan().Slice(Size / 2, Size / 4); ... }

@stevejgordon www.stevejgordon.co.uk | Method | Size | Mean | Ratio

| Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |------ |---------------:|-------:|-------:|-------:|------:|----------:| | Original | 100 | 154.9018 ns | 1.000 | 0.0534 | - | - | 224 B | | ArrayCopy | 100 | 24.5267 ns | 0.159 | 0.0051 | - | - | 128 B | | Span | 100 | 0.9233 ns | 0.006 | - | - | - | - | | | | | | | | | | | Original | 1000 | 727.2669 ns | 1.000 | 0.2670 | - | - | 1120 B | | ArrayCopy | 1000 | 104.7282 ns | 0.142 | 0.1627 | - | - | 1024 B | | Span | 1000 | 0.9016 ns | 0.000 | - | - | - | - | | | | | | | | | | | Original | 10000 | 7,332.0136 ns | 1.000 | 2.4109 | - | - | 10120 B | | ArrayCopy | 10000 | 801.1695 ns | 0.109 | 1.5917 | - | - | 10024 B | | Span | 10000 | 0.9095 ns | 0.000 | - | - | - | - |

@stevejgordon www.stevejgordon.co.uk S ReadOnlySpan<char> t e v e J G

o r d o n ReadOnlySpan<char>.Slice(start: 8) ReadOnlySpan<char> span = "Some string data".AsSpan(); G o r d o n

None

@stevejgordon www.stevejgordon.co.uk public void CalculateFibonacci() { const int arraySize =

20; Span<int> fib = stackalloc int[arraySize]; fib[0] = fib[1] = 1; // Sequence starts with 1 for (int i = 2; i < arraySize; ++i) { // Sum the previous two numbers. fib[i] = fib[i-1] + fib[i-2]; } }

@stevejgordon www.stevejgordon.co.uk public void CalculateFibonacci() { const int arraySize =

@stevejgordon www.stevejgordon.co.uk public Span<int> CalculateFibonacci() { const int arraySize =

20; Span<int> fib = stackalloc int[arraySize]; fib[0] = fib[1] = 1; // Sequence starts with 1 for (int i = 2; i < arraySize; ++i) { // Sum the previous two numbers. fib[i] = fib[i-1] + fib[i-2]; } return fib; }

@stevejgordon www.stevejgordon.co.uk public Span<int> CalculateFibonacci() { const int arraySize =

20; Span<int> fib = stackalloc int[arraySize]; fib[0] = fib[1] = 1; // Sequence starts with 1 for (int i = 2; i < arraySize; ++i) { // Sum the previous two numbers. fib[i] = fib[i-1] + fib[i-2]; } return fib; // CS8325 - Cannot use local 'fib' in this context // because it may expose referenced variables // outside of their declaration scope }

@stevejgordon www.stevejgordon.co.uk • It's a stack only Value Type (ref

struct) – Cannot live on the heap • Requires C# 7.2+ for ref struct feature • Cannot be boxed • Cannot be a field in a class or standard (non ref) struct • Cannot be used as an argument or local variable inside async methods • Cannot be captured by lambda expressions

@stevejgordon www.stevejgordon.co.uk • Similar to Span<T> but can live on

the heap • A readonly struct but not a ref struct • Slightly slower to slice into Memory<T> • Can call Span property to get a span within a method

@stevejgordon www.stevejgordon.co.uk // CS4012 Parameters or locals of type 'Span<byte>'

cannot be declared // in async methods or lambda expressions. private async Task SomethingAsync(Span<byte> data) { ... // Would be nice to do something with the Span here await Task.Delay(1000); }

@stevejgordon www.stevejgordon.co.uk private async Task SomethingAsync(Memory<byte> data) { ... await

Task.Delay(1000); }

@stevejgordon www.stevejgordon.co.uk private async Task SomethingAsync(Memory<byte> data) { Memory<byte> dataSliced

= data.Slice(0, 100); await Task.Delay(1000); }

@stevejgordon www.stevejgordon.co.uk private async Task SomethingAsync(Memory<byte> data) { Memory<byte> dataSliced

= data.Slice(0, 100); await Task.Delay(1000); } private void SomethingNotAsync(Span<byte> data) { // some code }

@stevejgordon www.stevejgordon.co.uk private async Task SomethingAsync(Memory<byte> data) { // CS4012

Parameters or locals of type 'Span<byte>' cannot be declared // in async methods or lambda expressions. var span = data.Span.Slice(1); SomethingNotAsync(span); await Task.Delay(1000); } private void SomethingNotAsync(Span<byte> data) { // some code }

@stevejgordon www.stevejgordon.co.uk private async Task SomethingAsync(Memory<byte> data) { SomethingNotAsync(data.Span.Slice(1)); await

Task.Delay(1000); } private void SomethingNotAsync(Span<byte> data) { // some code }

@stevejgordon BREAK TIME!

@stevejgordon www.stevejgordon.co.uk Microservice which: 1. Reads SQS message 2. Deserialise

the JSON message 3. Stores a copy of the message to S3 using an object key derived from properties of the message. S3ObjectKeyGenerator

@stevejgordon www.stevejgordon.co.uk | Method | Mean |Ratio | Gen 0

| Gen 1 | Gen 2 | Allocated | |------------- |-----------:|-----:|----------:|----------:|----------:|----------:| | Original | 1,088.0 ns | 1.00 | 0.1812 | - | - | 1144 B | | SpanBased | 449.0 ns | 0.41 | 0.0305 | - | - | 192 B | | StringCreate | 442.9 ns | 0.41 | 0.0305 | - | - | 192 B | ~2.5x Faster ~6x Less Allocations 18 million messages: Reduction of 17GB of allocations daily Removes approx. 2711 Gen 0 collections (562 vs. 3273)

@stevejgordon www.stevejgordon.co.uk • Pool of arrays for re-use • Found

in System.Buffers • ArrayPool<T>.Shared.Rent(int length) • You are likely to get an array larger than your minimum size • ArrayPool<T>.Shared.Return(T[] array, bool clearArray = false) • Warning: By default returned arrays are not cleared! • https://adamsitnik.com/Array-Pool/

@stevejgordon www.stevejgordon.co.uk public class Processor { public void DoSomeWorkVeryOften() {

var buffer = new byte[1000]; // allocates DoSomethingWithBuffer(buffer); } private void DoSomethingWithBuffer(byte[] buffer) { // use the array } }

@stevejgordon www.stevejgordon.co.uk public class Processor { public void DoSomeWorkVeryOften() {

var buffer = new byte[1000]; // allocates DoSomethingWithBuffer(buffer); } private void DoSomethingWithBuffer(byte[] buffer) { // use the array } }

@stevejgordon www.stevejgordon.co.uk public class Processor { public void DoSomeWorkVeryOften() {

var arrayPool = ArrayPool<byte>.Shared; var buffer = arrayPool.Rent(1000); DoSomethingWithBuffer(buffer); } private void DoSomethingWithBuffer(byte[] buffer) { // use the array } }

@stevejgordon www.stevejgordon.co.uk public class Processor { public void DoSomeWorkVeryOften() {

@stevejgordon www.stevejgordon.co.uk | Method | SizeInBytes | Mean | Gen

0 | Gen 1 | Gen 2 | Allocated | |-------------- |------------ |--------------:|--------:|--------:|--------:|----------:| | RentAndReturn | 20 | 29.397 ns | - | - | - | - | | Allocate | 20 | 6.563 ns | 0.0115 | - | - | 48 B | | RentAndReturn | 100 | 28.797 ns | - | - | - | - | | Allocate | 100 | 13.349 ns | 0.0306 | - | - | 128 B | | RentAndReturn | 1000 | 33.807 ns | - | - | - | - | | Allocate | 1000 | 84.908 ns | 0.2447 | - | - | 1024 B | | RentAndReturn | 10000 | 35.387 ns | - | - | - | - | | Allocate | 10000 | 978.090 ns | 2.3918 | - | - | 10024 B | | RentAndReturn | 100000 | 31.615 ns | - | - | - | - | | Allocate | 100000 | 12,875.858 ns | 31.2347 | 31.2347 | 31.2347 | 100024 B |

@stevejgordon www.stevejgordon.co.uk • Created by ASP.NET team to improve Kestrel

requests per second. • Improves I/O performance scenarios (~2x vs. streams) • Removes common hard to write, boilerplate code • Unlike streams, pipelines manages buffers for you from ArrayPool • Two sides to a pipe, PipeWriter and PipeReader • Can be awaited multiple times without multiple Task allocations in .NET Core 2.1 - IValueTaskSource

@stevejgordon www.stevejgordon.co.uk PipeWriter : IBufferWriter<byte> Pipe PipeReader Memory<byte> m =

pw.GetMemory(); … pw.Advance(1000) await pw.FlushAsync() ReadResult r = await reader.ReadAsync(); ReadOnlySequence<byte> b = r.Buffer;

@stevejgordon www.stevejgordon.co.uk Memory<T> Memory<T> Memory<T> ReadOnlySequence<T>

@stevejgordon www.stevejgordon.co.uk private static async Task ProcessLinesAsync(Socket socket) { var

pipe = new Pipe(); Task writing = FillPipeAsync(socket, pipe.Writer); Task reading = ReadPipeAsync(pipe.Reader); await Task.WhenAll(reading, writing); } https://channel9.msdn.com/Shows/On-NET/High-performance-IO-with-SystemIOPipelines

@stevejgordon www.stevejgordon.co.uk private static async Task ProcessLinesAsync(Socket socket) { var

pipe = new Pipe(); Task writing = FillPipeAsync(socket, pipe.Writer); Task reading = ReadPipeAsync(pipe.Reader); await Task.WhenAll(reading, writing); }

@stevejgordon www.stevejgordon.co.uk private static async Task ProcessLinesAsync(Socket socket) { var

@stevejgordon www.stevejgordon.co.uk private static async Task FillPipeAsync(Socket socket, PipeWriter writer)

{ while (true) { try { Memory<byte> memory = writer.GetMemory(); // Request memory from the pipe int bytesRead = await socket.ReceiveAsync(memory, SocketFlags.None); if (bytesRead == 0) break; writer.Advance(bytesRead); // Tell the PipeWriter how much was read from the Socket } catch { break; } FlushResult result = await writer.FlushAsync(); // Make the data available to the PipeReader if (result.IsCompleted) break; } writer.Complete(); // Signal to the reader that we're done writing }

@stevejgordon www.stevejgordon.co.uk private static async Task FillPipeAsync(Socket socket, PipeWriter writer)

@stevejgordon www.stevejgordon.co.uk private static async Task ReadPipeAsync(PipeReader reader) { while

(true) { ReadResult result = await reader.ReadAsync(); // will await until the writer flushes ReadOnlySequence<byte> buffer = result.Buffer; SequencePosition? position = null; do { position = buffer.PositionOf((byte)'\n’); // Find the EOL if (position != null) { ProcessLine(buffer.Slice(0, position.Value)); // Skip what we've already processed including \n buffer = buffer.Slice(buffer.GetPosition(1, position.Value)); } } while (position != null); reader.AdvanceTo(buffer.Start, buffer.End); // Tell PipeReader how much we consumed if (result.IsCompleted) // Stop reading if there’s no more data coming break; } reader.Complete(); // Mark the PipeReader as complete }

@stevejgordon www.stevejgordon.co.uk private static async Task ReadPipeAsync(PipeReader reader) { while

@stevejgordon www.stevejgordon.co.uk Microservice which: 1. Retrieves S3 object (TSV file)

from AWS 2. Decompresses file 3. Parses TSV to get 3 of 25 columns for each row 4. Indexes data to ElasticSearch CloudFrontParser

@stevejgordon www.stevejgordon.co.uk | Method | Mean |Ratio | Gen 0

| Gen 1 | Gen 2 | Allocated | |---------- |-----------:|-----:|----------:|----------:|----------:|-----------:| | Original | 8,500.9 ms | 1.00 | 1548000.0 | 267000.0 | 109000.0 | 7205.44 MB | | Optimised | 957.5 ms | 0.11 | 43000.0 | 20000.0 | 2000.0 | 242.41 MB | ~30x Less Heap Memory Allocations NOTE: ~203.5Mb are the string allocations for the parsed data

@stevejgordon www.stevejgordon.co.uk • In the box JSON APIs - System.Text.Json

• Low-Level – Utf8JsonReader and Utf8JsonWriter • Mid-Level – JsonDocument • High-Level – JsonSerializer and JsonDeserializer

@stevejgordon www.stevejgordon.co.uk Microservice which: 1. Perform ElasticSearch Bulk Index 2.

Deserialise JSON response to check for errors 3. Return a list of the IDs which errored WARNING: APIs have changed a little since this sample was written! BulkResponseParser

@stevejgordon www.stevejgordon.co.uk | Method | Mean | Ratio | Gen

0 | Gen 1 | Gen 2 | Allocated | |---------- |-------------:|------:|---------:|-------:|-------:|-----------:| | Original | 386,514.8 ns | 1.000 | 26.3672 | 0.4883 | - | 111408 B | | Optimised | 485.3 ns | 0.001 | 0.0181 | 0.0010 | - | 80 B | | Method | Mean | Ratio | Gen 0 | Gen 1 | Gen 2 | Allocated | |---------- |-------------:|------:|---------:|-------:|-------:|-----------:| | Original | 428,500 ns | 1.00 | 27.3428 | 0.4883 | - | 114.30 KB | | Optimised | 141,900 ns | 0.33 | 3.6621 | 0.2441 | - | 15.77 KB |

@stevejgordon www.stevejgordon.co.uk • Identify a quick win • Use a

scientific approach to demonstrate gains • Put gains into a monetary value • Cost to benefit ratio

@stevejgordon www.stevejgordon.co.uk This work is a small part of a

much bigger potential gain For a single microservice handling 18 million messages per day Reduction of at least 50% of allocations. Potential to at least double per instance throughput At least 1 less VM needed per year saving $1,700

@stevejgordon www.stevejgordon.co.uk • Measure, don't assume! • Be scientific; make

small changes each time and measure again • Focus on hot paths • Don't copy memory, slice it! Span<T> is less complex than it may first seem. • Use ArrayPools where appropriate to reduce array allocations • Consider Pipelines for I/O scenarios • Consider System.Text.Json APIs for high-performance JSON parsing

@stevejgordon www.stevejgordon.co.uk Pro .NET Memory Management By Konrad Kokosa

@stevejgordon www.stevejgordon.co.uk Pro .NET Benchmarking By Andrey Akinshin

www.stevejgordon.co.uk @stevejgordon Thanks for listening! @stevejgordon www.stevejgordon.co.uk youtube.stevejgordon.co.uk

@stevejgordon www.stevejgordon.co.uk 1. http://bit.ly/highperfcode90 2. https://www.youtube.com/watch?v=CwISe8blq38 3. https://github.com/stevejgordon/TurbochargedDemos 4. https://benchmarkdotnet.org

5. https://msdn.microsoft.com/en-us/magazine/mt814808.aspx 6. https://adamsitnik.com/Span/ 7. https://adamsitnik.com/Array-Pool/ 8. https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/ 9. https://blog.marcgravell.com/2018/07/pipe-dreams-part-1.html

Turbocharged: Writing High Performance C# and ....

Turbocharged: Writing High Performance C# and .NET Code (90 mins)

More Decks by Steve Gordon

Other Decks in Technology

Featured

Transcript