.NET Day 2023: Bringing the Power of Deep Learning to .NET with ONNX and ML.NET

With the advent of ChatGPT, everyone is talking about AI. And it is destined to become even more embedded into our daily lives. However, while .NET is usually not associated with machine learning, ML.NET is a powerful tool within our ecosystem and allows us to use our existing C#-skills to train and use machine learning models. The talk will start with an introductory look at ML.NET. Getting started with AutoML and Model Builder is easy and training such small models is feasible for everyone. However, training state-of-the-art large models is near impossible and thus, we can integrate existing, pretrained deep learning models. In the talk, we will use ONNX, an open format to exchange deep neural networks between tools and frameworks, to integrate a pretrained BERT model into a .NET application. BERT is a transformer-based natural language model, similar to the famous GPT-models, and will answer questions in natural language for us.

Transcript

1. Bringing the Power of Deep Learning to .NET with ONNX and ML.NET 29.08.2023 .NET Day Switzerland 2023 Alexander Kayed Image generated with clipdrop by stability.ai
2. 01.09.2023 WE KNOW HOW. 2
3. ENGINEER / ARCHITECT / SPEAKER / TRAINER @ NOSER ENGINEERING AG • Microservice, CQRS, event-sourced cloud app (.NET+ Angular) • GIS-Project: Topology Data, LoS, Lidar (WPF) • ChatGPT-based “Knowledge-Bot” (.NET, Semantic Kernel) • OSS: Akade.IndexedSet (Performance oriented multi-index data structure) • Focus: C# / .NET, ML / AI Alexander Kayed 01.09.2023 WE KNOW HOW. 3 Alex MSc ETH in Computer Science github.com/akade
4. 01.09.2023 WE KNOW HOW. 4 ML and .NET?
5. 01.09.2023 WE KNOW HOW. 5 ML.NET
6. 01.09.2023 WE KNOW HOW. 6 ML.NET & ONNX
7. 01.09.2023 WE KNOW HOW. 7 ML.NET & ONNX
8. Why not just call ChatGPT? 01.09.2023 WE KNOW HOW. 8 Image: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fdiscovered-this-v0-18nnu31v5tea1.jpg%3Fs%3D23afe7ca4c811106cec84501656b38e8afc06415
9. Why not just call ChatGPT? 01.09.2023 WE KNOW HOW. 9 Local training or local inference Approachable & limited time / money Privacy
10. Llama 2 01.09.2023 WE KNOW HOW. 10 • Most recent, fully open-sourced LLM by Meta • OpenAI models are not open-source https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md ~ 4537x A100 for a month ~ 539 tCO2 eq (5t/year/person CH) ~ 13.6M $ on Azure https://www.bafu.admin.ch/bafu/de/home/themen/klima/inkuerze.html https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/ 70B (109) Parameter ~260 GB (FP32) 3.3M h GPU Time 27.5K “Chats” (SFT) 1.4M “Ratings” (RLHF) 2 trillion (1012) tokens
11. ML.NET 01.09.2023 WE KNOW HOW. 11
12. ML.NET 01.09.2023 WE KNOW HOW. 12 Demo Time! Pokemon Image Set from https://www.kaggle.com/datasets/kvpratama/pokemon-images-dataset
13. Transfer Learning 01.09.2023 WE KNOW HOW. 13 • Reusing an existing convolutional neural network − Plane: 89% − Boat: 9% − Car: 1% − Apple: 0.5% − … Weights from a pretrained net
14. Transfer Learning 01.09.2023 WE KNOW HOW. 14 Weights from a pretrained net − Plane: 89% − Plant: 9% − Plant: 1% − Apple: 0.5% − …
15. Transfer Learning 01.09.2023 WE KNOW HOW. 15 − Car: 0.5% − House: 1% − Plant: 20% − Apple: 0.5% − … Freeze some weights Weights from a pretrained net
16. Transfer Learning 01.09.2023 WE KNOW HOW. 16 − Plant: 88% − Insects: 5% − Water: 4% − Fire: 2% − … Freeze some weights New output layer Weights from a pretrained net
17. ML.NET 01.09.2023 WE KNOW HOW. 17 Demo Time!
18. 01.09.2023 WE KNOW HOW. 23 ONNX
19. 01.09.2023 WE KNOW HOW. 24 Open Neural Network EXchange
20. 01.09.2023 WE KNOW HOW. 25 https://onnx.ai/supported-tools.html
21. 01.09.2023 WE KNOW HOW. 26 https://onnx.ai/supported-tools.html
22. 01.09.2023 WE KNOW HOW. 27 https://onnx.ai/supported-tools.html
23. 01.09.2023 WE KNOW HOW. 28 https://onnx.ai/supported-tools.html
24. 01.09.2023 WE KNOW HOW. 29 https://onnx.ai/supported-tools.html Demo Time!
25. 01.09.2023 WE KNOW HOW. 30 But, I want a “real LLM”?!
26. 31 But, I want a “real LLM”?!
28. 33 •Llama 2 7B •Fully local on this laptop •No dedicated GPU Demo Time!
29. 01.09.2023 WE KNOW HOW. 34 ML.NET & ONNX 1. Intro - Motivation - Llama 2 2. ML.NET - AutoML / Model Builder - Consumption - Training / Pipeline 3. ONNX / BERT - Idea - Model Zoo - BERT - Netron - ONNX Runtime 4. LlamaSharp - Llama2 7B, running locally - Llama.cpp / LlamaSharp Questions github.com/akade Visit us - at our booth - or at www.noser.com Quelle: https://github.com/dotnet/brand codingakade@mastadon.social
30. 01.09.2023 WE KNOW HOW. 35 ML.NET & ONNX 1. Intro - Motivation - Llama 2 2. ML.NET - AutoML / Model Builder - Consumption - Training / Pipeline 3. ONNX / BERT - Idea - Model Zoo - BERT - Netron - ONNX Runtime 4. LlamaSharp - Llama2 7B, running locally - Llama.cpp / LlamaSharp Thank you! github.com/akade Visit us - at our booth - or at www.noser.com Quelle: https://github.com/dotnet/brand codingakade@mastadon.social