Building Intelligent Applications with ML.NET

I had the pleasure of conducting a hands-on session titled "Building Intelligent Applications with ML.NET", where we dove into the world of machine learning with .NET — no Python required!

📌 Key Highlights:

Demystified core ML concepts and how they translate into real-world .NET applications
Explored Matrix Factorization for product recommendation systems
Ran a live demo using a co-purchase scenario, predicting what products customers might buy next
Discussed how .NET devs can harness ML.NET and AutoML to build smarter apps without switching ecosystems

Transcript

1. Building Intelligent Applications with ML.NET A Hands-on Session with Product Recommendation NISHAN WICKRAMARATHNA Associate Tech Lead Xeynergy
2. 4 Recommendation Systems Overview 1 What is Machine Learning? 5 Matrix Factorization: Core Concept 6 Hands-on: Product Recommendation in ML.NET 2 We will discuss.. ML.NET Overview 3 ML.NET Capabilities & Architecture
3. • A subfield of AI where computers learn patterns from data. • Unlike traditional programming, ML derives rules from data instead of being explicitly programmed. • Categories of ML: • Supervised Learning: Predict outcomes based on labeled data • Unsupervised Learning: Discover hidden patterns in unlabeled data • Reinforcement Learning: Learn via trial and error What is Machine Learning?
4. DEFINE THE PROBLEM DEPLOY AND MONITOR
5. • You don’t need to switch languages or stacks (no need to use Python) • ML.NET integrates directly into your C# applications • Production-ready and enterprise-tested (e.g., used by Bing, Outlook) • Supports scenarios like recommendations, predictions, classification, and forecasting. Why Should .NET Developers Care About ML?
6. • Cross-platform & Open-source • ML.NET runs on Windows, Linux, and macOS — it’s not just for Windows developers. • Interop with Other Languages • Models trained with ML.NET can be exported to ONNX and used in Python, Java, and other environments. Supports importing models trained in TensorFlow, ONNX, and LightGBM — you’re not locked in. • Enterprise-grade ML without Python • For teams that prefer strongly typed languages, ML.NET offers a robust alternative to Python-based stacks. Why Should Non-.NET Developers Care About ML.NET?
7. Data sourced from Machine Learning at Microsoft with ML.NET paper. Results for sentiment analysis, using ~900 MB of an Amazon review dataset. Higher accuracy and lower runtime are better. Why use ML.NET? • Open-source and cross-platform ML framework by Microsoft • Allows you to build custom ML models using C# or F# • Works in console apps, APIs, web apps, desktop apps, and Azure • Built on .NET, no need for separate Python environments • High performance and accuracy.
8. ML.NET Components • Model Builder: GUI tool for creating ML models quickly. • CLI Tools: For developers who prefer scripting • API (mlContext): Programmatic access to full pipeline • AutoML: Let ML.NET choose the best model for your data
9. ML.NET Supported Scenarios ML.NET supports most common business scenarios out-of-the-box.
10. Transformations Normalize, encode, featurize text 02 Choose trainer (e.g., regression, clustering) Data Loading Load from CSV, database, JSON, etc. How ML.NET Works – The Pipeline 03 01 Training
11. Prediction Save/load model and make predictions 05 Deploy as an API Evaluation Accuracy, RMSE, AUC depending on model type How ML.NET Works – The Pipeline 06 04 Deployment
12. This diagram represents the application code structure and the iterative process of model development: • Collect and load training data into an IDataView object • Specify a pipeline of operations to extract features and apply a machine learning algorithm • Train a model by calling Fit(IDataView) on the pipeline • Evaluate the model and iterate to improve • Save the model into binary format, for use in an application • Load the model back into an ITransformer object • Make predictions by calling PredictionEngineBase<TSrc,TDst>.Predict
13. The code in the following snippet demonstrates the simplest ML.NET application. This example constructs a linear regression model to predict house prices using house size and price data.
14. What is a Recommendation System? • Suggests items based on user preferences and behavior • Widely used in e-commerce, streaming, social media • Two common types: • Content-Based Filtering: Recommend based on item features • Collaborative Filtering: Recommend based on similar user behaviors
15. What is Matrix Factorization? Matrix Factorization is a collaborative filtering technique used in recommendation systems to predict missing values (e.g., what a user might like) by identifying latent features. Imagine a matrix with users as rows and products as columns. Each cell contains a user's rating (or purchase interaction) for a product. Most of this matrix is empty because users don’t interact with every item.
16. Movie A Movie B Movie C User 1 5 4 User 2 3 User 3 4 P1: Laptop P2: Mouse P3: Backpack P4: Monitor C1 1 1 0 0 C2 0 1 1 0 C3 1 0 0 1 Most values are missing. We don’t know how User 1 feels about Movie B, for example. Matrix Factorization will decompose this into two matrices: • User matrix (U): captures each user’s preferences across some hidden features (e.g., genre taste, humor preference, etc.) • Item matrix (V): captures how much each item (movie) expresses these hidden features
17. Let’s say we use 2 hidden factors (dimensions): U (Users): Genre Humor User 1 0.9 0.3 User 2 0.2 0.8 User 3 0.7 0.1 V (Movies): Genre Humor Movie A 0.8 0.2 Movie B 0.1 0.9 Movie C 0.7 0.3 To predict how much User 1 would like Movie B: Multiply: (0.9 × 0.1) + (0.3 × 0.9) = 0.09 + 0.27 = 0.36 Scale this appropriately to get predicted rating.
19. To make a single prediction, you have to create a PredictionEngine. PredictionEngine is not thread-safe. Additionally, you have to create an instance of it everywhere it is needed within your application. As your application grows, this process can become unmanageable. For improved performance and thread safety, use a combination of dependency injection and the PredictionEnginePool service, which creates an ObjectPool of PredictionEngine objects for use throughout your application. PredictionEnginePool
20. Set the watchForChanges parameter to true, and the PredictionEnginePool starts a FileSystemWatcher that listens to the file system change notifications and raises events when there is a change to the file. This prompts the PredictionEnginePool to automatically reload the model.
21. AutoML Preprocessing, training, and evaluation are an experimental and iterative process that requires multiple trials until you achieve satisfactory results. Because these tasks tend to be repetitive, AutoML can help automate these steps. In addition to automation, optimization techniques are used during the training and evaluation process to find and select algorithms and hyperparameters.
22. Whether you're just getting started with machine learning or you're an experienced user, AutoML provides solutions for automating the model development process. • Beginners - If you're new to machine learning, AutoML simplifies the model development process by providing a set of defaults that reduces the number of decisions you have to make when training your model. In doing so, you can focus on your data and the problem you're trying to solve and let AutoML do the rest. • Experienced users - If you have some experience with machine learning, you can customize, configure, and extend the defaults provided by AutoML based on your needs while still leveraging its automation capabilities. When should I use AutoML?
23. ML.NET Samples • Product Recommendation - Matrix Factorization Problem Sample • Movie Recommendation - Matrix Factorization Sample 1 (Program.cs) • Movie Recommendation - Matrix Factorization Sample 2 Documentation and Tutorials • What is ML.NET and How Does It Work? • What is Automated Machine Learning (AutoML)? • ML.NET AutoML Model Builder (Step-by-Step Walkthrough) • Deploy a Model in an ASP.NET Core Web API References Data & Research • Amazon Sales Dataset (Kaggle) • Machine Learning at Microsoft with ML.NET (Research Paper) Ecosystem & Showcase • Open Neural Network Exchange (ONNX) • Artificial Intelligence & ML Customer Showcase (Microsoft)
24. THANK YOU! NISHAN WICKRAMARATHNA Associate Tech Lead Xeynergy Do you have any questions? ❑ nishanw@railtrac.net ❑ nishanc.com