Generative AI For Beginners: Fine Tuning

Lesson 18 of the Generative AI For Beginners series.
Learn more at https://aka.ms/genai-beginners

This lesson introduces the concept of fine-tuning for pre-trained language models, explores the benefits and challenges of this approach, and provides guidance on when and how to use fine tuning to improve the performance of your generative AI models.

By the end of this lesson, you should be able to answer the following questions:
1. What is fine tuning for language models?
2. When, and why, is fine tuning useful?
3. How can I fine-tune a pre-trained model?
4. What are the limitations of fine-tuning?

Transcript

1. GENERATIVE AI FOR BEGINNERS aka.ms/genai-beginners F I N E T U N I N G Y O U R L L M Nitya Narasimhan, PhD
2. Welcome What is Fine Tuning? Why is Fine Tuning Useful? When should we use Fine Tuning? How do we Fine Tune an LLM? Best Practices & Limitations Summary
3. I N T R O D U C T I O N : F I N E T U N I N G A N L L M What does Fine-Tuning mean? Why is it useful and when should we adopt this approach given options like prompt engineering or retrieval augmented generation?
4. W H AT I S F I N E T U N I N G ? A common practice in machine learning where we retrain an existing model with new data to improve performance on task. This is an advanced technique that requires some expertise to get desired results. Incorrect usage may degrade performance.
5. W H Y S H O U L D I F I N E - T U N E ? Fine-Tuning may be appropriate if your desired response quality is not achievable with prompt engineering or RAG approaches. Another reason may be the cost efficiency achieved by reduced token usage or ability to upskill a cheaper model (within reason).
6. W H E N S H O U L D I F I N E - T U N E ? But the approach is valid only if the benefits outweigh costs. • Do you have a good use case? (format, edge cases, new skills) • Have you tried other options? • Did you factor in other costs? (compute, data, maintenance) • Did you confirm the benefits? (evaluation, region availability)
7. P R O C E S S : H OW D O W E F I N E T U N E ? Fine Tuning a model is not a trivial process. Is the foundation model fine-tunable and available? Do you have the right data? Do you have a compute environment to run the job? Do you have a hosting environment to deploy the fine-tuned model for use?
8. P R E P – T R A I N – E VA L U AT E - D E P LOY https://platform.openai.com/docs/guides/fine-tuning
9. T U T O R I A L U S E C A S E A factual chatbot that answers questions about periodic table elements using limericks
10. Prepare & Upload Dataset Tutorial Mode – We have a sample set with 10 records for tutorial walkthrough only. In real world usage, you will need 100+ samples for good results (with cost tradeoffs) Step 1
11. Create Fine-Tuning Job & Track Status to Completion Step 2
12. Get Deployed Model ID and Test it using normal Chat Completion API Step 3
13. Track job status, View evaluated metrics, Compare outcomes No-Code Approach via UI
14. S U M M A R Y R E C A P & N E X T S T E P S Fine Tuning is …
15. Welcome What is Fine Tuning? Why is Fine Tuning Useful? When should we use Fine Tuning? How do we Fine Tune an LLM? Best Practices & Limitations Summary