JumpStart to Build Generative AI with Amazon SageMaker

YouTube 영상: https://www.youtube.com/watch?v=-vMD3jGFnRE

Agenda
- Build Generative AI with SageMaker JumpStart
- How To Fine-tune and Train Pre-trained ML Models
- Optimize ML Inference with SageMaker Inference Recommender
- Benefits of ML in Cloud

Transcript

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 클릭 몇 번만으로 Generative AI 모델 만들기 김성민 AI/ML 스페셜리스트 솔루션즈 아키텍트, AWS
2. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. JumpStart to Build Generative AI with Amazon SageMaker
3. © 2023, Amazon Web Services, Inc. or its affiliates. Agenda Build Generative AI with SageMaker JumpStart How To Customize Pre-trained ML Models Optimize ML Inference with SageMaker Inference Recommender Benefits of ML in Cloud
4. © 2023, Amazon Web Services, Inc. or its affiliates. Generative AI – Text Generation Jack went to the university in the early 1970s as a graduate student and started the company that would become McDonald’s. Though he was a long time away from his family, he was an avid cook. “My mom is a huge foodie,” he says. “She likes to know things about what I’m eating.” Her recipes helped him develop an idea of how to go about the food business and eventually made him a fast-food millionaire in 1993. “I mad sure that my family was always watching,” he says. “And I always liked to tell them about everything I was doing on a regular basis.” That early obsession with recipes led to his own idea of what a McDonald’s menu would look like: a menu that would make people say hello to their food, even though they’ve been away.
5. © 2023, Amazon Web Services, Inc. or its affiliates. Generative AI – Text Generation DistilGPT2 GPT-3 Bloom AlexaTM …
6. © 2023, Amazon Web Services, Inc. or its affiliates. Key trends in AI/ML • Models are becoming more complex, with end users moving from classical ML to deep learning • State-of-the-art deep learning models are getting larger and larger as we find that larger models generalize better Source: NVIDIA Transformers 65M BERT 340M GPT-2 1.5B GPT-2 8B 8.3B T5 11B Turing-NLG 17B GPT-3 175B Megatron-Turing 530B GPT-3 1T 1 trillion MID 2017 2018 2019 MID 2019 LATE 2019 2020 MID 2020 LATE 2021 2022 Model size Time 15,000x increase in 5 years
7. © 2023, Amazon Web Services, Inc. or its affiliates. Challenges with building your own ML model Long time-to-train Large-scale Human-in-loop High cost-to-train Deep Expertise Required
8. © 2023, Amazon Web Services, Inc. or its affiliates. A fully managed service that enables data scientists and developers to quickly and easily build machine-learning based models into production smart applications. Amazon SageMaker
9. © 2023, Amazon Web Services, Inc. or its affiliates. ML Workflow ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy Amazon SageMaker Raw Data
10. © 2023, Amazon Web Services, Inc. or its affiliates. End-to-End Machine Learning Platform Zero setup Flexible Model Training Pay by the second $ Amazon SageMaker 손 쉬 운 기 계 학 습 모 델 생 성 , 훈 련 및 서 비 스 배포 완전 관리 서비 스
11. © 2023, Amazon Web Services, Inc. or its affiliates. Data Labeling Build Training & Tuning Deploy
12. © 2023, Amazon Web Services, Inc. or its affiliates. How to Train and Deploy Generative AI Models?
13. © 2023, Amazon Web Services, Inc. or its affiliates. “내가 더 멀리 보았다면 이는 거인들의 어깨 위에 서 있었기 때문이다.” - 아이작 뉴턴 DON’T necessarily have to start from scratch © “거인의 어깨위에 서서” @굿뉴스데일리
14. © 2023, Amazon Web Services, Inc. or its affiliates.
15. © 2023, Amazon Web Services, Inc. or its affiliates. Tasks Algori thms/models Vision Text Tabular Audio SageMaker JumpStart: ML hub for SageMaker Customers 400+ algorithms and pre-trained, state-of-the-art, open-source models from PyTorch Hub, TensorFlow Hub, and Hugging Face, etc.
16. © 2023, Amazon Web Services, Inc. or its affiliates. Browse and search SageMaker JumpStart content Search for topics or problem types, and get relevant results across all content Browse by content type to explore solutions, models, example notebooks, blogs, and video tutorials
17. © 2023, Amazon Web Services, Inc. or its affiliates. Easily launch pre-built solutions Launch solutions through cloud formation with a single click Easily manage assets from Amazon SageMaker JumpStart Open pre-populated notebooks for solutions to solve the business problems end to end
18. © 2023, Amazon Web Services, Inc. or its affiliates. Easily deploy or fine-tune models Deploy or fine-tune pre- trained models with a single click Open pre-populated notebooks to perform inference on deployed models 400+ pre-trained open source models from Hugging Face, PyTorch, TensorFlow Hubs
19. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Demo: Text Generation
20. © 2023, Amazon Web Services, Inc. or its affiliates.
21. © 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained ML Models? ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy Amazon SageMaker 한국어 Fine-tuning 가나다라 마바사아 … 1 2
22. © 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained Models • Train Fine-tunable pre-trained models • Fine-tune and train from scratch with Hugging Face, or other open source models.
23. © 2023, Amazon Web Services, Inc. or its affiliates. Train and Deploy Fine-tunable Models on SageMaker
24. © 2023, Amazon Web Services, Inc. or its affiliates. Train and Deploy Fine-tunable Models on SageMaker
25. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Hugging Face on Amazon SageMaker Scale and accelerate your NLP projects with Hugging Face Integrations
26. © 2023, Amazon Web Services, Inc. or its affiliates. What are the Hugging Face libraries? Open-source Datasets, Tokenizers and Transformers Popular 52k+ GitHub stars (March 2021), 1M+ downloads per month Intuitive NLP-specific Python frontends based on PyTorch or TensorFlow State of the art Transformer-based models are state-of-the-art, enable transfer-learning and scale Comprehensive Model zoo with 7000+ model architectures, 160+ languages
27. © 2023, Amazon Web Services, Inc. or its affiliates. A strong partnership to make NLP easy & accessible for all Hugging Face is the most popular Open Source company providing state of the art NLP technology Hugging Face SageMaker offers high performance resources to train and use NLP Models AWS https://huggingface.co/ https://aws.amazon.com/sagemaker/
28. © 2023, Amazon Web Services, Inc. or its affiliates. Integrated workflow with Amazon SageMaker
29. © 2023, Amazon Web Services, Inc. or its affiliates. How to Fine-tune and Train pre-trained ML Models? ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy 한국어 Fine-tuning 가나다라 마바사아 … 1 2 + Amazon SageMaker Hugging Face
30. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Demo: Hugging Face
31. © 2023, Amazon Web Services, Inc. or its affiliates.
32. © 2023, Amazon Web Services, Inc. or its affiliates. Deploy Hugging Face Model on SageMaker Define HF Task Deploy on SageMaker Build HF Model with SageMaker SDK 1 2 3
33. © 2023, Amazon Web Services, Inc. or its affiliates. Train Hugging Face Model on SageMaker 1 2 3 4
34. © 2023, Amazon Web Services, Inc. or its affiliates. Train Deploy
35. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Training How does training happen HuggingFace validation(optional) test(optional) ECR(1) S3 ML Instance ml.p3.2xlarge HuggingFace linear-learner PCA DeepAR BlazingText Image classification … Object Detection DLC Images(2) S3 SageMaker Notebook SageMaker Training Job train Model (1) ECR: Elastic Container Registry (2) DLC: Deep Learning Container
36. © 2023, Amazon Web Services, Inc. or its affiliates. Train PyTorch or TensorFlow models with SageMaker SDK 1 2 3
37. © 2023, Amazon Web Services, Inc. or its affiliates. Deploy PyTorch or TensorFlow models with SageMaker SDK 1 2 3
38. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. SageMaker Model Hosting
39. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Amazon SageMaker Amazon S3 Amazon ECR
40. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Amazon SageMaker Amazon S3 Amazon ECR Model artifacts
41. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Amazon SageMaker Amazon S3 Amazon ECR Model artifacts Inference Image
42. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment Hosting Services Inference Image Training Image Training Data Model artifacts Endpoint Amazon SageMaker Amazon S3 Amazon ECR Model artifacts Inference Image
43. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment SageMaker Endpoints (Private API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)
44. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon SageMaker Deployment SageMaker Endpoints (Public API) Auto Scaling group Availability Zone 1 Availability Zone 2 Availability Zone 3 Elastic Load Balancing Model Endpoint Amazon API Gateway Client Deployment / Hosting Amazon SageMaker ML Compute Instances Input Data (Request) Prediction (Response)
45. © 2023, Amazon Web Services, Inc. or its affiliates. SageMaker ML instance options B A L A N C I N G B E T W E E N C O S T A N D P E R F O R M A N C E High throughput, and low-latency access to CUDA GPU INSTANCES P3 G4 Low throughput, low cost, most flexible CPU INSTANCES C5 Inf1: High throughput, high performance, and lowest cost in the cloud CUSTOM CHIP Inf1
46. © 2023, Amazon Web Services, Inc. or its affiliates. ML instance ML instance Endpoint Load testing K N O W Y O U R E N D P O I N T S Artificial requests Amazon SageMaker endpoint Endpoint Auto-scaling group Availability Zone 1 Availability Zone 2 ML instance ML instance ML instance ML instance Amazon CloudWatch Elastic Load Balancing
47. © 2023, Amazon Web Services, Inc. or its affiliates. SageMaker Inference Recommender F E A T U R E S Designed for MLOps engineers and data scientists to reduce time to get models into production Run extensive load tests that include production requirements – throughput, latency Load tests Get endpoint configuration settings that meet your production requirements Endpoint recommendations Instance recommendations Instance type recommendation for initial deployments
49. © 2023, Amazon Web Services, Inc. or its affiliates. Get an instance recommendation in minutes
50. © 2023, Amazon Web Services, Inc. or its affiliates. Run custom load tests across instance types
51. © 2023, Amazon Web Services, Inc. or its affiliates. Review endpoint recommendations
52. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Benefits of ML in Amazon SageMaker
53. © 2023, Amazon Web Services, Inc. or its affiliates. Challenges with building your own ML model Long time-to-train Large-scale Human-in-loop High cost-to-train Deep Expertise Required
54. © 2023, Amazon Web Services, Inc. or its affiliates. Fully integrated with Amazon SageMaker Monitor Monitor for bias and feature attribution drifts Automate pipeline Deploy Flexible deployment with best-price performance Fine-tune Customization for specific domains Human-in- the-Loop Provide feedback, label data, active learning through human-in-loop
55. © 2023, Amazon Web Services, Inc. or its affiliates. End-to-End Machine Learning with Amazon SageMaker ML Problem Framing Real-World Problem Define ML Problem Data Preparation Build Training Deploy SageMaker JumpStart + Hugging Face • SageMaker Endpoint • SageMaker Inference Recommender SageMaker Training Job SageMaker Studio Raw Data
56. © 2023, Amazon Web Services, Inc. or its affiliates. Put machine learning in the hands of every developer Our mission at
57. © 2023, Amazon Web Services, Inc. or its affiliates. Call-to-Action • AWS Machine Learning Low-Code Immersion Day • https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64- 4e5a-a02c-a6c88516ab02/en-US • SageMaker Immersion Day • https://sagemaker-immersionday.workshop.aws/ • Using HuggingFace Models on Amazon SageMaker • https://catalog.us-east-1.prod.workshops.aws/workshops/74da6491-2f74- 466a-86a5-3bcce6541904/en-US • Amazon SageMaker Canvas Immersion Day • https://catalog.workshops.aws/canvas-immersion-day/en-US
58. Thank you © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.