NLP/LLM Fundamentals & AI Career Guide (By: Kamran Amjad) - AI Career Guide: Mapping Your Path

Talk by Kamran Amjad (https://www.linkedin.com/in/kamran-amjad-bb891aa3/) at AI Career Guide: Mapping Your Path by GDG Lahore

Transcript

1. AI Career Guide Mapping Your Path Speaker: Kamran Amjad Principal Machine Learning Engineer at Arbisoft
2. pollev.com/kamranamjad601
3. First !! Let’s delve into the recent advancements and hype about AI Most of the hype you see now a days is based on text generation tools e.g. bard, chatgpt etc.
4. Unleashing the Power of Text Navigating the world of Natural Language Processing
5. 01 Introduction to NLP Table of contents 02 NLP Fundamentals 03 Relevance in Modern Tech 04 Core Task in NLP 05 Large Language Models (LLMs) 06 Applications of LLMs 07 Challenges 08 Career Choices 09 Q&A | Conclusion
6. What is Natural Language Processing (NLP) 01 Kamran Amjad - (Principal ML Engineer - Arbisoft)
7. What is NLP • branch of AI focused on interactions between computers and human language • enables machines to understand, interpret, and generate human language in a meaningful way • NLP tasks usually include ◦ text/speech processing ◦ sentiment analysis ◦ translation ◦ named entity recognition (NER) ◦ part-of-speech identification ◦ summarization ◦ machine conversation (chatbot) ◦ text generation Bard, ChatGPT
8. Fundamentals of Natural Language Processing (NLP) 02 Kamran Amjad - (Principal ML Engineer - Arbisoft)
9. Techniques in NLP Tokenization • Breaking down text into smaller units (tokens) • Enables processing of individual elements Stopword Removal • Removing common, less informative words (e.g., "the," "is") from text • reduces noise in the data Lemmatization/Stemming • Reducing words to their base/root • aiding in analysis and understanding Part-of-Speech (POS) Tagging • Label words with their grammatical roles (e.g., noun, verb, adjective) • Provides insights into the syntactic structure of a sentence Dependency Parsing • relationships between words in a sentence to determine syntactic structure • Identifies which words depend on others for meaning Word Embeddings • Representing words as vectors in a continuous vector space • Captures semantic relationships between words, allowing for numerical processing Semantic Similarity • Measuring the similarity in meaning between words or sentences • Used in tasks like information retrieval, question answering, and recommendation systems
10. Some Advanced Techniques in NLP Named Entity Recognition (NER) • Identifying and classifying entities in text into predefined categories (e.g., names of people, locations, organizations). • Extracts key information, facilitates information retrieval Coreference Resolution • Identifying and connecting pronouns or noun phrases in a text to their referents • Resolves ambiguity and establishes connections between different mentions of the same entity, improving overall understanding Sentiment Analysis • Determining the sentiment or emotion expressed in a piece of text • Valuable for understanding public opinion, customer feedback analysis Text Summarization • Generating concise and coherent summaries of longer texts while retaining the core information • Facilitates quick understanding of large volumes of text, aids in information retrieval
11. Relevance in Modern Technology 03 (AI)
12. Enhanced Human-Computer Interaction Voice assistants like Siri, Alexa, and Google Assistant utilize NLP to understand and respond to user queries Facilitating Multilingual Communication Applications include real-time translation in video conferencing tools, making global collaboration seamless Automating Customer Service This is particularly valuable in industries with high customer interaction volumes, such as e-commerce and banking (chatbot etc.) Insight Extraction from Big Data This is invaluable for market research, sentiment analysis, and understanding customer feedback.
13. Personalized Content Recommendations in content recommendation engines to understand user preferences and serve tailored content (e.g. YouTube,Netflix,Spotify etc.) Transforming Healthcare NLP is used to analyze clinical notes, extracting valuable information for decision support, research, and patient care Legal and Compliance It helps identify risks, extract key terms, and ensure regulatory compliance Social informatics and Sentiment Analysis NLP is used to analyze social media conversations, understanding public sentiment towards products, brands, or events Enabling Accessibility NLP-driven technologies like speech recognition and text-to-speech tools empower individuals with disabilities
14. But.. what is CORE task in NLP? 04 (AI)
15. Modelling the Natural Language but now you ask: “what is language modelling and how do we do it?”
16. What is Language Modelling? “Language modeling is the task of predicting the probability of a sequence of words occurring in a given context” Probabilistic way of generating language On to the important question!! Basically determining the rules that make the language
17. Neural Language Models Early Approaches to Language Modeling - traditional statistical models like N-grams and HMMs - had limitations in capturing long-range dependencies and context Introduction of Recurrent Neural Networks (RNNs) - significant advancement in language modeling - could capture sequential information: better context understanding in text Challenges with Long Sequences: - RNNs: difficulties in handling long sequences * - Transformer Architecture and Attention Mechanism:
18. Auto Regressive Model
19. Neural Language Models The seminal paper by Google that revolutionized NLP “Attention is all you need” Transformer Architecture - Transformer model was introduced in 2017 - Transformers use attention mechanisms to process input data in parallel - making them highly efficient - effective for long-range dependencies
20. Large Language Models LLMs 05 (AI)
21. Large Language Model Language Model with Large Number of Parameters For scale estimation: 2018 - Bert -> 340 million - Roberta -> 354 million 2023 (at-least 1000x larger) - PaLM (Google) → 540 billion - GPT-4 (Open-AI) -> 1.76 trillion To understand what parameter is in a NN and how number of parameters work: Understanding and Calculating the number of Parameters in CNNs
22. Some Notable Language Models - BERT (bidirectional encoded representation from transformers) - T5 (Text to Text Transfer Transformer) - Transformer-XL - PaLM (Pathways Language Model) - LaMDA (Language Model for Dialogue Applications) -> BARD - GPT (Generative Pretrained Transformer) GPT-3, 3.5, 4
23. Usual hype about some LLMs Now a days, we hear lots and lots of hype about: - BARD (by Google) - uses PaLM - ChatGPT (by OpenAI) - uses GPT-4 And then “instruction tuned” - In simple terms, language models are further trained on instruction - response data - This gives rise to responses that are more tuned towards human interpretation
24. WORKSHOP Text Generation with Miniature GPT
25. WORKSHOP
26. Applications of LLMs 06 (AI)
27. Use-cases of LLMs Zero shot capabilities make them like a swiss knife
28. Language Generation • Writing articles, stories, and other text content • Chatbots and assistants • Writing Code • Conversational Agents and Chatbots • Summarization Classification Other • Text Classification • Sentiment Analysis • Scoring content • Accessibility Tools • Translation • Question Answering • Preparing Datasets for other problems • Question Answer • Recommendation systems Zero shot abilities of these LLMs are making it possible for these tasks being handled by them without extra training. NLP is going through the most rapid transition any field is facing. These are just some examples, there are multiple others
29. Google search “Supercharging Search with generative AI”
30. Challenges in the field 07 (AI)
31. NLP Challenges - Ambiguity in Language - Context Dependency - Sarcasm, Irony, and Figurative Language - Handling Multilingual Text
32. LLM Challenges • are generic • one size fit all is not always required • comes at a higher cost • GPT-4, GPT-3.5, Llama-V2, etc. have a high number of parameters • higher parameters higher processing costs and times
33. Career Choices 08 (AI)
34. Career Pathways in ML / AI Some of the possible roles in this domain: - NLP Engineer - CV Engineer - AI Engineer - Research Scientist - Data Scientist - ML Engineer Skills and Knowledge Required - Machine Learning knowledge - AI aptitude - Software Engineering knowledge
35. Getting Started 09 (AI)
36. • Strong grip on Algorithms, Knowledge of Data Structure • Strong programming skills • Learn Python • Software development skills —-------------------------------------------------------------------------- • Understand and learn Machine Learning fundamentals • Understand Linguistic Concepts • Explore NLP Libraries and Frameworks • Work on Small Projects (self) • Participate in NLP Challenges and Competitions • Read Research Papers and Blogs • Experiment with Pre-trained Models Tips for Getting Started in ML and NLP
37. Coursera - "Natural Language Processing Specialization" Udacity - "Natural Language Processing Nanodegree" fast.ai - "Practical Deep Learning for Coders" Stanford University's CS224N - "Natural Language Processing with Deep Learning" (Free Lectures) Some resources to learn more
38. CREDITS: This presentation template was created by Slidesgo and includes icons by Flaticon, infographics & images by Freepik and content by Eliana Delacour Thanks! Any questions? kamran.amjad@arbisoft.com +92-3206897670