GPT → Generative Pretrained Transformer GPT GPT-2 GPT-3 ChatGPT(GPT-3.5) GPT-4 2018 Improving Language Understanding by Generative Pre-Training Language Models are Unsupervised Multi- task Learners Training language models to follow instructions with human feedback * Language Models are Few-Shot Learners 2019 2020 2022 2023 * GPT-3.5 is built on top of InstructGPT with a different data collection setup (technical report) Rapid growth …
• Language Models (like GPT-X), ◦ are chaotic ◦ model a “giant mass of people” ~ Minqi Jiang, MetaAI • For different prompts, you can get wildly different outputs • We must “ “snip out” ” the ugly, less-preferred parts stuff that’s learned stuff we care about
Feedback • Provides a friendlier interface to interact with LMs • Biases the underlying model to generate human-aligned content • Improves reliability, honesty, and safety of LLMs “ “How do we get LLMs to sound more human?” ”
Feedback 1. Pretrain a LLM on a body of text [GPT-X, for instance] 2. Train a Reward Model (RM) → “ “how would a human feel?” ” 3. Finetune using RL [LLM agent predicts words and is scored]
Treat ChatGPT as you would a friend • Want something? Just ask for it! • The art of “ “Prompt Engineering” ” with ChatGPT Use ChatGPT as a personal tutor!
will only get better from here on • Students should can learn how to operate these tools • While LLMs can improve productivity, it’s not the be-all-end-all AI tools lower the activation energy to get started!!! ChatGPT for Students