Deep Dive into Google AI Studio (By: Mashhood Rastgar) - DevFest Lahore 2023

Workshop by Mashhood Rastgar (https://www.linkedin.com/in/mashhoodr/) at DevFest Lahore 2023 by GDG Lahore.

Transcript

1. Google AI Studio Deep Dive Mashhood Rastgar Head of Engineering @ Taleemabad Generated by Imagen 2
2. Hello! I am Mashhood. I am currently Head of Engineering at Taleemabad where we are infusing AI with education, And community leader as a Google Developer Expert, I also run a micro-podcast, karachiwala.dev
3. How many people here are using Bard? bard.google.com
4. Understanding Generative AI
5. https://towardsdatascience.com/designing-your-neural-networks-a5e4617027ed?gi=8d1d6c92c512
8. What is a LLM? https://rpradeepmenon.medium.com/introduction-to-large-language-models-and-the-transformer-architecture-534408ed7e61 [... ] [... ] [... ] [... ] 0.02 0.03 0.9 0.01 0.0 … Dogs Rain Drops Fish Wind … and cats raining It’s
9. Generated by Imagen 2
10. https://rpradeepmenon.medium.com/introduction-to-large-language-models-and-the-transformer-architecture-534408ed7e61
11. What are tokens? Tokens can be thought of as pieces of words. 1 token ~= 4 chars in English 1 token ~= ¾ words 100 tokens ~= 75 words Or 1-2 sentence ~= 30 tokens 1 paragraph ~= 100 tokens 1,500 words ~= 2048 tokens
12. What are parameters? https://rpradeepmenon.medium.com/introduction-to-large-language-models-and-the-transformer-architecture-534408ed7e61
15. https://www.deepset.ai/blog/llm-finetuning
16. Multi-modal Capabilities. https://www.marktechpost.com/2023/09/14/meet-next-gpt-an-end-to-end-general-purpose-any-to-any-multimodal-
17. Beware of Hallucinations. https://www.simform.com/blog/llm-hallucinations/
18. Open source and private models.
22. Foundation models vs Fine-tuned models. Foundational models offer broad knowledge, like a library, while fine-tuned models are specialized experts, trained on specific tasks for higher accuracy. The ability to “ask questions” is something not present in a foundational model. Lama2 by Meta has released both foundational and fine tuned models. https://medium.com/mantisnlp/supervised-fine-tuning-customizing-llms-a2c1edbf22c3
24. github.com/jmorganca/ollama
26. What is a LangChain? LangChain is an open-source framework designed to simplify the creation of applications using large language models (LLMs). It provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.
27. Google AI Studio Demo
28. Prompt Engineering.
30. The prompt below is an attempt to complete a sentiment analysis task using a zero-shot prompt. Text: i'll bet the video game is a lot more fun than the film. Sentiment: Zero-shot Prompts
31. The prompt below is an attempt to complete a sentiment analysis task using a few-shot prompt. Text: (lawrence bounces) all over the stage, dancing, running, sweating, mopping his face and generally displaying the wacky talent that brought him fame in the first place. Sentiment: positive Text: despite all evidence to the contrary, this clunker has somehow managed to pose as an actual feature movie, the kind that charges full admission and gets hyped on tv and purports to amuse small children and ostensible adults. Sentiment: negative Text: for the first time in years, de niro digs deep emotionally, perhaps because he's been stirred by the powerful work of his co-stars. Sentiment: positive Text: i'll bet the video game is a lot more fun than the film. Sentiment: Few-shot Prompt
32. Few shot prompting can be expensive due to a larger context, we can try giving the instruction directly. Please label the sentiment towards the movie of the given movie review. The sentiment label should be "positive" or "negative". Text: i'll bet the video game is a lot more fun than the film. Sentiment: Describe what is quantum physics to a 6-year-old. Instruction Prompting
34. Tree of Thoughts Prompting https://www.promptingguide.ai/techniques/tot
35. https://github.com/OpenBMB/ChatDev https://github.com/Significant-Gravitas/AutoGPT
36. Building RAG Pipelines
37. What is a Embedding? https://www.pinecone.io/learn/vector-embeddings/
38. What is a Vector Database? https://www.pinecone.io/learn/vector-embeddings/
39. https://www.pinecone.io/learn/vector-database/
41. Insert into Pinecone. You can add one vector at a time into PineconeDB. However in our case we had several thousand vectors generated for our content so we opted for their upsert API which allows for 100 vectors at a time.
42. Search with Pinecone.
43. Resources - Introduction to Generative AI (https://www.cloudskillsboost.google/paths/118) - Prompting Guide (https://www.promptingguide.ai/) - ChatGPT Prompt Engineering for Developers (https://www.deeplearning.ai/short-courses/)
44. Thank you!
48. Activity (10 mins) - Visit bard.google.com - Let’s write some prompts to get comfortable - Try out the multi-modal capability - Make the model hallucinate - Bonus: Ask it some math questions!
49. LLMs are great for… Entity extraction Classification Summarization Sentiment Analysis Translation …
51. 1. Prompt to generate a quiz. 2. Prompt to generate a quiz in json. You are a product marketer targeting a Gen Z audience. Create exciting and fresh advertising copy for products and their simple description. Keep copy under a few sentences long. Let’s build a quiz app.
52. 1. Prompt to generate a script for the video. 2. Prompt to generate a quiz in json. 3. Let’s build a automated animation.
53. Let’s understand them settings… • Temperature. • Max Outputs. • Safety Settings. • Top K • Top P • Stop Sequence https://www.marktechpost.com/2023/09/14/meet-next-gpt-an-end-to-end-general-purpose-any-to-any-multimodal-
54. Types of inputs. • Free form prompts • Structured prompts • Chat prompts https://www.marktechpost.com/2023/09/14/meet-next-gpt-an-end-to-end-general-purpose-any-to-any-multimodal-
55. Activity (15 mins) • Build something
56. What is a transformer? Input embeddings represent words as numbers, which machine learning models can then process. These embeddings are like a dictionary that helps the model understand the meaning of words by placing them in a mathematical space where similar words are located near each other. https://rpradeepmenon.medium.com/introduction-to-large-language-models-and-the-transformer-architecture-534408ed7e61
57. https://medium.com/@zafaralibagh6/simple-tutorial-on-word-embedding-and-word2vec-43d477624b6d
58. Use-cases Recommendations Question answering Informational retrieval
59. Vector Databases
61. Defining the book mapping problem.
63. Dynamic Book Mapping We have 100s of publishers which issue books with for the same SNC Student Learning Objectives. How can we automatically map our Lesson Plans to these book chapters?
64. Building Semantic Search
65. Introducing Maker Suite!
67. Semantic Matching. Using Google’s PaLM API we can generate embedding.
68. Results? ❏ Very effective for some subjects / grades. ❏ OpenAI wins on multi-lingual mapping (Islamiat is in Urdu) ❏ Some LLM models work better with some subjects. ❏ Objective is to get around 90% accuracy in mappings! ❏ Right now we are using it to recommend SLOs, however in the future we intend to fully automate.
69. youtube.com/watch?v=zjkBMFhNj_g
70. Rinse and repeat with other LLMs. ❏ OpenAI ❏ Hugging Face ❏ minilm ❏ Mpnet ❏ Lama2 70b ❏ Falcon 180b ❏ Alpaca 13b . . . https://www.searchenginejournal.com/new-open-source-llm-with-zero-guardrails-rivals-google-palm-2/496212/#close