Generative AI to you Android with Gemini API by Eric Ampire

DevFest Montreal talk 2024 :
https://youtu.be/CCm97du5ido

Transcript

1. Generative AI to your Android with Gemini API Sherbrooke Eric Ampire
2. About me
3. About me Mobile Engineer at Bombardier Recreational Products (BRP) - Google Developer Expert for Android
5. Want to reach out Scan the QR Code
6. What’s Gemini ? Montréal
7. Montréal A family of multimodal large language models developed by Google DeepMind and Google Research
8. Gemini is multi-modal Large Language Model
9. Gemini is multi-modal Large Language Model
10. (Android AICore)
11. Gemini Nano
12. Gemini Nano Most efficient model for on-device tasks
13. Gemini Pro
14. Gemini Pro Best model for scaling across a wide range of tasks
15. Gemini Ultra
16. Gemini Ultra Largest and most capable model for highly complex tasks.
17. Potential Applications
18. Potential Applications • Education : Personalized learning experiences, interactive tutoring knowledge.
19. Potential Applications • Education : Personalized learning experiences, interactive tutoring knowledge. • Healthcare: Assisting doctors, analyzing medical data, and enhancing coms.
20. Potential Applications • Education : Personalized learning experiences, interactive tutoring knowledge. • Healthcare: Assisting doctors, analyzing medical data, and enhancing coms. • Customer service : Personalized interactions, resolving complex queries, and improving customer satisfaction.
21. Montréal Gemini Nano
22. Montréal AI Core
23. AI Core • Android AICore, a new system service for on-device foundation models starting from Android 14
24. AI Core • Android AICore, a new system service for on-device foundation models starting from Android 14 • AICore enables Low Rank Adaptation (LoRA) fine tuning with Gemini Nano.
25. AI Core • Android AICore, a new system service for on-device foundation models starting from Android 14 • AICore enables Low Rank Adaptation (LoRA) fine tuning with Gemini Nano. • AICore takes advantage of new ML hardware like the latest Google Tensor TPU and NPUs in flagship Qualcomm Technologies, Samsung S.LSI and MediaTek silicon
26. Architecture
27. Privacy: your data is not sent to a remote server Advantage
28. Privacy: your data is not sent to a remote server Fast, everything is process On-Device Advantage
29. Privacy: your data is not sent to a remote server Fast, everything is process On-Device Indirect Internet Access Advantage
30. Privacy: your data is not sent to a remote server Fast, everything is process On-Device Indirect Internet Access Advantage AICore does not have direct internet access. All internet requests, including model downloads, are routed through the open-source Private Compute Services
31. What is Private Compute Core • Android’s Private Compute Core is an open source, secure environment that is isolated from the rest of the operating system and apps
32. What is Private Compute Core • Android’s Private Compute Core is an open source, secure environment that is isolated from the rest of the operating system and apps • Introduced with Android 12, It is a secure environment that isolates sensitive data to perform private computations while ensuring that this data never leaves the device
33. Private Compute Core - Limited data access • The Private Compute Core is completely isolated from the rest of the system and from third-party applications, which means that not even Google has access to the data. Only local machine learning APIs, which are strictly controlled, can interact with private data.
34. Private Compute Core - Indirect Internet Access • AICore does not have direct internet access. All internet requests, including model downloads, are routed through the open-source Private Compute Services companion APK
35. Private Compute Core - Indirect Internet Access • AICore does not have direct internet access. All internet requests, including model downloads, are routed through the open-source Private Compute Services companion APK • Learn more : bit.ly/private-compute
36. Text Rephrasing: Modify the tone and style of text Use cases
37. Text Rephrasing: Modify the tone and style of text Smart Reply: Generate relevant responses within a chat Use cases
38. Text Rephrasing: Modify the tone and style of text Smart Reply: Generate relevant responses within a chat Proofreading: Correct spelling and grammatical errors. Use cases
39. Text Rephrasing: Modify the tone and style of text Smart Reply: Generate relevant responses within a chat Proofreading: Correct spelling and grammatical errors. Condense lengthy documents into concise summaries Use cases
40. Use cases - Recorder app
41. Use cases - Smart Reply in Gboard
42. Montréal Gemini Nano is currently in Private Preview
43. Gemini Nano is currently in Private Preview • Supported Devices: AICore is currently available on Pixel 9 series devices, Google Pixel 8 Series devices, Samsung S24 Series devices, Realme GT 6, Motorola Edge 50 Ultra, Motorola Razr 50 Ultra, Xiaomi 14T/Pro, and Xiaomi MIX Flip
44. Gemini Nano is currently in Private Preview • Supported Devices: AICore is currently available on Pixel 9 series devices, Google Pixel 8 Series devices, Samsung S24 Series devices, Realme GT 6, Motorola Edge 50 Ultra, Motorola Razr 50 Ultra, Xiaomi 14T/Pro, and Xiaomi MIX Flip • Supported Modalities: AICore currently supports text modality for Gemini Nano
45. Montréal Getting started with the Gemini API
47. This is an example of a Cloud-specific section title slide.
48. This is an example of a Cloud-specific section title slide.
49. This is an example of a Cloud-specific section title slide.
50. This is an example of a Cloud-specific section title slide.
51. This is an example of a Cloud-specific section title slide.
52. This is an example of a Cloud-specific section title slide.
53. Montréal Getting started with the Gemini API in Android
54. Add the SDK dependency to your project dependencies { // add the dependency for the Google AI client SDK for Android implementation("com.google.ai.client.generativeai:generativeai:0.7.0") }
55. Initialized the Generative Model val model = GenerativeModel( // Specify a Gemini model appropriate for your use case modelName = "gemini-1.5-flash", // Access your API key as a Build Configuration variable apiKey = "gemini-key" )
56. Initialized the Generative Model with More settings val generationConfig = generationConfig { temperature = 0.15f topK = 32 topP = 1f maxOutputTokens = 4096 }
57. Initialized the Generative Model with More settings val safetySettings = listOf( SafetySetting( HarmCategory.HARASSMENT, BlockThreshold.MEDIUM_AND_ABOVE ), SafetySetting( HarmCategory.HATE_SPEECH, BlockThreshold.MEDIUM_AND_ABOVE ) )
58. Initialized the Generative Model with More settings val model = GenerativeModel( model = "gemini-1.5-flash-001", apiKey = "gemini-key", generationConfig = generationConfig, safetySettings = safetySettings )
59. Initialized the Generative Model val model = GenerativeModel( model = "gemini-1.5-flash-001", apiKey = "gemini-key", generationConfig = generationConfig, safetySettings = safetySettings )
60. Write your first prompt val prompt = "Write a story about a magic backpack." coroutineScope.launch { val response = model.generateContent(prompt) }
61. Generate text from images and other media coroutineScope.launch { val response = model.generateContent( content { image(bitmap) text("What is the object in this picture?") } ) }
62. Multi-turn chat You can also support multi-turn conversations. Initialize a chat with the startChat() function. You can optionally provide a message history. Then call the sendMessage() function to send chat messages.
63. Multi-turn chat val chat = model.startChat( history = listOf( content(role = "user") { text("Hello, I have 2 dogs in my house.") }, content(role = "model") { text("What would you like to know?") } ) ) coroutineScope.launch { val response = chat.sendMessage("How many paws are in my house?") }
64. Stream the response You can achieve faster interactions by not waiting for the entire result from the model generation, and instead use streaming to handle partial results. Use generateContentStream() to stream a response.
65. Stream the response coroutineScope.launch { var outputContent = "" model.generateContentStream(inputContent).collect { response -> outputContent += response.text } }
66. Demo Montréal
67. Summary Gemini is a family of multimodal large language models developed by Google DeepMind and Google Research
68. Code Sherbrooke
69. Thanks Sherbrooke