Get started with the Gemini API in Android apps (client SDK)

Learn how to build mobile applications using Google's Gemini API in Kotlin.

Transcript

1. Love Otudor Get started with the Gemini API in Android apps (client SDK) Android Developer, FrontendLabs @lamoureSparus
2. Google Cloud Proprietary & Confidential Set up your project and your API key Generate text from text-only input 01 02 03 04 In this codelab, you'll learn how to: 2 Build multi-turn conversations (chat) Generate text from text-and-image input (multimodal) Build with AI
3. Google Cloud Proprietary & Confidential Demo: 3 Build with AI
4. Google Cloud Proprietary & Confidential Demo: 4 Build with AI
5. Prerequisites Build with AI 1. Familiar with using Android Studio to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher. The client SDK for Android described in this tutorial lets you access the Gemini Pro models which run on Google's servers. For use cases that involve processing sensitive data, offline availability, or for cost savings for frequently used user flows, you may want to consider accessing Gemini Nano which runs on-device. For more details, refer to the Android (on-device) tutorial.
6. Google Cloud Proprietary & Confidential Prerequisites 6 Build with AI 1. Familiar with using Android Studio to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher.
7. Google Cloud Proprietary & Confidential 01 Set up your project and your API key 7 Build with AI
8. Google Cloud Proprietary & Confidential Set up your project and your API key 8 Build with AI 1. Create a key in Google AI Studio. bit.ly/GeminiApiKey
9. Google Cloud Proprietary & Confidential Set up your project and your API key 9 Build with AI 2. Click on create API key.
10. Google Cloud Proprietary & Confidential Set up your project and your API key 10 Build with AI 3. Create an API key in a new project if you don't have one already Or Select a project from your existing write-access Google Cloud projects. 4. Copy your API key for use.
11. Google Cloud Proprietary & Confidential Set up your project and your API key 11 Build with AI 5. Scan the starter project. bit.ly/GeminiAndroid
12. Google Cloud Proprietary & Confidential 12 Starter Code Walkthrough
13. Google Cloud Proprietary & Confidential Set up your project and your API key 13 Build with AI 6a. Add Secrets Gradle plugin for Android. In your project's root build.gradle file: We are using Gradle version catalogs to maintain dependencies and plugins in a scalable way.
14. Google Cloud Proprietary & Confidential Set up your project and your API key 14 Build with AI 6b. Add Secrets Gradle plugin for Android. In your app-level build.gradle file:
15. Google Cloud Proprietary & Confidential Set up your project and your API key 15 Build with AI 7. Enable build config in your module's build.gradle:
16. Google Cloud Proprietary & Confidential Set up your project and your API key 16 Build with AI 8. Add and Secure your API key. bit.ly/SecureApiKey
17. Google Cloud Proprietary & Confidential 02 Generate text from text-only input 17 Build with AI
18. Google Cloud Proprietary & Confidential Generate text from text-only input 18 Build with AI In TextViewModel: 1. Initialize the Generative Model using gemini-1.0-pro as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
19. Google Cloud Proprietary & Confidential Generate text from text-only input 19 Build with AI 3. Create a function summarize, that gets the inputText to be summarized as String: a. Create a String value prompt of "Summarize the following text for me: " and append the inputted string to it. b. In a coroutine, try to get the generated response by calling generativeModel.generateContent(prompt). c. Update the UiState with the response.
20. Google Cloud Proprietary & Confidential Your function should look like this: 20 Build with AI
21. Google Cloud Proprietary & Confidential 21 Build with AI In TextScreen() composable: 4. On click of the Submit button, call Gemini to summarize the text from prompt. 5. Update the UI accordingly. 6. Check out Solution_1_Generate-text-from-text-only-input branch for the full code. Generate text from text-only input
22. Google Cloud Proprietary & Confidential 03 Generate text from text-and-image input 22 Build with AI
23. Google Cloud Proprietary & Confidential Supported Image Formats 23 Build with AI
24. Prerequisites Build with AI 1. Familiar with using Android Studio to develop Android apps. 2. Android Studio (latest version) 3. Your Android app must target API level 21 or higher. Prompts using the Gemini API cannot exceed 20MB in size. The Gemini API provides a File API for temporarily storing media files for use in prompting, which lets you provide prompt data beyond the 20MB limit. For more information on using the Files API and file formats supported for prompting, see Prompting with media files.
25. Google Cloud Proprietary & Confidential Generate text from text-and-image input (multimodal) 25 Build with AI In PhotoReasoningViewModel: 1. Initialize the Generative Model using gemini-1.0-pro-vision-latest as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
26. Google Cloud Proprietary & Confidential 26 Build with AI 3. Create a function reason, that gets the userInput text and selectedImages as a list of bitmap: a. Create a string value prompt of "Look at the image(s), and then answer the following question: " and append the inputted string to it. b. Create an inputContent value for the model, adding each image and the prompt, using content { }. Generate text from text-and-image input (multimodal)
27. Google Cloud Proprietary & Confidential 27 Build with AI 3. Cont. c. In a coroutine, try to get the generated response by calling generativeModel.generateContent(inputContent). d. Update the UiState with the response. Generate text from text-and-image input (multimodal)
28. Google Cloud Proprietary & Confidential Your function should look like this: 28 Build with AI
29. Google Cloud Proprietary & Confidential 29 Build with AI In PhotoScreen composable: 4. Call the reason function in the onReasonClicked lambda of PhotoReasoningScreen and pass in the selected images and input text. 5. Update the UI accordingly. 6. Check out Solution_2_Multimodal branch for the full code. Generate text from text-and-image input (multimodal)
30. Google Cloud Proprietary & Confidential 04 Build multi-turn conversations (chat) 30 Build with AI
31. Google Cloud Proprietary & Confidential Build multi-turn conversations (chat) 31 Build with AI In ChatViewModel: 1. Initialize the Generative Model using gemini-1.0-pro as modelName. 2. Access your API key as a Build Configuration variable, using BuildConfig.apiKey
32. Google Cloud Proprietary & Confidential 32 Build with AI 3. (Optional) Create a default chat history, which is a list of Content like so: Build multi-turn conversations (chat)
33. Google Cloud Proprietary & Confidential 33 Build with AI 4. Create a chat instance value called chat, to track the ongoing conversation by calling generativeModel.startChat(defaultHistory). You can pass in the defaultHistory if you created one. Build multi-turn conversations (chat)
34. Google Cloud Proprietary & Confidential 34 Build with AI 5. Create a function sendMessage, that has a String parameter called userMessage: a. Update the UiState by adding a pending User ChatMessage with userMessage as its text. By calling _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
35. Google Cloud Proprietary & Confidential 35 Build with AI 5. Cont. b. In a coroutine, try to get the generated response by calling chat.sendMessage(userMessage). c. Call the replaceLastPendingMessage() to update the last users message as not pending. Build multi-turn conversations (chat)
36. Google Cloud Proprietary & Confidential 36 Build with AI 5. Cont. d. If the generated response text is not null, update the UiState with a Model ChatMessage having the generated response as its text. _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
37. Google Cloud Proprietary & Confidential 37 Build with AI 5. Cont. e. In your catch block, call: replaceLastPendingMessage() and update the Uistate by adding an Error ChatMessage with the exception’s localizedMessage as its text. _uiState.value.addMessage(ChatMessage(...)). Build multi-turn conversations (chat)
38. Google Cloud Proprietary & Confidential Your function should look like this: 38 Build with AI
39. Google Cloud Proprietary & Confidential 39 Build with AI In ChatScreen composable: 6. In the ChatList composable, replace the dummyChatHistory with the messages from the UiState. 7. Call the chatViewmodel’s sendMessage function in the onSendMessage lambda of the MessageInput composable and pass in the input text. Build multi-turn conversations (chat)
40. Google Cloud Proprietary & Confidential 40 Build with AI In ChatScreen composable: 8. Check out Solution_3_Chat branch for the full code. Build multi-turn conversations (chat)
41. Thank you Build with AI