從 Gemini API 到 LINE Bot

隨著AI工具的出現，各種應用迅速崛起，我們將展示如何低成本打造個人QA機器人，建立客戶支援生態系統的AI原型服務。在當前技術迅速發展的背景下，自動化與智慧型客戶服務解決方案日益顯得重要。Gemini API與Bot為開發者提供了一條低成本、高效率建立QA機器人、打造客戶支援生態系統AI原型的途徑。

此次分享聚焦兩大主題：

了解Gemini API和Line Bot基礎：Gemini API提供廣泛的數據和服務存取能力，Line Bot則是亞洲市場上受歡迎的溝通平台。開發者需要先瞭解這些技術的功能、限制與最佳實踐。

設計用戶體驗：開發QA機器人時，考慮用戶互動方式極為關鍵。這包括創建自然直觀的對話、友好的歡迎信息、理解自然語言的能力，以及提供快速準確的回應。透過第三方服務與Gemini的結合，可以讓開發者連接到公司數據庫或其他API，擴展機器人的功能和回答範圍。

The advent of AI tools has led to a surge in applications, including the creation of low-cost, self-built QA Bots for customer support ecosystems. In this fast-evolving tech landscape, the importance of automated and smart customer service solutions is growing. Gemini API and Bot present a unique chance for developers to create efficient, cost-effective QA robots, establishing prototype AI customer service systems.

This presentation focuses on two main areas:

Understanding Gemini API and Line Bot Basics: Gemini API provides extensive access to data and services, and Line Bot is a popular communication platform in Asia. Developers must first grasp these technologies' capabilities, limitations, and best practices.

Designing User Experience: When developing QA robots, it's essential to consider user interaction. This involves creating natural and intuitive dialogues, friendly greetings, understanding natural language, and delivering quick, accurate responses. Leveraging third-party services with Gemini can extend the robot's functionalities and response range by connecting to databases and other APIs.

Transcript

1. Caesar Chi 2024/03 從 Gemini API 到 Line Bot Build with AI 2024 Taichung https://gdg.community.dev/events/details/google-gdg-taichung-presents-build-with-ai-2024-taichung-3/
2. http://exma-square.co/
3. Caesar Chi clonncd ⼯程師成長⼿冊 clonn
4. 個⼈經歷 • Career • 2024 - EXMA-Square • 2021 - TransIot — CTO • 2020 - Undercover - Tech infra • 2018 - Awoo Tech Manager • 2017 - EXMA-Square • 2016 - Hiiir Tech Manager • 2014 - Mitac full-stack developer • 2012 - Dlink Front end developer • Community • JSDC core-team Caesar
5. Why Gemini
8. https://cloud.google.com/generative-ai-studio?hl=zh-tw
10. https://aistudio.google.com/app/prompts/new_chat?hl=zh-tw
13. API 版本說明 - 多模態
14. Gemini 在 Node.js const { GoogleGenerativeAI } = require("@google/generative-ai"); // Access your API key as an environment variable (see "Set up your API key" above) const genAI = new GoogleGenerativeAI(process.env.API_KEY);
15. const { GoogleGenerativeAI } = require("@google/generative-ai"); // Access your API key as an environment variable (see "Set up your API key" above) const genAI = new GoogleGenerativeAI(process.env.API_KEY); async function run() { // For text-only input, use the gemini-pro model const model = genAI.getGenerativeModel({ model: "gemini-pro"}); const prompt = "Write a story about a magic backpack." const result = await model.generateContent(prompt); const response = await result.response; const text = response.text(); console.log(text); } run();
16. Gemini 多模態
17. // Converts local file information to a GoogleGenerativeAI.Part object. function fileToGenerativePart(path, mimeType) { return { inlineData: { data: Buffer.from(fs.readFileSync(path)).toString("base64"), mimeType }, }; } async function run() { // For text-and-image input (multimodal), use the gemini-pro-vision model const model = genAI.getGenerativeModel({ model: "gemini-pro-vision" }); const prompt = "What's different between these pictures?"; const imageParts = [ fileToGenerativePart("image1.png", "image/png"), fileToGenerativePart("image2.jpeg", "image/jpeg"), ]; const result = await model.generateContent([prompt, ...imageParts]); const response = await result.response; const text = response.text(); console.log(text);
18. // Converts local file information to a GoogleGenerativeAI.Part object. function fileToGenerativePart(path, mimeType) { return { inlineData: { data: Buffer.from(fs.readFileSync(path)).toString("base64"), mimeType }, }; } async function run() { // For text-and-image input (multimodal), use the gemini-pro-vision model const model = genAI.getGenerativeModel({ model: "gemini-pro-vision" }); const prompt = "What's different between these pictures?"; const imageParts = [ fileToGenerativePart("image1.png", "image/png"), fileToGenerativePart("image2.jpeg", "image/jpeg"), ]; const result = await model.generateContent([prompt, ...imageParts]); const response = await result.response; const text = response.text(); console.log(text);
19. async function run() { // For text-only input, use the gemini-pro model const model = genAI.getGenerativeModel({ model: "gemini-pro"}); const chat = model.startChat({ history: [ { role: "user", parts: [{ text: "Hello, I have 2 dogs in my house." }], }, { role: "model", parts: [{ text: "Great to meet you. What would you like to know?" }], }, ], generationConfig: { maxOutputTokens: 100, }, }); const msg = "How many paws are in my house?"; const result = await chat.sendMessage(msg); const response = await result.response; const text = response.text();
20. //... const result = await model.generateContentStream([prompt, ...imageParts]); let text = ''; for await (const chunk of result.stream) { const chunkText = chunk.text(); console.log(chunkText); text += chunkText; } //...
22. 安全性篩選器
23. https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/con fi gure-safety-attributes
24. https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/con fi gure-safety-attributes
25. // Initialize Vertex with your Cloud project and location const vertexAI = new VertexAI({project: projectId, location: location}); // Instantiate the model const generativeModel = vertexAI.getGenerativeModel({ model: model, // The following parameters are optional // They can also be passed to individual content generation requests safety_settings: [ { category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE, }, ], generation_config: {max_output_tokens: 256}, }); const request = { contents: [{role: 'user', parts: [{text: 'Tell me something dangerous.'}]}], };
26. Why LINE Bot
28. https://today.line.me/tw/v2/article/Qw3PmQe
31. How to build it?
32. https://ai.google.dev/docs?hl=zh-tw
33. LINE BOT 模式設定 • 純⽂字模式 • 多重聊天內容模式 • 圖片 / 影片模式
34. https://aistudio.google.com/app/prompts/new_chat?hl=zh-tw
38. Node.js
39. Node.js
40. 純⽂字模式 if (event.message.type === "text") { const msg = await gemini.textOnly(event.message.text); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); }
41. https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase
42. 多模態輸入 if (event.message.type === "image") { const imageBinary = await line.getImageBinary(event.message.id); const msg = await gemini.multimodal(imageBinary); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); } JavaScript
43. https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase
44. 聊天模式 if (event.message.type === "text") { const msg = await gemini.chat(event.message.text); await line.reply(event.replyToken, [{ type: "text", text: msg }]); return res.end(); }
45. https://techblog.lycorp.co.jp/zh-hant/linebot-x-gemini- fi rebase
46. Reference • https://github.com/kkdai/linebot-smart-namecard • https://github.com/babaohuang/GeminiProChat • https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web
47. Limitation?
48. https://ai.google.dev/models/gemini?hl=zh-tw
50. https://ai.google.dev/docs/migrate_to_cloud?hl=zh-tw
51. https://blog.jason.tools/2016/07/The-most-expensive-free.html
52. No Code Builder
53. 技術投資有賺有賠 開發之前請詳細參閱 README
54. botsonic https://writesonic.com/botsonic
55. TAR
56. chatfuel
57. botcore
58. Think about …
63. Node.js
66. Node.js
67. AI for Developer Taiwan https://www.facebook.com/groups/1362642437933526/ 「AI for Developer」！請點選以下連結加入社群！ https://bit.ly/4az4vJt https://line.me/ti/g2/D2m1DA83Qb0KJCjd2mLyHD7y4yB1CA0WUJsOMA? utm_source=invitation&utm_medium=link_copy&utm_campaign=default
68. Caesar Chi clonncd ⼯程師成長⼿冊 clonn
69. http://exma-square.co/