Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Geminews + Gemini 2.0 demo

Geminews + Gemini 2.0 demo

News Crawler via Langchain.RB and Gemini APIs

How can we get an LLM to be updated to today’s news? Gen AI is great at answering questions.. from the past. After the LLM was trained, all you can do is RAG. How about crawling the web for latest news with Gemini for multimodal extraction and offering summarization by your favorite topic? It all gets more exciting thanks to Andrei’s langchainrb gem.

Code: https://github.com/palladius/gemini-news-crawler

Intended audience: Ruby lovers, and/or Gen AI aficionados looking for a simple and a more visionary RAG implementation.

Avatar for Riccardo Carlesso

Riccardo Carlesso

March 10, 2025
Tweet

More Decks by Riccardo Carlesso

Other Decks in Programming

Transcript

  1. 📰News Crawler🕷 via 🐍🔗Langchain and ♊Gemini APIs Function calling and

    Embeddings made easy with Langchain and Gemini. In python, of course. This is a Python community, right? Warning: sarcasm is all around. Zurich, GDG Cloud, 19 Dec 2024 Warning: sarcasm detected
  2. 📰News Crawler🕷 via 💎🔗Langchain.RB and ♊Gemini APIs Function calling and

    Embeddings made easy with Langchain and Gemini. Yep, with ruby. Proprietary & Confidential Zurich, GDG Cloud, 19 Dec 2024
  3. 🐦 @SREccardo www.linkedin.com/in/riccardocarlesso/ https://www.linkedin.com/in/riccardocarlesso/ Former sysadmin, Ruby [on Rails] developer,

    network admin. 11-years Googler, passionate about Reliability, SRE, DevOps, Maths, yellow, 🎹 piano, triathlons 󰖱🏃󰚬, caipirinhas and Amarone. As a Developer🥑Advocate for Google Cloud, I preach SRE, DevOps, the Google Way of doing things to various audiences. About me
  4. Google Cloud Proprietary & Confidential Contents 01 Intro to GenAI

    02 The Problem 03 NewCrawler app 04 Four Demos! 4
  5. Google Cloud Proprietary & Confidential Questions for you 1. Are

    you familiar with the definition of LLM ? 2. Are you familiar with the definition of Embedding ? 3. Do you use a Vector DB for your daily job ? 4. Are you using Prompts for work? (Not just to plan your holidays or writing emails for you) a. Do you do Prompt Engineering? 5
  6. Google Cloud Proprietary & Confidential 01 What’s GenAI? Content generation,

    Embeddings, functions, and Gemini I know you probably know it already. But still, bear with me, ok? 6
  7. Google Cloud Artificial Intelligence NLP 7 Artificial Intelligence, Machine Learning,

    Data Science, Deep Learning… Machine Learning — unsupervised, supervised, reinforcement learning Deep Learning — ANN, CNN, RNN… Generative AI — GAN, VAE, Transformers… LLMs — Transformers BERT, GPT-3, LaMDA (Bard), Megatron, Bloom, Wav2Vec, Jurassic-1, PaLM, LlaMa, .. Image Gen — GAN, VAE DALL E, Firefly, Getty, Imagen, Midjourney, Stable Diffusion, ..
  8. Is this a cat? This is a cat. This is

    not a cat. Yes. Go read this huge pile of books. So, you’ve learned about cats and a ton of other concepts. What’s a cat? A cat is… Traditional Programming Wave of Neural Networks (~2012) Generative Language Models
  9. It helps to ______________ while I ____________. LLMs (Large Language

    Models): LLMs have no concept of logic, facts, or meaning beyond what's learnable by probabilities of proximal words Generative AI => A smart autocomplete
  10. Gemini is the most capable and general model we’ve ever

    built, and is the result of a large-scale collaborative effort by teams across Google, including Google DeepMind and Google Research. NEEDS ART The next chapter of Generative AI innovation
  11. Proprietary + Confidential Embeddings Embeddings are a way of representing

    data–almost any kind of data, like text, images, videos, users, music, whatever–as points in space where the locations of those points in space are semantically meaningful. The best way to intuitively understand what this means is by example, so let’s take a look at one of the most famous embeddings, Word2Vec. Word2Vec (short for word to vector) was a technique invented by Google in 2013 for embedding words. It takes as input a word and spits out an n-dimensional coordinate (or “vector”) so that when you plot these word vectors in space, synonyms cluster.
  12. Proprietary + Confidential PostgreS + pgvector Pgvector in PostgresSQL •

    Open-source vector similarity search for Postgres • Store your vectors with the rest of your data • See https://github.com/pgvector/pgvector/ $ rails generate neighbor:vector $ rails db:migrate In PostgreS: In Rails:
  13. Proprietary + Confidential Embeddings in Rails GemiNews • Postgres •

    Pgvector https://github.com/pgvector/pgvector/ Embedding model From app/models/concerns/embeddable.rb: For Articles similarity: textembedding-gecko-multilingual Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings Field used: just :title (why?) Gem Neighbor https://github.com/ankane/neighbor • Gets nearest neighbor to ActiveRecord or vector • Supports: L2 distance, cosine, … • Records returned from nearest_neighbors will have a neighbor_distance attribute 🫨
  14. Proprietary + Confidential GemiNews: 5 article neighbors 5 neighbors 5

    distances (injected to AR by neighbor gem - try this at home, Python!) Example: https://gemini-news-crawler-dev-x42ijqglgq-ew.a.run.app/articles/10364
  15. Google Cloud Proprietary & Confidential LLMs ain’t perfect.. • Fast-changing

    • Lack of tooling • IP/Copyright issues • Risky • Hallucinations • Jailbraking • Unreliable / unpredictable ◦ 🤞 for my demo! • Outdated data • Relevant knowledge is not used 25
  16. How do we fix LLMs tendency to hallucinate and not

    be up-to-date? a Prompt Reliability Engineer
  17. Google Cloud Proprietary & Confidential Month date Lorem ipsum dolor

    sit amet 28 Month date Lorem ipsum dolor sit amet Month date Lorem ipsum dolor sit amet Month date Lorem ipsum dolor sit amet Month date Lorem ipsum dolor sit amet Month date Lorem ipsum dolor sit amet Month date Lorem ipsum dolor sit amet RAG: Retrieval Augmented Generation
  18. Google Cloud Proprietary & Confidential 31 Solution 2: function calling…

    … later (yes, you can do RAG with it, too!) https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling
  19. Google Cloud Proprietary & Confidential The app • crawler/: 1

    crawler (ruby) ◦ Just to quickly feed 10k articles to ActiveRecord (feedjira) ◦ Parses dozens of RSS randomly assembled. ◦ Calculated embeddings for 10k articles. • webapp/: 1 Rails App ◦ Ruby version: 3.2.2 ◦ Rails version: 7.1.3.2 ◦ Database: 🐘 PostgreS ◦ Efficiently Dockerized ◦ Builds in ~4m30s ◦ Plenty of ENVs. Code: https://github.com/palladius/gemini-news-crawler/ 33
  20. Do you ever read technical articles where amazing Generative programs

    are built with 🦜🔗Langchain? -- a jealous Ruby dev
  21. Google Cloud Proprietary & Confidential LangchainRB by Andrei Bondarev (Source

    Labs) - Ruby “equivalent” of Langchain, but so much more. - Support most LLMs (Anthropic, Gemini, Mistral, Ollama, OpenAI, ..) - Supports Function Calling, Chatting, Embeddings, Agents/Assistants, Vector Search - Assistant Tools: database, Google Search, weather, Wikipedia, … - MIT license - Actively Maintained -> discord 37 github.com/patterns-ai-core/langchainrb GeminiLLM = Langchain::LLM::GoogleGemini.new(api_key: gemini_key)
  22. Google Cloud Proprietary & Confidential Deployment • Google Cloud Build

    -> Cloud Run (google-way knative) ◦ https://gemini-news-crawler-dev-x42ijqglgq-ew.a.run.app/ ◦ https://gemini-news-crawler-prod-x42ijqglgq-ew.a.run.app/ • Triggered on push to main on github repo [CB 🏛] 38
  23. Google Cloud Proprietary & Confidential Some stats 39 Apps Crawler

    + webapp 2 Commits To github 277 Builds on Cloud Build 226 Branch Just main 1
  24. Google Cloud Proprietary & Confidential The app in brief •

    A PostgreS DB with 10’000 Articles from April 2024. • Each article has 3 precomputed embeddings. ◦ Title, summary, whole article. • Lot of monkeypatching. • Some 💔 broken functionality • Demo proof - I wish! 41
  25. Google Cloud Proprietary & Confidential The ugly 💔 • Crawler

    disregarded here. ◦ Only works in localhost (hard to deploy bg jobs in the cloud) ◦ Writes to GCS but I’m not taking the data from there • UI is UGLY essential ◦ No responsive frontend apart from what Tailwind provides. • Lot of monkeypatching ◦ Langchain (too slow to file PRs and wait for main -> new gem) ◦ Syntactic sugar (nice demo printfs) 42
  26. Google Cloud Proprietary & Confidential 04 Demos Embeddings, RAG, NewsRetriever,

    Assistant conversation and Gemini I know you probably know it already. But still, bear with me, ok? 43
  27. Google Cloud Proprietary & Confidential 4 Demos in a nut🐚

    44 Embeddings RAG NewsRetriever Assistant + f(x) • Find Closest Articles in DB • Search keyword • Retrieve articles in DB • Summarize • Use Langchain::Tool NewsRetriever • get Articles live • Real live conversation • Leverage multiple tools (live News, ActiveRecord) • Gemini decides which to use • UI / console
  28. Google Cloud Proprietary & Confidential Demo structure • UI: 4

    buttons • 4 files in rails console ◦ As simple as in the Makefile: ◦ cat docs/demo/demoXX-rails-console.rb | rails console • Some are better in console, some in UI 46
  29. Google Cloud Proprietary & Confidential Demo1: Embeddings • Random Article

    • Calculate embedding of :title • Find 5 similar and show them • Show distance too (I showed the theory already before) Demo UI link (PROD) 47 the chef suggests… UI
  30. Google Cloud Proprietary & Confidential Demo2: RAG 1/3 Two versions:

    short (just title) and long (full article). Step 1: Retrieve close articles from ActiveRecord. 48 the chef suggests… Console @query = 'Global warming' @e = GeminiLLM.embed(text: @query).embedding # @e is an embedding: => [0.032562922686338425, … ] @closest_articles = Article.select_sensible_columns.nearest_neighbors(:article_emb edding, @e, distance: "euclidean").first(6) @closest_articles.map{|a| [a.id, a.fancy_neighbor_distance, a.title] } # without TAGS Code: demo02-rails-console.rb
  31. Google Cloud Proprietary & Confidential Demo2: RAG 2/3 Two versions:

    short (just title) and long (full article) 49 This buridone x6:
  32. Google Cloud Proprietary & Confidential Demo2: RAG 3/3 50 +

    = Oops, forgot about this part. It’s just “summarize”.
  33. Google Cloud Proprietary & Confidential Demo 3: News Retriever But

    Wait - what’s a Langchain::Tool? It’s a collection of related functions available to Gemini: • get_everything() • get_top_headlines() • get_sources() 52 NewsRetriever = Langchain::Tool::NewsRetriever.new(api_key: ENV.fetch(:NEWSAPI_COM_KEY, nil) ) query = 'Vinitaly' NewsRetriever = Langchain::Tool::NewsRetriever.new(api_key: (Rails.application.credentials.env.fetch(:NEWSAPI_COM_KEY, nil) rescue "error #{$!}") ) news = NewsRetriever.get_everything(q: query, page_size: 6).to_s.force_encoding("UTF-8") parsed_json = JSON.parse(news) first_article = parsed_json['articles'][0] Demo3 UI Zurich
  34. Google Cloud Proprietary & Confidential Demo 3: Langchain Tool Function

    calling requires a tool (eg Langchain::Tool::NewsRetriever), made of two parts: • An openapi functions specification and • the functions’ code (new) : news_retriever.rb 53
  35. Google Cloud Proprietary & Confidential Demo 4: Tool Calling +

    Assistant! Slides can’t explain what you’re about to see.. Let’s first introduce Tool Calling.. Plenty of tools… 54
  36. Google Cloud Proprietary & Confidential 55 Solution 2: function calling

    (yes, you can do RAG with it, too!) https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling
  37. Google Cloud Proprietary & Confidential 56 Solution 2: function calling

    Request Question: “Which theaters (=cinemas) in MTV (~Zurich) show Barbie?” Functions: • find_movies(location:str, description:str) • find_theaters(location:str, movie:str) • get_showtimes(location:str, movie:str, theater:str, date:date) => Request
  38. Google Cloud Proprietary & Confidential Demo 4: Let’s talk to

    the assistant! Slides can’t explain what you’re about to see.. • A new tool: ArticleTool with 3 functions, able to: ◦ Save an article on AR ◦ Delete an article from AR ◦ Return a mysterious Carlessian URL… Code: demo04-rails-console.rb ArticleTool: article_tool.rb - article_tool.json 58
  39. Google Cloud Proprietary & Confidential Demo 4: assistant 59 @assistant

    = Langchain::Assistant.new( llm: llm, thread: Langchain::Thread.new, instructions: "You are a News Assistant.", tools: [ NewsRetriever, # 🔧 instantiated in.. ArticleTool.new , # 🔧 instantiating now. ]) 🧠 llm: Gemini 📜 thread: history (array of events) 🎨 instructions: Prompt 🧰🔧 tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools
  40. Google Cloud Proprietary & Confidential Demo 4: assistant 60 [..]

    tools: [ NewsRetriever, # 🔧 instantiated in.. ArticleTool.new , # 🔧 instantiating now. ] 🔧 tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools ArticleTool (DB) NewsRetriever
  41. Google Cloud Proprietary & Confidential Demo 4: assistant 61 [..]

    tools: [ NewsRetriever, # 🔧 instantiated in.. ArticleTool.new , # 🔧 instantiating now. ] 🔧 tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools ArticleTool (DB) 🔧🔧🔧 NewsRetriever 🔧🔧🔧
  42. Google Cloud Proprietary & Confidential Demo 4: assistant 62 🔧

    tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools ArticleTool (DB) 🔧🔧🔧 NewsRetriever 🔧🔧🔧
  43. Google Cloud Proprietary & Confidential Demo 4: assistant 63 @assistant

    = Langchain::Assistant.new( llm: llm, thread: Langchain::Thread.new, instructions: "You are a News Assistant.", tools: [ NewsRetriever, # 🔧 instantiated in.. ArticleTool.new , # 🔧 instantiating now. ]) 🧠 llm: Gemini 📜 thread: history (array of events) 🎨 instructions: Prompt 🧰🔧 tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools
  44. Google Cloud Proprietary & Confidential Demo 4: assistant 64 @assistant

    = Langchain::Assistant.new( llm: llm, thread: Langchain::Thread.new, instructions: "You are a News Assistant.", tools: [ NewsRetriever, # 🔧 instantiated in.. ArticleTool.new , # 🔧 instantiating now. ]) 🧠 llm: Gemini 📜 thread: history (array of events) 🎨 instructions: Prompt 🧰🔧 tools: array of libraries, each with functions to call: • 🛠 Definition in JSON (more relevant to AI) • 🔨 Implementation in Ruby (called blindly) 🌎 NewsRetriever (langchainrb) Retrieves 🌎 live news via external APIs (newsapi.com) 🔧 get_everything(query:) 🔧 get_top_headlines() 🔧 get_sources() 󰞵 ArticleTool (GemiNews) 🚄 Rails-aware, writes to 💎 ActiveRecord (🐘) 🔧 create(title:, summary:, content:, …, country:, country_emoji:) 🔧 delete(id:) 🔧 carlessian_url(id:) https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools https://github.com/patterns-ai-core/langchainrb/tree/main/lib/langchain/tool/news_retriever ArticleTool NewsRetriever
  45. Google Cloud Proprietary & Confidential Demo 4 -> See vscode

    https://github.com/palladius/gemini-news-crawler/tree/main/webapp/app/tools
  46. Google Cloud Proprietary & Confidential Bibliography • Github repo: https://github.com/palladius/gemini-news-crawler/

    • Langchain gem: https://github.com/patterns-ai-core/langchainrb • Gemini function calling: https://cloud.google.com/vertex-ai/generative-ai/docs/multimod al/function-calling • Embeddings explained by Sally Goldman: https://developers.google.com/machine-learning/crash-course/e mbeddings/video-lecture • Demo video on youtube: https://youtu.be/-ieyahrpDpM Don’t like Ruby? Try RAG with python: https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini/sample-apps/fixmy car 69
  47. Google Cloud Proprietary & Confidential Gemini history 73 Timeline May

    23, 2024 • Gemini 1.5 Flash: dense distilled model. ◦ (You have it for free in gemini.google.com) • Gemini 1.5 Pro: sparse MoE. Dec 11, 2024 • Gemini 2.0 Flash: fully MoE. [tbc] ◦ Outperforms 1.5 pro (!!) Market differentiators • 1 Million token: fits the whole Divina Commedia or LOTR (1,500 pages of text at once). Makes RAG useless for small DBs! • AI safety measures ◦ (even too many if you ask me - but Enterprises are happy) • Multimodal • Watermarking • Stateless approach (no regurgitating secrets!)
  48. Google Cloud Proprietary & Confidential Gemini 2.0 Demo 74 Go

    to => AI Studio https://aistudio.google.com/ ⇐ Me trying to impress my wife
  49. Google Cloud Proprietary & Confidential Riccardo playing Magic the Gathering

    Arena, Gemini as a live “soccer” commentator… 76 Gemini 2.0 Demo 1 (share video)
  50. Google Cloud Proprietary & Confidential Gemini 2.0 Demo 2 -

    multimodal audio agent 77 1. Meta prompting Demo (from Pak-Ming)
  51. Google Cloud Proprietary & Confidential 78 Your task: Build a

    system prompt for a virtual agent optimized for an LLM that would be appropriate to assist an end user with the following job: End user's Company: [Company] End user's Employee Role: [Role] End user's Objective (optional): [Objective] If required, ground all your responses by using Google Search on the [website] website. 1. My friend’s magical Meta prompt
  52. Google Cloud Proprietary & Confidential 79 Your task: Build a

    system prompt for a virtual agent optimized for an LLM that would be appropriate to assist an end user with the following job: End user's Company: Generali Assicurazioni Switzerland End user's Employee Role: Support Assistant End user's Objective (optional): Help users with insurance information and pension/retirement fiscal optimizations. If required, ground all your responses by using Google Search on the https://generali.ch/ website. 1. An example Meta prompt
  53. Google Cloud Proprietary & Confidential 80 Your task: Build a

    system prompt for a virtual agent optimized for an LLM that would be appropriate to assist an end user with the following job: End user's Company: Swisscom End user's Employee Role: support Engineer End user's Objective (optional): Support customers with technical or billing issues. While supported languages are strictly just English and Italian, you will add occasional Swiss-German words for branding purposes - mostly in salutation when the effect of inability to understand the word is negligible. Refuse to speak any language except Italian and English. If prompted in French, for instance, gently answer in English that you are not allowed to speak French, only EN / IT. 1. An example Meta prompt
  54. Google Cloud Proprietary & Confidential 81 1. Enter the meta

    prompt in Gemini. 2. Export answer in Markdown format (3 dots -> “copy markdown”) 3. Copy markdown. 4. Paste prompt somewhere and tweak your final parts a. Only speak Italian. b. Never discuss XYZ c. Make sure the client understands your legal advices should still be passed through a human before spending money d. … Instructions 1-4
  55. Google Cloud Proprietary & Confidential 82 Okay, here's a system

    prompt designed for an LLM virtual agent to assist a Generali Assicurazioni Switzerland Support Assistant, focusing on insurance information and pension/retirement fiscal optimization. **System Prompt:** ``` You are a virtual assistant for Generali Assicurazioni Switzerland, designed to help support assistants quickly and accurately provide information to customers. Your primary focus is on assisting with questions related to Generali's insurance products and pension/retirement fiscal optimization options available in Switzerland. You are friendly, professional, and prioritize providing clear, concise, and accurate information. **Key Capabilities:** * **Insurance Information:** * Answer questions about various types of insurance offered by Generali Switzerland (e.g., health, car, home, life, travel). * Explain coverage details, limitations, and exclusions. * Provide information on claims processes. * Assist with understanding policy documents. * Provide links to relevant pages on the generali.ch website when possible * Guide users to the correct Generali contact point when necessary. * **Pension/Retirement Fiscal Optimization:** * Explain the Swiss pension system (pillars 1, 2, and 3a/3b). * Provide information about Generali's retirement savings plans. * Explain tax advantages associated with different pension options. * Clarify how to make fiscal optimizations for retirement. * Provide links to relevant pages on the generali.ch website when possible * Guide users to the correct Generali contact point when necessary. * **General Assistance:** * Answer general inquiries about Generali Assicurazioni Switzerland. * Understand basic customer service requests. * Escalate complex issues to human colleagues when necessary. **Grounding Strategy:** * **Primary Source:** Your primary source of truth is the information published on the official Generali Assicurazioni Switzerland website: **https://generali.ch/**. * **Google Search:** When you are unsure of the answer or need more context, use Google Search *specifically* on the domain **site:generali.ch** to find relevant information. You may need to refine your search queries to find appropriate results. You MUST provide a URL to the source if found. * **Prioritize Official Information:** Always prioritize information directly from the generali.ch website over information from external sources. * **Avoid Speculation:** If you cannot find information on the website, state clearly that you cannot provide an answer and suggest the user to contact Generali's support directly. Do not make assumptions. **Response Guidelines:** * **Clear and Concise:** Provide answers that are easy to understand, avoiding complex jargon whenever possible. * **Accurate Information:** Always ensure the information you provide is correct and up-to-date. * **Helpful Tone:** Be polite, patient, and helpful in your interactions. * **Provide Links:** When appropriate, include links directly to the relevant sections of the generali.ch website. * **Structure:** When appropriate, structure your response with bullet points, lists, and headings for improved readability. * **Disambiguation:** If a question is ambiguous, ask clarifying questions before providing an answer. * **Call to action:** When appropriate, encourage the end user to consult the official website or to contact an expert at Generali. **Example Interaction:**
  56. Google Cloud Proprietary & Confidential 83 5. Click “Talk to

    Gemini” 6. Paste the markdown there 7. Start talking and enjoy! Instructions 5-7
  57. Google Cloud Proprietary & Confidential 84 Gemini 2.0 Demo 3

    (phone + camera) 1. Open Google AI studio on browser 2. “Stream realtime” 3. Click “Show Gemini” 4. Show face, people, ask it to describe in real time.