Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build NLP Apps with Python, Even If You're a To...

Build NLP Apps with Python, Even If You're a Total Newbie!

Bilge Yücel

November 19, 2023
Tweet

More Decks by Bilge Yücel

Other Decks in Technology

Transcript

  1. • 🥑 Developer Advocate at deepset • 🏗 Open source

    LLM Framework: Haystack • 🍕 First time at Hamburg Python Pizza • 📍 Istanbul, Turkey • 💃 Latin music Twitter: @bilgeycl Linkedin: Bilge Yucel GitHub: @bilgeyucel Bilge Yücel Developer Advocate 🥑 deepset
  2. Text Embeddings - Vectors • Manageable by computers • Different

    techniques: ⚬ Sparse: TF-IDF, BM25... ⚬ Dense: Trained models (Sentence Transformers, Cohere, OpenAI...) • Often has 768 dimensions
  3. Vector Databases • Databases that store high-dimensional vectors • Optimized

    for vectors: ⚬ Vector search ⚬ CRUD operations ⚬ Metadata filtering
  4. Retrieval • Getting the most relevant information to the query

    • Used for semantic search, question answering and more Query
  5. Large Language Models (LLMs) • Big language models • Human-like

    output • Text generation: summarization, generative QA, writing code, chat…
  6. @bilgeycl @Haystack_AI Haystack What is Haystack? • Fully open-source framework

    built in Python for custom LLM applications • Provides tools that developers need to build state-of-the-art NLP systems • Building blocks: Pipelines & Components