WeAreDevelopers World Congress 2025: Semantic AI - Why Embeddings Might Matter More Than LLMs

Large Language Models (LLMs) get all the attention, but Embedding Models might be the real unsung heroes of AI systems. Christian shows how embedding-driven semantics can guard against hallucinations, dynamically route tasks, and supercharge Retrieval-Augmented Generation (RAG) workflows. Combined with Small or Large Language Models, embeddings offer a scalable way to build AI systems that aim to be more accurate, efficient, and context-aware. If you're ready to move beyond LLM hype, discover a semantic-centric approach in a pragmatic way.

Transcript

1. Semantic AI Why Embeddings Might Matter More Than LLMs Christian Weyer | Co-Founder & CTO | Thinktecture AG | christian.weyer@thinktecture.com
2. Semantic AI Why Embeddings Might Matter More Than LLMs AI MODELS FOR OUR SOFTWARE 2
3. Semantic AI Why Embeddings Might Matter More Than LLMs Classical applications & UIs 3 API-based data Document-based data
4. Semantic AI Why Embeddings Might Matter More Than LLMs Language-enabled “UIs” – Talk-to-TT 4
5. Semantic AI Why Embeddings Might Matter More Than LLMs LLM ALL-THE-THINGS? 5
6. Language Models understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. ⚠ Non-deterministic: same input can lead to different outputs. Embedding Models capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. ✅ Deterministic: same input always results in the same embedding. Semantic AI Why Embeddings Might Matter More Than LLMs 6 🫱 🫲 Semantic AI Generative AI
7. Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO LIGHTWEIGHT RAG [RETRIEVAL-AUGMENTED GENERATION] 7
8. Semantic AI Why Embeddings Might Matter More Than LLMs Talking to documents (Retrieval-augmented generation) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er w / sources LLM Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “What should I do…?” Vector DB 8
9. § Python Frameworks § LangChain § FastEmbed § Lightweight & efficient for generating text embeddings § Embedding model § jinaai/jina-embeddings-v2-base-de (local, no GPU req) – 768 dims § Vector store § PostgreSql (pgvector) vector store § LLM/SLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Lightweight RAG 9
10. Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO STRUCTURED OUTPUT 10
11. § Tools integration is being standardized with MCP Semantic AI Why Embeddings Might Matter More Than LLMs Talking to APIs (Function / Tool calling) 11 “When is CW available for a two-days workshop?” System Prompt (+ employee data) + Schema (for structured output) Web API Availability business logic
12. § Python Frameworks § Pydantic § Instructor § Methodology § Schema with JSON Mode (opt. Function Calling) § SLM / LLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Structured Output 12
13. Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO SEMANTIC GUARDING & ROUTING 13
14. Semantic AI Why Embeddings Might Matter More Than LLMs Semantics-based decisions for user interactions Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Target RAG Target API Call Target … something else … Fine-tuned Language Model Embedding Model 14
15. Guarding § Python Frameworks § llm-guard § HuggingFace Transformers § NLP / NLU model § deepset/ deberta-v3-base-injection (local, no GPU req.) Routing § Python Frameworks § semantic-routing § FastEmbed § Embedding model § intfloat/ multilingual-e5-large (local, no GPU req.) – 1024 dims § Vector store § PostgreSql (pgvector) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Semantic Guarding & Routing 15
16. Semantic AI Why Embeddings Might Matter More Than LLMs RECAP 16
17. Model type Core function Output type Embeddings role & relevance Embedding Models Encode meaning as vectors Deterministic vectors Embeddings are the product Language Models (SLMs / LLMs) Generate, transform, and understand language Human-readable text or structured data Embeddings are the foundation NLP / NLU Models Classify, decide, or detect patterns Labels, scores, decisions Embeddings are the backbone Semantic AI Why Embeddings Might Matter More Than LLMs Models, models, models – not without embeddings 17
18. Thank you! Christian Weyer christian.weyer@thinktecture.com