Everything Old is New Again: Why Information Retrieval Still Powers AI Search

As generative AI reshapes how users discover information, many assume traditional search principles are obsolete. This talk argues the opposite. From ranking and relevance to retrieval pipelines and evaluation, modern AI Search systems are built on decades of information retrieval research. Attendees will learn how foundational IR concepts quietly underpin LLM-powered search experiences—and why understanding them is essential for building, optimising, and auditing AI-driven discovery systems.

Transcript

2. Who is Dawn Anderson?  SEO practitioner for almost 20 years  International SEO conference speaker since 2017  Boutique agency owner (Bertey)  SEO consultant  Information retrieval & AI search world interloper  Commissioned to write a book on AI SEO currently
3. Pomeranian lover
4. Everything old is new again Why information retrieval still powers AI search
5. The AI Search Paradigm The paradigm shift is huge with a steep learning curve
6. AI Search is Standing on Strong Shoulders
7. The Generative AI Explosion
8. The Myth of the 'All- Knowing' AI
9. The Achilles Heel: Hallucinations
10. The Knowledge Cutoff Problem
11. Bridging the Gap with Grounding
12. Enter RAG (Retrieval- Augmented Generation)
13. The 'R' in RAG Does the Heavy Lifting The generation is only as good as the context you provide it (retrieval)
14. What is Information Retrieval (IR)? • Computer science field behind search as we know it • Many nuanced specialisms within the field • Close relatives, offspring and siblings in: o Natural language processing o Recommender systems o AI search / Generative information retrieval o Knowledge graphs and structured data specialisms
15. A Brief History of IR
16. The Architecture of AI Search
17. The Modern AI Search Pipeline
18. Data Ingestion
19. The Art of Chunking LLMs have context windows with limitations
20. Chunking Strategies • Fixed-size vs. semantic chunking • There are many different types of chunking • Fixed size (word, sentence, paragraph) is very rigid • Semantic chunking takes into consideration meaning / context
21. Metadata: The Unsung Hero • The importance of tagging chunks with metadata (date, author, source) • Allows for search filtering later • Classic IR approaches adapted for AI search
22. Indexing & Storage The concept of search indexes
23. The Inverted Index (Lexical Search) How traditional keyword search works.
24. Enter Vector Databases Introducing vector storage for semantic search.
25. Embeddings: Text to Numbers How embedding models translate text
26. Lexical vs. Semantic Storage • Comparing traditional indexes with vector indexes
27. Query Processing CLASSIC SEARCH – STEMMING, SYNONYMS AI SEARCH – QUERY REWRITING, QUERY EXPANSION (FAN OUT)
28. Handling Conversational Ambiguity • Query rewriting to resolve pronoun and conversational context prior to searching
29. Query Expansion – Adding Context to a Query to Improve Recall
30. Probably 'Query Fan Out'
31. Intent Detection
32. Self-Querying Retrieval – Advanced RAG Technique • Reduces hallucination using meta-data filtering
33. Retrieval & Ranking (The Core IR Concepts) The Goal: Relevance
34. Precision vs. Recall - The classic IR trade-off • Precision: Out of all the documents we retrieved, how many were actually useful? • Recall: Out of all the useful documents in the database, how many did we manage to find?
35. BM25: The Algorithm That Won't Die
36. Semantic Similarity (K-Nearest Neighbors) • How vector search ranks results • Distance between query vector and document vectors • Query as centroid
37. The Flaws of Vector Search - Why vector search isn't a silver bullet • Cat chasing dog • Dog chasing cat • In vector search these are mostly the same but... obviously they are NOT • Vector search struggles with exact phrasing, part numbers, and negations
38. The Winning Combo: Hybrid Search Combining BM25 and Vector Search
39. Reciprocal Rank Fusion (RRF) • Hybrid merging different search scores
40. The Two-Stage Pipeline: Retrieve & Re- Rank The industry standard for high- accuracy AI search
41. Cross-Encoders vs. Bi-Encoders • Vector search uses bi-encoders • Re-ranking uses cross-encoders
42. Learning to Rank (LTR) Using machine learning to optimise ranking Training a model to weigh different signals What matters most in different contexts for different queries? Tune from learnings
43. Context Injection in LLM Models – Advanced RAG Technique Source: https://apxml.com/courses/getting-started-rag/chapter-4-rag-generation-augmentation/context- injection-methods
44. The "Lost in the Middle" Phenomenon A quirk of how LLMs read context
45. Citation and Provenance Providing sources for AI answers by mapping LLM output to retrieved chunks
46. Streaming Answers Improving perceived latency
47. Fallback Strategies What to do when no relevant data is found Good LLM systems should return "I don't know" Often they just guess the most probabilistic answer LLM must be instructed to answer "I don't know"
48. Evaluation (How Do We Know It Works?) Evaluation: The Hardest Part of AI Search
49. Classic IR Metrics MRR (Mean Reciprocal Rank) 1 NDCG (Normalised Discounted Cumulative Gain) 2 MAP (Mean Average Precision) 3
50. RAG-Specific Evaluation Frameworks Introducing RAGAS / TruLens
51. RAG- Evaluation Was good context retrieved? Did the LLM stick to the context? Did it actually answer the user's question?
52. LLM-as-a-Judge Using powerful models to grade cheaper models
53. The Importance of Golden Datasets Creating a baseline for testing
54. The Future – The Feedback Loop Using user behaviour to improve search
55. The Future - Beyond RAG: Agentic Search The shift from single queries to multi-step reasoning
56. The Future - Multi-Modal Retrieval Searching across images, video, and audio
57. Respect the Foundations Generative AI may be the shiny user interface but IR is the reliable pipeline.
58. Thank you • X – dawnieando • LinkedIn – MsDawnAnderson • Threads – dawnieando • Bluesky – dawnieando • Bertey.com