How embeddings unlock the path to intent-driven optimization

Transcript

2. 2 DEPT®/SEO Brein | @frankvndijk Let’s go back to April… When I talked about vector embeddings at brightonSEO https://youtu.be/orJJYpRz1IA?si=mXb1-pbjPGcEzvDM
3. 3 DEPT®/SEO Brein | @frankvndijk My normal setup My vacation setup To enjoy my time off, I changed my setup
4. 4 DEPT®/SEO Brein | @frankvndijk New SF update! Screaming Frog just integrated vector embeddings in their crawls, making my scripts a bit useless…
5. 5 DEPT®/SEO Brein | @frankvndijk
6. 6 DEPT®/SEO Brein | @frankvndijk Source: Google Trends (query: vector embeddings)
7. 7 DEPT®/SEO Brein | @frankvndijk What are embeddings? “Embeddings are numerical representations of data (like words, images, or audio) in a multi-dimensional space” Images Audio Text Embedding model 0.9 0.7 0.2 0.6
8. 8 DEPT®/SEO Brein | @frankvndijk cat dog 0.9 0.7 -0.3 0.6 0.9 0.6 -0.2 0.8 Cosine similarity
9. 9 DEPT®/SEO Brein | @frankvndijk cat dog lion
10. 10 DEPT®/SEO Brein | @frankvndijk Content is now data Unstructured data is now transformed into something both computers and humans can understand Images Audio Text Embedding model 0.9 0.7 0.2 0.6
11. 11 DEPT®/SEO Brein | @frankvndijk Correct settings in SF Make sure that your crawl configurator is set properly: • Extraction => Store Rendered HTML • Rendering => JavaScript
12. 12 DEPT®/SEO Brein | @frankvndijk Connect with LLM Make the connection with your favorite LLM in your crawl: • Add your API from LLM • Choose the “Extract embeddings form page content” template
13. 13 DEPT®/SEO Brein | @frankvndijk Enable embeddings Enable and connect embeddings with the prompt: • Connect your prompt • Enable ‘semantic similarity’ & ‘low relevance’
14. 14 DEPT®/SEO Brein | @frankvndijk Run crawl analyses Before when can actually do stuff with the embeddings analyse your crawl: • Crawl analyses => Start
15. 15 DEPT®/SEO Brein | @frankvndijk Embedding information Now you can find new stuff inside of your crawl • Semantic similarity based on cosine similarity
16. 16 DEPT®/SEO Brein | @frankvndijk Perfect for finding internal linking opportunities! By lowering the threshold: 0.85 => Specific websites 0.7 => Broader websites Test this yourself!
17. 17 DEPT®/SEO Brein | @frankvndijk Visualize your website based on embeddings and find clusters SEO terms SEO events/activities SEO strategy SEO tooling
18. 18 DEPT®/SEO Brein | @frankvndijk https://www.screamingfrog.co.uk/seo-spider/tutorials/how-to-use-vector-embeddings-for-redirect-mapping/ Automate redirect mapping by crawling two domains. Make you sure you: • Use list mode without crawl depth • Filter your tab with advance table search
19. 19 DEPT®/SEO Brein | @frankvndijk My favorite: find what content is actually relevant for a certain query How it works: • Search for a query • Spot areas where your content doesn't answer search queries See how seobrein.nl is not really answering the question
20. 20 DEPT®/SEO Brein | @frankvndijk Start with SF If you want to begin working with vector embeddings, Screaming Frog is where you should start https://www.screamingfrog.co.uk/blog/seo-spider-22/ If you know, you know 👀
21. 21 DEPT®/SEO Brein | @frankvndijk Cosine similarity can help you with that Making sure your content matches the search intent is crucial…. Great content is useless if it doesn’t answer the user’s question
22. 22 DEPT®/SEO Brein | @frankvndijk Use case After optimizing cosine similarity for a client: • Before: 32.63% of AI Overviews contained a link • After: 54.48% of AI Overviews contained a link +67% +49% Increase in clicks Increase in display of links
23. 23 DEPT®/SEO Brein | @frankvndijk Source: https://searchengineland.com/google-ai-overview-links-top-12-organic-rankings-449216 It makes sense, right? because cosine similarity is a measure of relevance and semantic meaning, which is precisely what a search engine is designed to deliver
24. 24 DEPT®/SEO Brein | @frankvndijk First, we need to analyse what’s happening in the SERP, what is Google showing us? AI overview shown for the query “Wat is SEO?”
25. 25 DEPT®/SEO Brein | @frankvndijk We need to check what content is used to generate the AI overview +3 more sources
26. 26 DEPT®/SEO Brein | @frankvndijk Remember seobrein.nl? Why isn’t it showing up? 👀 0.46 0.37 0.41
27. 27 DEPT®/SEO Brein | @frankvndijk And what about the others, how similar are they to the query? +3 more sources 0.84 0.86 0.81
28. 28 DEPT®/SEO Brein | @frankvndijk Should we optimize for a cosine similarity of 1.0?
29. 29 DEPT®/SEO Brein | @frankvndijk Retrieval Augmented Generation Identify Retrieve information Generate AI Searching for information most similar to the query
30. 30 DEPT®/SEO Brein | @frankvndijk Don’t forget your SEO It’s the foundation of a successful GEO strategy. Without a strong SEO base, your GEO strategy will fail
31. 31 DEPT®/SEO Brein | @frankvndijk Success is no longer just about matching a query, it’s about deeply understanding the intent and the questions behind it, and answering them
32. 32 DEPT®/SEO Brein | @frankvndijk PHASE 1 PHASE 2 PHASE 3 PHASE 4 PHASE 5 Identifying sources Extracting content Generate embeddings Calculating (cosine) similarity Optimizing content Scraping competitors Scraping AI overviews Optimizing agent
33. 33 DEPT®/SEO Brein | @frankvndijk 1. Identifying sources Scraping Google AI overviews Data for SEO SerpAPI
34. 34 DEPT®/SEO Brein | @frankvndijk 2. Extracting content Scraping competitors JavaScript Python
35. 35 DEPT®/SEO Brein | @frankvndijk 2. Extracting content Scraping your own content JavaScript Python Chunk your content into pieces
36. 36 DEPT®/SEO Brein | @frankvndijk Google won’t use your full content to generate an AI overview We check each piece individually ??? ??? ??? ???
37. 37 DEPT®/SEO Brein | @frankvndijk We don’t just wanna check the search query, we wanna check the full intent Ahrefs Also Asked Gemini Topic Sub-topic Sub-topic Sub-topic Sub-topic Sub-topic Sub-topic
38. 38 DEPT®/SEO Brein | @frankvndijk We can take a look behind the scenes at the competition
39. 39 DEPT®/SEO Brein | @frankvndijk 3. Generate embeddings Using AI to generate embeddings OpenAI Gemini
40. 40 DEPT®/SEO Brein | @frankvndijk 4. Calculating similarity Check what content is close to query JavaScript Python
41. 41 DEPT®/SEO Brein | @frankvndijk 5. Optimizing content Check what content is close to query OpenAI Gemini
42. 42 DEPT®/SEO Brein | @frankvndijk Build it your own way Build workflows (N8N) Code it (Python/JavaScript) Use Google Sheets
43. 43 DEPT®/SEO Brein | @frankvndijk Google just released
44. 44 DEPT®/SEO Brein | @frankvndijk EmbeddingGemma “Translates text into meaningful numbers, based on the Gemini architecture” Open source
45. 45 DEPT®/SEO Brein | @frankvndijk SF is your best friend Screaming Frog is also the place to go for running these types of local models.
46. 46 DEPT®/SEO Brein | @frankvndijk Endless opportunities Vector embeddings can and should be used for a lot of different optimizations
47. 47 DEPT®/SEO Brein | @frankvndijk [0.9, 0.1, 0.2] [0.8, 0.2, 0.3] Now we can tell if the link is relevant
48. 48 DEPT®/SEO Brein | @frankvndijk Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Anchor relevance If there’s no high cosine similarity, the anchor is likely not relevant
49. 49 DEPT®/SEO Brein | @frankvndijk We zijn vanavond bijeen met SEO specialisten Amsterdam om over SEO te praten Mapping hreflang tags A difference in language doesn’t mean a difference in semantic value We are meeting with SEO specialists in Amsterdam tonight to talk about SEO. Vi mødes i aften med SEO-specialister fra Amsterdam for at tale om SEO. Šį vakarą susitinkame su Amsterdamo SEO specialistais, kad aptartume SEO klausimus. 😎 😎 🤯 🫠
50. 50 DEPT®/SEO Brein | @frankvndijk We’re not just improving SEO, we’re improving the user experience, exactly what Google wants to deliver.
51. 51 DEPT®/SEO Brein | @frankvndijk Your [SEO/GEO/AEO] strategy should still draw the map and set the destination, let vector embeddings be the compass that guides you throughout the journey