Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TW-Bert Good for Users, Good for SEOs (Brighton...

TW-Bert Good for Users, Good for SEOs (Brighton SEO April 2024)

Term Weighting BERT helps Google fix issues with showing the right results based on users intent which BERT alone didn't quite achieve. The step forward involves a clever but simple statistical model: find out why Google need it, how it works and how as SEOs we can begin to optimise for it.

davetheseo

April 27, 2024
Tweet

Other Decks in Marketing & SEO

Transcript

  1. TW-BERT: Good for Users, Good for SEOs Dave Cousin Oban

    International @davetheseo https://bit.ly/tw-bert https://www.linkedin.com/in/davecousin/
  2. obaninternational.com What we’re going to cover Why BERT needs improving?

    What’s Term Weighting? What is BERT? What’s TW-BERT? Where this fits in Google’s System? How to start optimising obaninternational.com
  3. obaninternational.com obaninternational.com Humans understand context intuitively Nike Running Shoes Nike

    Running Shoes Running Shoes obaninternational.com obaninternational.com
  4. obaninternational.com obaninternational.com What TW-BERT is basically •A model •Developed by

    Google Researchers •Explained in a paper called ‘End to End Query Term Weighting’* * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815
  5. obaninternational.com obaninternational.com What TW-BERT is basically •TW-BERT helps(/will help) Google

    think like a human • Deciding what results to show • What order to show them •Massively improves search intent understanding * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815 It understand the importance of words and sequences of words Whereas BERT currently deals with wordpieces Because…
  6. obaninternational.com obaninternational.com The problem for users •BERT was meant to

    understand intent better than ever •It helps Google… “grasp the subtle nuances of language that computers don’t quite understand the way humans do”
  7. obaninternational.com obaninternational.com The problem for users • Users have over

    the past couple of years been maybe more critical of Google’s results than ever:
  8. obaninternational.com obaninternational.com The problem for SEOs •We’ve all had SERPs

    where we used to be top and we see who took over and think WTF
  9. obaninternational.com obaninternational.com Real or imagined? • There’s even a 2024

    research paper saying Google’s results are worse: https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf • Ok it’s not due to BERT • But BERT hasn’t fixed it • There IS more Spam
  10. obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words

    are important? •Non Weighted: • Match MORE terms Where can I get great Korean BBQ in Brighton
  11. obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words

    are important? Weighted: • Match IMPORTANT terms Where can I get great Korean BBQ in Brighton
  12. obaninternational.com obaninternational.com Words, n-grams and wordpieces Wordpieces (19): I|want|##ed|a|new|##er|day|##ly|##s|##paper|from|the|##agent|to| ##day|get|##est|one|##morrow

    I wanted a newer daily newspaper from the newsagent today, I want to get the newest one tomorrow Unigrams (16): |I|wanted|a|newer|daily|newspaper|from|the|newsagent|today|want| to|get|newest|one|tomorrow| Bigrams (17): |I wanted|wanted a|a newer|newer daily|daily newspaper|newspaper from|from the|the newsagent|newsagent today|today, i|i want|want to|to get|get the|the newest|newest one|one tomorrow|
  13. obaninternational.com obaninternational.com What is BERT? •Deep Learning based Lexical Retriever

    for NLP Bidirectional Encoder Representations from Transformers
  14. obaninternational.com obaninternational.com What is BERT? •Based on a (relatively) huge

    corpus or training material Bidirectional Encoder Representations from Transformers
  15. obaninternational.com obaninternational.com What is BERT? •This allows the creation of

    a neural network (right) Bidirectional Encoder Representations from Transformers
  16. obaninternational.com obaninternational.com What is BERT? •Each wordpiece is mapped to

    others with direction & magnitude Bidirectional Encoder Representations from Transformers
  17. obaninternational.com obaninternational.com •Weights all wordpieces •Dynamic and considers: •Context •Meaning

    •Semantics •Sentiment What BERT does to search queries? •Expands queries: not just synonyms also semantic & contextual terms e.g. +‘restaurant’ to ‘Korean bbq’ bose noise cancel ##ing head with micro and blue tooth ##phone
  18. obaninternational.com 29 The basics of how search engines work Crawl

    (+Render) Index Ranking Algorithm Search / SERP Send Fetch Results Query
  19. obaninternational.com 30 IR and where BERT fits in Index Match

    Score Ranking Algorithm BERT Search Candidate Docs Fetch Results Query Weighted expanded 3 : 2 Scored Pairs
  20. obaninternational.com obaninternational.com TW-BERT’s bridging solution BERT’s Wordpiece Island TW-BERT Statistical

    Bridge Opened 2023 Fast Efficient Can Misunderstand Deeper Understanding of Language Slow Expensive Isle of Deep Learning
  21. obaninternational.com 35 And where TW-BERT fits Index Match Score Ranking

    Algorithm TW- BERT BERT Search Candidate Docs Fetch Results Query Weighted, expanded 3 : 2 Scored Pairs Feedback Loop
  22. obaninternational.com obaninternational.com The paper: End to End Query Term Weighting

    •Findings Vs BERT & Deep learning: Better overall precision Particularly good for top 10, top 100
  23. obaninternational.com obaninternational.com The paper: End to End Query Term Weighting

    •Findings Vs BERT & Deep learning: Performs very well for 1-10 word terms
  24. obaninternational.com obaninternational.com Is Google using TW-BERT? •Probably….. a bit •Why

    wouldn’t they? •It helps fix a major flaw •Some put July/August fluctuations down to TW- BERT: Source: Algoroo
  25. obaninternational.com obaninternational.com If Google are, then is this it? •Can

    tweak this and significance •More feedback loop data = more certainty and accuracy •Easy to copy for other search engines
  26. obaninternational.com obaninternational.com Are N-Grams the new keywords? •No KWs aren’t

    dead •Just a bit limp – another step in their diminishing importance
  27. obaninternational.com obaninternational.com Significance of n-grams in SEO •We have to

    optimise for intent first •KW stuffing will do nothing if the content doesn’t deliver
  28. obaninternational.com obaninternational.com Significance of improved query expansion •More competition from

    relevant content without the same KWs •Users will use more long tail & natural language
  29. obaninternational.com obaninternational.com Are you ready for TW-BERT? •Less competition from

    irrelevant results •But what if you have the irrelevant result? •Put yourself in the shoes of searchers?
  30. obaninternational.com obaninternational.com How to start optimising now: Match intent •Understand

    intent •What is ranking? •Who has gained? •What are they doing? •Look at SGE too for intent
  31. obaninternational.com obaninternational.com How to start optimising now: n-gram optimisation •Identify

    important n- grams in: • Related hero search terms • Questions or 5+ words 1. How to make a beef goulash? 2. How to cook a beef goulash? 3. Cook a beef goulash 4. Beef goulash recipe 5. Step by step beef goulash 6. Beef goulash how to
  32. obaninternational.com obaninternational.com How to stay optimised •Check every 3-6 months

    for key terms: • How’s language changing? • Is knowledge changing? • Consider intent in different countries • What is it? • Is it a scam? • How To Invest? • Focus on BitCoin • News • Prices • Latest scams 2012 2017 2024 ‘cryptocurrency’ intent over time
  33. obaninternational.com obaninternational.com So Good for Users, Google and SEOs? •

    Improved intent matching helps Users find what they want faster • And see less answers full of brand mentions Users • More likely to keep using Google as a search engine, • Brand can’t cheat their way to the top, some will buy ads instead Google • SEOs get more relevant, qualified traffic • Don’t have to go after irrelevant terms SEOs @davetheseo https://bit.ly/tw-bert linkedin.com/in/davecousin