Share
Embed
TW-Bert Good for Users, Good for SEOs (Brighton SEO April 2024)
Term Weighting BERT helps Google fix issues with showing the right results based on users intent which BERT alone didn't quite achieve. The step forward involves a clever but simple statistical model: find out why Google need it, how it works and how as SEOs we can begin to optimise for it.
davetheseo
April 27, 2024
Transcript
48 slides
-
TW-BERT: Good for Users, Good for SEOs Dave Cousin Oban
International @davetheseo https://bit.ly/tw-bert https://www.linkedin.com/in/davecousin/ -
obaninternational.com What we’re going to cover Why BERT needs improving?
What’s Term Weighting? What is BERT? What’s TW-BERT? Where this fits in Google’s System? How to start optimising obaninternational.com -
obaninternational.com obaninternational.com Humans understand context intuitively Nike Running Shoes Nike
Running Shoes Running Shoes obaninternational.com obaninternational.com -
obaninternational.com obaninternational.com What TW-BERT is basically •A model •Developed by
Google Researchers •Explained in a paper called ‘End to End Query Term Weighting’* * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815 -
obaninternational.com obaninternational.com What TW-BERT is basically •TW-BERT helps(/will help) Google
think like a human • Deciding what results to show • What order to show them •Massively improves search intent understanding * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815 It understand the importance of words and sequences of words Whereas BERT currently deals with wordpieces Because… -
obaninternational.com obaninternational.com The problem for users •BERT was meant to
understand intent better than ever •It helps Google… “grasp the subtle nuances of language that computers don’t quite understand the way humans do” -
obaninternational.com obaninternational.com The problem for users • Users have over
the past couple of years been maybe more critical of Google’s results than ever: -
obaninternational.com obaninternational.com The problem for SEOs •We’ve all had SERPs
where we used to be top and we see who took over and think WTF -
obaninternational.com obaninternational.com Real or imagined? • There’s even a 2024
research paper saying Google’s results are worse: https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf • Ok it’s not due to BERT • But BERT hasn’t fixed it • There IS more Spam -
obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words
are important? •Non Weighted: • Match MORE terms Where can I get great Korean BBQ in Brighton -
obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words
are important? Weighted: • Match IMPORTANT terms Where can I get great Korean BBQ in Brighton -
obaninternational.com obaninternational.com Words, n-grams and wordpieces Wordpieces (19): I|want|##ed|a|new|##er|day|##ly|##s|##paper|from|the|##agent|to| ##day|get|##est|one|##morrow
I wanted a newer daily newspaper from the newsagent today, I want to get the newest one tomorrow Unigrams (16): |I|wanted|a|newer|daily|newspaper|from|the|newsagent|today|want| to|get|newest|one|tomorrow| Bigrams (17): |I wanted|wanted a|a newer|newer daily|daily newspaper|newspaper from|from the|the newsagent|newsagent today|today, i|i want|want to|to get|get the|the newest|newest one|one tomorrow| -
obaninternational.com obaninternational.com Importance of n-grams •Humans, understand other human’s without
consciously thinking about it 30,000 Wordpieces 10 million+ N-grams -
obaninternational.com obaninternational.com Importance of n-grams •We identify phrases or terms
we recognise 30,000 Wordpieces 10 million+ N-grams -
obaninternational.com obaninternational.com Importance of n-grams •Challenge to replicate in NLP
30,000 Wordpieces 10 million+ N-grams -
obaninternational.com obaninternational.com Importance of n-grams •Needs massive high dimensional neural
networks and training corpora 30,000 Wordpieces 10 million+ N-grams -
obaninternational.com obaninternational.com What is BERT? •Deep Learning based Lexical Retriever
for NLP Bidirectional Encoder Representations from Transformers -
obaninternational.com obaninternational.com What is BERT? •Based on a (relatively) huge
corpus or training material Bidirectional Encoder Representations from Transformers -
obaninternational.com obaninternational.com What is BERT? •This allows the creation of
a neural network (right) Bidirectional Encoder Representations from Transformers -
obaninternational.com obaninternational.com What is BERT? •Each wordpiece is mapped to
others with direction & magnitude Bidirectional Encoder Representations from Transformers -
obaninternational.com obaninternational.com •Weights all wordpieces •Dynamic and considers: •Context •Meaning
•Semantics •Sentiment What BERT does to search queries? •Expands queries: not just synonyms also semantic & contextual terms e.g. +‘restaurant’ to ‘Korean bbq’ bose noise cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com obaninternational.com What BERT does to search queries? bose noise
cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com obaninternational.com What BERT does to search queries? bose noise
cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com 29 The basics of how search engines work Crawl
(+Render) Index Ranking Algorithm Search / SERP Send Fetch Results Query -
obaninternational.com 30 IR and where BERT fits in Index Match
Score Ranking Algorithm BERT Search Candidate Docs Fetch Results Query Weighted expanded 3 : 2 Scored Pairs -
obaninternational.com obaninternational.com TW-BERT’s bridging solution BERT’s Wordpiece Island TW-BERT Statistical
Bridge Opened 2023 Fast Efficient Can Misunderstand Deeper Understanding of Language Slow Expensive Isle of Deep Learning -
obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise
cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise
cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise
cancel ##ing head with micro and blue tooth ##phone -
obaninternational.com 35 And where TW-BERT fits Index Match Score Ranking
Algorithm TW- BERT BERT Search Candidate Docs Fetch Results Query Weighted, expanded 3 : 2 Scored Pairs Feedback Loop -
obaninternational.com obaninternational.com The paper: End to End Query Term Weighting
•Findings Vs BERT & Deep learning: -
obaninternational.com obaninternational.com The paper: End to End Query Term Weighting
•Findings Vs BERT & Deep learning: Better overall precision Particularly good for top 10, top 100 -
obaninternational.com obaninternational.com The paper: End to End Query Term Weighting
•Findings Vs BERT & Deep learning: Performs very well for 1-10 word terms -
obaninternational.com obaninternational.com Is Google using TW-BERT? •Probably….. a bit •Why
wouldn’t they? •It helps fix a major flaw •Some put July/August fluctuations down to TW- BERT: Source: Algoroo -
obaninternational.com obaninternational.com If Google are, then is this it? •Can
tweak this and significance •More feedback loop data = more certainty and accuracy •Easy to copy for other search engines -
obaninternational.com obaninternational.com Are N-Grams the new keywords? •No KWs aren’t
dead •Just a bit limp – another step in their diminishing importance -
obaninternational.com obaninternational.com Significance of n-grams in SEO •We have to
optimise for intent first •KW stuffing will do nothing if the content doesn’t deliver -
obaninternational.com obaninternational.com Significance of improved query expansion •More competition from
relevant content without the same KWs •Users will use more long tail & natural language -
obaninternational.com obaninternational.com Are you ready for TW-BERT? •Less competition from
irrelevant results •But what if you have the irrelevant result? •Put yourself in the shoes of searchers? -
obaninternational.com obaninternational.com How to start optimising now: Match intent •Understand
intent •What is ranking? •Who has gained? •What are they doing? •Look at SGE too for intent -
obaninternational.com obaninternational.com How to start optimising now: n-gram optimisation •Identify
important n- grams in: • Related hero search terms • Questions or 5+ words 1. How to make a beef goulash? 2. How to cook a beef goulash? 3. Cook a beef goulash 4. Beef goulash recipe 5. Step by step beef goulash 6. Beef goulash how to -
obaninternational.com obaninternational.com How to stay optimised •Check every 3-6 months
for key terms: • How’s language changing? • Is knowledge changing? • Consider intent in different countries • What is it? • Is it a scam? • How To Invest? • Focus on BitCoin • News • Prices • Latest scams 2012 2017 2024 ‘cryptocurrency’ intent over time -
obaninternational.com obaninternational.com So Good for Users, Google and SEOs? •
Improved intent matching helps Users find what they want faster • And see less answers full of brand mentions Users • More likely to keep using Google as a search engine, • Brand can’t cheat their way to the top, some will buy ads instead Google • SEOs get more relevant, qualified traffic • Don’t have to go after irrelevant terms SEOs @davetheseo https://bit.ly/tw-bert linkedin.com/in/davecousin