$30 off During Our Annual Pro Sale. View Details »

Yokozuna, Distributed Search You Don't Think About

Avatar for Ryan Zezeski Ryan Zezeski
May 14, 2013
Technology
4
830

Yokozuna, Distributed Search You Don't Think About

Discussion and demos of Yokozuna at RICON|East 2013.

Avatar for Ryan Zezeski

Ryan Zezeski

May 14, 2013
Tweet

More Decks by Ryan Zezeski

See All by Ryan Zezeski
Yokozuna: Scaling Solr With Riak
Avatar for Ryan Zezeski rzezeski
4
1.7k

Other Decks in Technology

See All in Technology
AWSに革命を起こすかもしれない新サービス・アップデートについてのお話
Avatar for Yuuki Yamashita yama3133
0
510
半年で、AIゼロ知識から AI中心開発組織の変革担当に至るまで
Avatar for ryu rfdnxbro
0
140
「もしもデータ基盤開発で『強くてニューゲーム』ができたなら今の僕はどんなデータ基盤を作っただろう」
Avatar for AEON aeonpeople
0
250
Amazon Bedrock Knowledge Bases × メタデータ活用で実現する検証可能な RAG 設計
Avatar for Tomoaki tomoaki25
6
2.4k
特別捜査官等研修会
Avatar for Nomizo Nomizo nomizone
0
580
Introduce marp-ai-slide-generator
Avatar for Itaru Tomita itarutomy
0
130
投資戦略を量産せよ 2 - マケデコセミナー(2025/12/26)
Avatar for tomo gamella
0
440
株式会社ビザスク_AI__Engineering_Summit_Tokyo_2025_登壇資料.pdf
Avatar for VISASQ eikohashiba
1
120
ハッカソンから社内プロダクトへ AIエージェント ko☆shi 開発で学んだ4つの重要要素
Avatar for Tech Leverages leveragestech
0
210
さくらのクラウド開発ふりかえり2025
Avatar for kazeburo kazeburo
2
1.2k
日本の AI 開発と世界の潮流 / GenAI Development in Japan
Avatar for Yoshitaka Haribara hariby
1
480
20251222_サンフランシスコサバイバル術
Avatar for ponponmikan ponponmikankan
2
140

Featured

See All Featured
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
2.8k
Between Models and Reality
Avatar for mayunak mayunak
0
150
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
38
3k
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
0
170
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.1k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
Avatar for Eileen M. Uchitelle eileencodes
26
3.3k
WCS-LA-2024
Avatar for Leonardo Collado-Torres lcolladotor
0
390
Are puppies a ranking factor?
Avatar for Jono Alderson jonoalderson
0
2.4k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.4k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
274
41k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
38
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
31
9.8k

Transcript

  1. Yokozuna Distributed Search You Don’t Think About Ryan Zezeski May

    14th 2013 1 Tuesday, May 21, 13

  2. Live Demo 2 Tuesday, May 21, 13

  3. Live Demos 3 Tuesday, May 21, 13

  4. PROBLEM? 4 Tuesday, May 21, 13

  5. PROBLEM? SOLUTION! 5 Tuesday, May 21, 13

  6. Solution Pre-made 6 Tuesday, May 21, 13

  7. Piece At A Time 7 Tuesday, May 21, 13

  8. Goals • Don’t screw up • Show how Yokozuna doesn’t

    make you think (too hard) • Teach you about Search • Neat things you can do with Yokozuna 8 Tuesday, May 21, 13

  9. PROBLEM: SEARCH FOR COMMITS ABOUT SPECIFIC FEATURE/BUG MAKE IT GOOGLE-LIKE

    9 Tuesday, May 21, 13

  10. SOLUTION: INDEX COMMITS IN YOKOZUNA - “COMMIT LOG SEARCHER” (CLS)

    10 Tuesday, May 21, 13

  11. Anatomy of a Commit Msg 11 Tuesday, May 21, 13

  12. Primary Key 12 Tuesday, May 21, 13

  13. Any Node Will Do 13 Tuesday, May 21, 13

  14. Term Query 14 Tuesday, May 21, 13

  15. Query Any Node 15 Tuesday, May 21, 13

  16. Boolean (1) repo:riak_kv repo:riak_core 16 Tuesday, May 21, 13

  17. Boolean (2) repo:riak_kv AND author:”Ryan Zezeski” 17 Tuesday, May 21,

    13

  18. Boolean (3) commit_author:"Ryan Zezeski" OR commit_author:"Joseph Blomstedt" NOT commit_repo:riak_kv 18

    Tuesday, May 21, 13

  19. Range (1) commit_repo:riak_core AND commit_dt:[NOW-1YEAR TO NOW] 19 Tuesday, May

    21, 13

  20. Range (2) commit_repo:riak_core AND commit_dt:[NOW-1YEAR TO NOW] I RAN THIS

    ON 2013-05-10 sort=dt asc 20 Tuesday, May 21, 13

  21. Wildcard (1) *:* GET TOTAL COUNT FIRST 21 Tuesday, May

    21, 13

  22. Wildcard (2) commit_repo:riak_* NOTICE COUNT IS LOWER 22 Tuesday, May

    21, 13

  23. WHAT ABOUT SEARCHING SUMMARY AND BODY? 23 Tuesday, May 21,

    13

  24. THE INVERTED INDEX 24 Tuesday, May 21, 13

  25. AN INDEX - BUT INVERTED 25 Tuesday, May 21, 13

  26. EVERYONE KNOWS WHAT IT IS 26 Tuesday, May 21, 13

  27. EVEN NON-TECH PEOPLE 27 Tuesday, May 21, 13

  28. YES...EVEN YOUR PARENTS 28 Tuesday, May 21, 13

  29. What’s In A Book? 29 Tuesday, May 21, 13

  30. • WORDS • PARAGRAPHS • SECTIONS • CHAPTERS • ETC.

    30 Tuesday, May 21, 13

  31. AND PAGE NUMBERS 31 Tuesday, May 21, 13

  32. 32 Tuesday, May 21, 13

  33. PAGE NUMBERS ARE AN IMPLICIT INDEX 33 Tuesday, May 21,

    13

  34. PAGE NUMBER TO WORDS WORD TO PAGE NUMBERS INVERTED 34

    Tuesday, May 21, 13

  35. STOLEN FROM BLOG OF RICKY HO: http:// horicky.blogspot.com/2013/02/text-processing- part-2-inverted-index.html 35

    Tuesday, May 21, 13

  36. HOW DO YOU GET THE WORDS IN THE FIRST PLACE?

    36 Tuesday, May 21, 13

  37. Analysis - The Iceberg That Sunk The Titanic 37 Tuesday,

    May 21, 13

  38. Phrase (1) subject:hinted OR subject:handoff OR body:hinted OR body:handoff 38

    Tuesday, May 21, 13

  39. Phrase (2) subject:”hinted handoff” OR body:”hinted handoff” 39 Tuesday, May

    21, 13

  40. Phrase (3) subject:”partition vnode” OR body:”partition vnode” 40 Tuesday, May

    21, 13

  41. Phrase (4) subject:”partition vnode”~4 OR body:”partition vnode”~4 41 Tuesday, May

    21, 13

  42. Exact Term subject:behavior OR body:behavior 42 Tuesday, May 21, 13

  43. Fuzzy Term subject:behavior~1 OR body:behavior~1 43 Tuesday, May 21, 13

  44. Ranking ADD SCORE TO FL SCORE ADDED TO EVERY RESULT

    44 Tuesday, May 21, 13

  45. RECALL, PRECISION, AND RELEVANCY, OH MY! 45 Tuesday, May 21,

    13

  46. RELEVANCY - FOR A GIVEN QUERY & DOC SET THERE

    IS AN IDEAL ANSWER OF ONLY RELEVANT DOCS 46 Tuesday, May 21, 13

  47. RECALL = WHAT % OF IDEAL ANSWER SET WAS RETRIEVED

    47 Tuesday, May 21, 13

  48. PRECISION = WHAT % OF ANSWER IS RELEVANT 48 Tuesday,

    May 21, 13

  49. RECALLvs.PRECISION AS YOU INCREASE RECALL YOU DEGRADE PRECISION 49 Tuesday,

    May 21, 13

  50. SOLR DETERMINES RELEVANCY VIA THE NOTION OF SIMILARITY 50 Tuesday,

    May 21, 13

  51. SOLR USES TF-IDF: TERM FREQUENCY, INVERSE DOCUMENT FREQUENCY 51 Tuesday,

    May 21, 13

  52. Dismax + Facets + Highlighting FACETS HIGHLIGHTING DISMAX 52 Tuesday,

    May 21, 13

  53. FACET - A TAXONOMY OF YOUR QUERY BASED ON FIELD’S

    VALUES 53 Tuesday, May 21, 13

  54. FACETS ALLOW “DRILL DOWN” - THEY GUIDE THE USER 54

    Tuesday, May 21, 13

  55. HIGHLIGHTING GIVES YOUR RESULTS CONTEXT - ALLOWS QUICKER DETERMINATION OF

    RELEVANCY 55 Tuesday, May 21, 13

  56. DISMAX - DISjunction MAX - A QUERY HANDLER MEANT FOR

    DIRECT USER INPUT 56 Tuesday, May 21, 13

  57. All Nodes Up 57 Tuesday, May 21, 13

  58. All Nodes Up - Query 58 Tuesday, May 21, 13

  59. Node 4 Down 59 Tuesday, May 21, 13

  60. Node 4 Down - Query 60 Tuesday, May 21, 13

  61. Node 3 & 4 Down 61 Tuesday, May 21, 13

  62. Node 3 & 4 Down - Query 62 Tuesday, May

    21, 13

  63. REPLICATION PROVIDES HIGH AVAILABILITY 2 3 4 1 START WITH

    4 NODES 63 Tuesday, May 21, 13

  64. Write 3 Replicas 2 3 4 1 64 Tuesday, May

    21, 13

  65. Take 2 Nodes Down 2 3 4 1 1 REPLICA

    STILL AVAILABLE 65 Tuesday, May 21, 13

  66. WHAT IF DATA IS WRITTEN WHILE NODES ARE DOWN? 66

    Tuesday, May 21, 13

  67. YZ Not Stored Yet 67 Tuesday, May 21, 13

  68. Store YZ Log 68 Tuesday, May 21, 13

  69. Query YZ - Node 1& 2 69 Tuesday, May 21,

    13

  70. Set XFer Limit To 0 70 Tuesday, May 21, 13

  71. Start Nodes 3 & 4 71 Tuesday, May 21, 13

  72. Query Solr Direct WHEN MAKING THIS DEMO I WAS EXPECTING

    THIS TO BE 0 BUT I FORGOT ABOUT AAE WHICH STARTED KICKING IN BEFORE HANDOFF - SELF HEALING FTW! 72 Tuesday, May 21, 13

  73. Set Xfer Limit To 64 73 Tuesday, May 21, 13

  74. Handoff Occurs 74 Tuesday, May 21, 13

  75. 0 Pending Xfers 75 Tuesday, May 21, 13

  76. Solr Direct (Again) NOTICE IT’S NOW 301, UP FROM 54,

    MORE PROOF THAT HANDOFF OCCURRED - NOTE THIS QUERY IS GOING DIRECT TO ONLY 1 SHARD 76 Tuesday, May 21, 13

  77. Query Node 4 YZ NOW HIT YOKOZUNA ON NODE4 (NOTICE

    CHANGE IN PORT #) - THIS WILL RUN A DIST SEARCH AND THUS RETURN CORRECT COUNT 77 Tuesday, May 21, 13

  78. Data Ownership A VNODE THE RING 78 Tuesday, May 21,

    13

  79. Node Down X X X X X X X X

    X X 79 Tuesday, May 21, 13

  80. Write Fallback X X X X X X X X

    X X 80 Tuesday, May 21, 13

  81. Node Up HINTED HANDOFF WILL MOVE REPLICA TO PRIMARY 81

    Tuesday, May 21, 13

  82. WHAT IF YOU RM -RF THE INDEX DIR? 82 Tuesday,

    May 21, 13

  83. Kill The Data RM -RF THE INDEX DIRECTORY KILL THE

    SOLR PROC 83 Tuesday, May 21, 13

  84. Auto Restart YOKOZUNA NOTICES SOLR DIED AND AUTOMATICALLY RESTARTS IT

    84 Tuesday, May 21, 13

  85. Node 4 - 0 Results 85 Tuesday, May 21, 13

  86. AAE Notices Missing Data 86 Tuesday, May 21, 13

  87. Node 4 - 13 Results DATA IS RE-INDEXED OVER TIME

    87 Tuesday, May 21, 13

  88. More AAE Repair 88 Tuesday, May 21, 13

  89. Node 4 - 128 Results MORE INDEXES ARE REPAIRED, THIS

    CONTINUES UNTIL AAE REPAIRS ALL INDEXES 89 Tuesday, May 21, 13

  90. WHAT EVEN IS ACTIVE ANTI- ENTROPY? 90 Tuesday, May 21,

    13

  91. Mo Systems Mo Failure • index update could get lost

    • files can become truncated/corrupted • accidental `rm -rf` • segfault at right time • etc... 91 Tuesday, May 21, 13

  92. MYRAID OF FAILURE SCENARIOS - FROM OBVIOUS TO NEARLY INVISIBLE

    92 Tuesday, May 21, 13

  93. ENTROPY IS DAMAGE AAE IS SELF HEALING STRIKER!!!! EHEM, I

    MEAN, ENTROPY!!!! 93 Tuesday, May 21, 13

  94. REPAIR EFFICIENTLY - NOT STUPIDLY 94 Tuesday, May 21, 13

  95. Learn You Some Merkle For A Great Good BIG UPS

    TO @jtuple FOR THE AAE DIAGRAMS 95 Tuesday, May 21, 13

  96. Segments EACH SEGMENT IS LIST OF KEY-HASH PAIRS 96 Tuesday,

    May 21, 13

  97. Segment Hashes HASH OF HASHES IN SEGMENT 97 Tuesday, May

    21, 13

  98. Hash O’ Hashes 98 Tuesday, May 21, 13

  99. WHAT HAPPENS DURING EXCHANGE? 99 Tuesday, May 21, 13

  100. Start With 2 Trees 100 Tuesday, May 21, 13

  101. Compare Top Hashes TOP HASHES DON’T MATCH - SOMETHING IS

    DIFFERENT 101 Tuesday, May 21, 13

  102. Compare Child Hashes NARROW DOWN THE DIVERGENT SEGMENT 102 Tuesday,

    May 21, 13

  103. Recur NARROW DOWN THE DIVERGENT SEGMENT CONT... 103 Tuesday, May

    21, 13

  104. Iter Key-Hash Pairs ITER FINAL LIST OF HASHES TO FIND

    DIVERGENT KEYS 104 Tuesday, May 21, 13

  105. Repair Divergent Keys REPAIR (RE-INDEX) KEYS THAT ARE DIVERGENT (RED)

    105 Tuesday, May 21, 13

  106. CODE FOR DETECTION AND REPAIR - NOT PREVENTION 106 Tuesday,

    May 21, 13

  107. WHAT HAPPENS IF 3 NODES GO DOWN? 107 Tuesday, May

    21, 13

  108. Stop 3 Nodes 108 Tuesday, May 21, 13

  109. Query 109 Tuesday, May 21, 13

  110. CONSISTENCY vs. AVAILABILITY 110 Tuesday, May 21, 13

  111. Uptime - Story of 9s UPTIME = (MTBF - MTTR)

    / MTBF 111 Tuesday, May 21, 13

  112. Uptime is Flawed IF THE SYSTEM IS DOWN, BUT NO

    ONE MAKES A REQUEST, IS IT REALLY DOWN? 112 Tuesday, May 21, 13

  113. Yield - Uptime of the People YIELD = QUERIES COMPLETED

    / QUERIES OFFERED 113 Tuesday, May 21, 13

  114. Harvest vs. Yield HARVEST = DATA AVAIL / COMPLETE DATA

    IF FACE OF FAILURE YOU CAN’T HAVE BOTH FOR A SINGLE REQUEST 114 Tuesday, May 21, 13

  115. IN TIMES OF TROUBLE - YOKOZUNA CHOOSES HARVEST FOR QUERIES

    115 Tuesday, May 21, 13

  116. TECHNICALLY - YOKOZUNA IS ALWAYS < 100% HARVEST IN A

    NON- QUIESCENT CLUSTER 116 Tuesday, May 21, 13

  117. YOKOZUNA FAVORS YIELD FOR WRITES 117 Tuesday, May 21, 13

  118. ONCE RIAK 1.4 SHIPS - YOKOZUNA LANDS IN MASTER 118

    Tuesday, May 21, 13

  119. THANK YOU HTTP://GITHUB.COM/BASHO/YOKOZUNA 119 Tuesday, May 21, 13