Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Yokozuna, Distributed Search You Don't Think About

Yokozuna, Distributed Search You Don't Think About

Discussion and demos of Yokozuna at RICON|East 2013.

Ryan Zezeski

May 14, 2013
Tweet

More Decks by Ryan Zezeski

Other Decks in Technology

Transcript

  1. Goals • Don’t screw up • Show how Yokozuna doesn’t

    make you think (too hard) • Teach you about Search • Neat things you can do with Yokozuna 8 Tuesday, May 21, 13
  2. Range (2) commit_repo:riak_core AND commit_dt:[NOW-1YEAR TO NOW] I RAN THIS

    ON 2013-05-10 sort=dt asc 20 Tuesday, May 21, 13
  3. RELEVANCY - FOR A GIVEN QUERY & DOC SET THERE

    IS AN IDEAL ANSWER OF ONLY RELEVANT DOCS 46 Tuesday, May 21, 13
  4. DISMAX - DISjunction MAX - A QUERY HANDLER MEANT FOR

    DIRECT USER INPUT 56 Tuesday, May 21, 13
  5. Take 2 Nodes Down 2 3 4 1 1 REPLICA

    STILL AVAILABLE 65 Tuesday, May 21, 13
  6. Query Solr Direct WHEN MAKING THIS DEMO I WAS EXPECTING

    THIS TO BE 0 BUT I FORGOT ABOUT AAE WHICH STARTED KICKING IN BEFORE HANDOFF - SELF HEALING FTW! 72 Tuesday, May 21, 13
  7. Solr Direct (Again) NOTICE IT’S NOW 301, UP FROM 54,

    MORE PROOF THAT HANDOFF OCCURRED - NOTE THIS QUERY IS GOING DIRECT TO ONLY 1 SHARD 76 Tuesday, May 21, 13
  8. Query Node 4 YZ NOW HIT YOKOZUNA ON NODE4 (NOTICE

    CHANGE IN PORT #) - THIS WILL RUN A DIST SEARCH AND THUS RETURN CORRECT COUNT 77 Tuesday, May 21, 13
  9. Node Down X X X X X X X X

    X X 79 Tuesday, May 21, 13
  10. Write Fallback X X X X X X X X

    X X 80 Tuesday, May 21, 13
  11. Kill The Data RM -RF THE INDEX DIRECTORY KILL THE

    SOLR PROC 83 Tuesday, May 21, 13
  12. Node 4 - 128 Results MORE INDEXES ARE REPAIRED, THIS

    CONTINUES UNTIL AAE REPAIRS ALL INDEXES 89 Tuesday, May 21, 13
  13. Mo Systems Mo Failure • index update could get lost

    • files can become truncated/corrupted • accidental `rm -rf` • segfault at right time • etc... 91 Tuesday, May 21, 13
  14. ENTROPY IS DAMAGE AAE IS SELF HEALING STRIKER!!!! EHEM, I

    MEAN, ENTROPY!!!! 93 Tuesday, May 21, 13
  15. Learn You Some Merkle For A Great Good BIG UPS

    TO @jtuple FOR THE AAE DIAGRAMS 95 Tuesday, May 21, 13
  16. Iter Key-Hash Pairs ITER FINAL LIST OF HASHES TO FIND

    DIVERGENT KEYS 104 Tuesday, May 21, 13
  17. Uptime - Story of 9s UPTIME = (MTBF - MTTR)

    / MTBF 111 Tuesday, May 21, 13
  18. Uptime is Flawed IF THE SYSTEM IS DOWN, BUT NO

    ONE MAKES A REQUEST, IS IT REALLY DOWN? 112 Tuesday, May 21, 13
  19. Yield - Uptime of the People YIELD = QUERIES COMPLETED

    / QUERIES OFFERED 113 Tuesday, May 21, 13
  20. Harvest vs. Yield HARVEST = DATA AVAIL / COMPLETE DATA

    IF FACE OF FAILURE YOU CAN’T HAVE BOTH FOR A SINGLE REQUEST 114 Tuesday, May 21, 13
  21. TECHNICALLY - YOKOZUNA IS ALWAYS < 100% HARVEST IN A

    NON- QUIESCENT CLUSTER 116 Tuesday, May 21, 13