Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Eventbrite’s Search-based Approach to Recommend...

Elastic Co
February 18, 2016

Eventbrite’s Search-based Approach to Recommendations

Eventbrite is building an Elasticsearch-powered, content- and behavior-based recommendation system to match users with events they are sure to enjoy. Come learn about their approach, successes, failures, and future work at the intersection of recommendation and search.

Elastic Co

February 18, 2016
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. Hi! I'm John Berryman Hi! I'm John Berryman Former: Aerospace

    Engineer Former: Search Tech Consultant @JnBrymn Current: Recommendation Engineer at Eventbrite Current: Facial Hair Artist Future: Author?
  2. Exciting Times at Exciting Times at We have been very

    successful focusing on the organizer! Hundreds of thousands of new events every month. Only now starting to focus on the consumer. Audacious Goal: Become the Marketplace for Upcoming Events A marketplace built on Search and Recommendations.
  3. Talking Points Talking Points Search - the user's interface for

    discovery Recommendation - how it all works Recommendation served from the Search Engine Recommendation eats Search The Brite Future
  4. Sometime get complaints from organizers that their events are not

    easy to find. We don't seem to get many complaints from consumers... No personalization, no understanding of the context of the user. Search is Good Search is Good ...but not perfect
  5. Collaborative Filtering Collaborative Filtering Based upon user interactions (event views/purchases)

    Answers "What might you like based upon what others with similar interactions like?" Methods: Matrix Decomposition Count Cooccurrence
  6. Matrix Decomposition Matrix Decomposition Benefits: Remove "noise" from data Introduces

    the idea of a genre Can compare event-to-event, user-to- user, and event-to-user Drawback: Computationally INTENSE! Why would we go to the trouble of creating a less accurate representation? Singular Value Decomposition Alternating Least Squares
  7. Collaborative Filtering Collaborative Filtering Output: recommended events related events recommended

    users "Exotic" Output: user-category affinity category-terminology affinity User123 → {Event345: 9.2, Event456: 7.2,... Event678 → {Event345: 8.8, Event456: 3.4,... Event789 → {User123: 7.3, User234: 6.1, ...} User123 → {Music: 6.4, Beer: 5.1, Dance: 4.9,... Wine → {barrel: 5.4, oak: 4.8, bouquet: 1.2,...
  8. The Cold Start Problem The Cold Start Problem Collaborative-filtering is

    based upon interactions. New events have no interactions. New events have no recommendations. (Note: this is not a problem for Amazon or Netflix)
  9. Content-Based Content-Based Recommendation Recommendation recommending related events is easy given

    some event X collect "important" terms from X's fields search for other events with X's "important" terms filter by location and category of X return remaining events sorted by relevance ... in other words Elasticsearch "MoreLikeThis" query
  10. Content-Based Content-Based Recommendation Recommendation recommending events to users is harder

    you must find content first scoop up user searches collect content from events that users interact with ask users to tell us about them! (user profiles!) to make the recommendation just search! (no fancy pants Machine Learning) ...
  11. Why? Why? Why would we want to combine search and

    recommendations? First we must understand search: Search is like a one big giant table of stuff. Search is good at finding matching docs. especially "find a doc that contains X" Search is good a scoring and sorting docs.
  12. Why? Why? Search allows recommendation methods to be combined. Search

    can incorporate business logic like location filtering. And you can still do search, but now informed by knowledge of the user making search. Why would we want to combine search and recommendations? We call this personalized search
  13. Content-Based Content-Based Recommendation in Search Recommendation in Search Already saw

    this. It's trivial. Related Events = MoreLikeThis Event recommendation = Perform a search with user's "content"
  14. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 1:

    for every user get recommended events and boost by those specific id curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "terms": { "id": ["Event456","Event567","Event678",...] }}}}} User123 → {Event456: 6.4, Event567: 5.1, Event678: 4.9,... "should": { "terms": { "id": ["Event456","Event567","Event678",...]
  15. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 2:

    find documents with this user in the user "users_who_might_like" field curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "term": { "users_who_might_like": "User123" }}}}} User123 → {Event456: 6.4, Event567: 5.1, Event678: 4.9, ...} "should": { "terms": { "users_who_might_like": "User123"
  16. Collaborative-Filtering Collaborative-Filtering Recommendation in Search Recommendation in Search Approach 3:

    index a "related_events" field and search with the events the user actually attended curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "term": { "related_events": ["Event456","Event567","Event678"] }}}}} Event789 → {Event456: 6.4, Event567: 5.1, Event678: 4.9, ...} "should": { "terms": { "related_events": ["Event456","Event567","Event678"]
  17. Easy Peasy Easy Peasy Recommendation in Search Recommendation in Search

    Given any user preference data, index it and then query for it using the same pattern. curl -XGET http://eventbrite_search:9200/events -d '{ "query": { "bool": { "must": { "multi_match": { "fields": ["title","description"], "query": "hello" } }, "should": { "terms": { "affinity_tags": ["20s","male","sporty","deal_seeker] }}}}} "should": { "terms": { "affinity_tags": ["20s","male","sporty","deal_seeker]
  18. Recommendation Recommendation Reimagined Reimagined The goal of recommendation is to

    provide the best events possible based upon the best information available.
  19. Recommendation Recommendation Reimagined Reimagined "Best information available" has two parts

    knowledge of events primary data: title, desc, category, date/time, location metadata: total ticket sales, tickets left, recent sales knowledge of user location previous event views previous event attendance previous searches profile information day of week/time preference cost preference or nothing at all
  20. Recommendation Recommendation Reimagined Reimagined But what happened to Search? It's

    still there. It's part of the knowledge about the user. It's the most important part. It's what the user wants right now!
  21. Challenges Challenges Possibilities Possibilities Personalized Search Related Events Newsletters Browse/Explore/Discovery

    Some sparsity (smaller locales, semantic content in events) How to combine so many signals Chicken:Egg :: Users:Recommendations