Surfacing the Products You Love on Spring

https://www.elastic.co/elasticon/conf/2018/sf/agenda?sess=surfacing-the-products-you-love-on-spring

Spring is a digital shopping destination that helps customers quickly find the products they love. They’re using Elasticsearch to return more relevant search results on their catalog of more than 300,000 products.

Learn how the Spring team builds a product catalog with Elasticsearch. Hear how they ensure product data is synced with information from external vendors — including product-specific metadata. Take a closer look at their Elasticsearch ingestion process, where they use cron jobs and NSQ to update products at least once every 12 hours. Plus, learn about projects the Spring team has tackled to improve their index and better understand their users' intent, including text search improvements with natural language processing (NLP) and refining products by color.

Transcript

1. Surfacing the Products You Love on Spring Julie Qiu Spring @jqiu25
2. @jqiu25 ES Building Search at Spring
3. 3 @jqiu25
6. Julie Qiu Engineering Lead Catalog Platform Spring @jqiu25
7. Get product data into database Search challenges @jqiu25
8. Get product data into database Keep database in sync with Elasticsearch Search challenges @jqiu25
9. 2,000+ brands 500,000+ products 1 checkout experience @jqiu25
12. How do I find the products that I want?
14. HEELS BLUE DRESS Finding the perfect outfit @jqiu25
16. Finding the perfect outfit @jqiu25
17. Finding the perfect outfit Search is hard @jqiu25
18. Finding the perfect outfit Search is hard Price matters @jqiu25
19. Finding the perfect outfit Search is hard Price matters Too many shoes @jqiu25
20. Finding the perfect outfit @jqiu25
21. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? @jqiu25
22. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? No need for words @jqiu25
23. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? No need for words This time, it’s personal @jqiu25
24. Search is hard: Finding a pair of heels
25. Let’s search for a pair of heels. heels @jqiu25
27. 14,559 results @jqiu25
28. @jqiu25
29. Out of Stock @jqiu25
30. Out of Stock Now $50 @jqiu25
31. Out of Stock Back to $695 Now $50 @jqiu25
32. Out of Stock Back to $695 Now $50 @jqiu25
33. ES Search is only half the story @jqiu25
34. • Get product data from suppliers Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
35. Getting product data @jqiu25
36. Some product data is pushed @jqiu25
37. Some product data is pulled @jqiu25
38. Product data ingestion Catalog Ingestion System @jqiu25
39. • Get product data from suppliers • Sync Postgres with Elasticsearch Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
40. Sync database with Elasticsearch ES repopulate (nightly) ES Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
41. Products Index Alias products (alias) products_20180131030000 products_20180131030000 products_20180129030000 products_group (index_group) (index) @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
42. Alexandre Birman Elouise Suede Platform Sandals $695 $278 @jqiu25
43. Out of Stock @jqiu25
44. • Get product data from suppliers • Sync database with Elasticsearch • Real-time updates Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
45. Real-time updates @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
46. Real-time updates Consistent Data Channon Heeled Sandal by Club Monaco @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
47. Consistent Data Channon Heeled Sandal by Club Monaco Regularly Updated $298 Now $139.30 Sizes: • 38 • 39 • 40 Unavailable Real-time updates @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
48. Real-time updates ES repopulate (nightly) ES Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
49. Real-time updates ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
50. Club Monaco Channon Heeled Sandal $298 @jqiu25
51. Club Monaco Channon Heeled Sandal $298 Club Monaco Channon Heeled Sandal $298 $199 @jqiu25
52. Price Matters: Finding a pair of heels on sale
53. • Mark products on sale Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
54. Mark products on sale ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
55. Mark products on sale @jqiu25
56. Mark products on sale Sale Start and End Dates @jqiu25
57. Mark products on sale Sale Amount @jqiu25
59. • Mark products on sale • Store prices in Elasticsearch Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
60. Store prices in Elasticsearch Sale Tagger ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
61. • Mark products on sale • Store prices in Elasticsearch • Compute sale prices Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
62. Compute sale prices { "price":"409", "prices":[ { "active_dates":{ "start":"2000-01-01T00:00:00Z", "end":"2100-01-01T00:00:00Z" }, "price":"409", "discount_amount":48, } ] } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
63. Compute sale prices Consumer API ES @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
65. Too many shoes: Refining search results
66. Let’s search for a pair of blue heels. blue heels @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
68. What I expect when I search for “blue heels” Too Many Shoes Price Matters Search Is Hard Adding NLP Visual Browse Personalized Experience @jqiu25
69. Baby Powder Candied Veg Royal Silk Actual colors of these heels @jqiu25 Too Many Shoes Price Matters Search Is Hard Adding NLP Visual Browse Personalized Experience
70. • Categorize data Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
71. Categorizing data: Sometimes it’s simple @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
72. What brand are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
73. Vince Camuto Shoes of Prey Club Monaco What brand are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
74. Categorizing data: Sometimes it’s simple Sometimes it’s complicated @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
75. What color are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
76. What color are these heels? Candied Veg Royal Silk Baby Powder @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
77. Color Categorization API Categorize data @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
78. • Categorize data • Aggregate documents Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
79. Aggregate documents @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
80. • Categorize data • Aggregate documents • Display results Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
82. @jqiu25 We found a pair of heels! Club Monaco Channon Heeled Sandal $298 $199
84. Let’s search for a blue dress blue dress @jqiu25
86. blue dress shirt blue dress shoes @jqiu25
87. What is a blue dress? (for humans) @jqiu25
88. What is a blue dress? (for machines) @jqiu25
89. What is a blue dress? Incorporating Natural Language Processing
90. @jqiu25
91. • Get Twiggle product features Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
92. Get Twiggle product features Catalog Ingestion System ES Elasticsearch Cron Jobs NSQ @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
93. • Get Twiggle product features • Index Twiggle’s product ontology Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
94. Index Twiggle’s product ontology { “$concept_sm”: [“Dress”, “FashionProduct”], “$color_o”: {"$concept_sm": [”Blue”]}, “design_patterns_o”: {"$concept_sm": [”Floral”, “Pattern”]}, “material_so”: {"$concept_sm": [”Silk”]}, “persona_so”: {"$concept_sm": [”Women”]}, /* …more features… */ } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
95. • Get Twiggle product features • Index Twiggle’s product ontology • Interpret search queries Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
96. Interpreting search queries @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
97. Interpret search queries “blue dress with a floral pattern under 1600” { “@combine”: [ {“$concept_sm”: {“@eq”: “Dress”}}, {“$color_sm”: {“@eq”: “Blue”}}. {“$$price”: {“usd”: {“@lte”:1600}}}. {“design_patterns_o”:{ “$concept_sm”: {”@eq”: “Floral”} }}}, ] } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
99. What I picture wearing in my mind @jqiu25
100. Assorted dresses @jqiu25
101. Sort of what I want @jqiu25
102. No need for words: Similarity and visual browse
103. • Classify product attributes Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
104. • Name • Brand • Price Classify product attributes • Size • Category • Color @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
105. • Classify product attributes • Generate similar products Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
106. Sort of what I want @jqiu25
107. Generate similar products @jqiu25 Emilio Pucci Ruffle Sleeve Dress $1,280
108. Generate similar products @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
109. • Classify product attributes • Generate similar products • Browse by similar products Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
111. This time, it’s personal: We look good in different blue dresses
112. • Classify product attributes (better) This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
113. Attributes we know • Name • Brand • Price • Size • Category • Color @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
114. • Style • Occasion • Season Attributes we want to know • Shape • Trend • … and more! @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
115. How do we define style? @jqiu25
116. Catalog Ingestion System ES Elasticsearch Sync Jobs Attribute Tagging System @jqiu25 NSQ Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
117. • Classify product attributes (better) • Understand user preferences This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
118. Understand style preferences @jqiu25
119. Understand style preferences @jqiu25
120. Understand style preferences @jqiu25
121. Track user events Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
122. Track user events Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
123. Track user events Favorited products Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
124. Track user events Brands loved Favorited products Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
125. • Classify product attributes (better) • Understand user preferences • Personalize results This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
126. Personalize results Popular Products ES Personalized Products User Preferences User Events @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
127. Blue dresses personalized for me! women > clothing > dresses Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 Roland Mouret GERZON GOWN – NAVY WOOL $3,200 Laundry by Shelli Segal MATTE CREPE MERMAID GOWN $295 $177 @jqiu25
128. We found a blue dress! Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25
130. @jqiu25 ES Building Search at Spring
131. It’s more than looking at raw data.
132. It’s more than keyword matching.
133. It’s about intent.
134. Thanks! Julie Qiu @jqiu25 Promo Link: bit.ly/spring-elasticon