MongoDB for eCommerce

Building an eCommerce system is not easy, and building a platform is even harder. When it comes to data in eCommerce, there is nothing definite, no real structure you could stick to, and no final requirements. Something as obvious as the "item you add to cart" could be overly complicated when it comes to data. In this talk I will cover how we used MongoDB in a real world ecommerce system at OpenSky - an ecommerce startup in NYC - and how that changed the way we think about data and improved overall system design and performance

Transcript

1. MongoDB for eCommerce Bulat Shakirzyanov http://goo.gl/pXWnC Wednesday, February 15, 12
2. Hello, world! • @avalanche123 • github.com/avalanche123 • avalanche123.com Wednesday, February 15, 12
3. eCommerce is hard Wednesday, February 15, 12
4. Wednesday, February 15, 12
5. an ecommerce platform Wednesday, February 15, 12
6. unlimited sellers, suppliers and shoppers Wednesday, February 15, 12
7. Wednesday, February 15, 12
8. unlimited product categories Wednesday, February 15, 12
9. Apparel Electronics Jewelry Books Appliances Tools Wednesday, February 15, 12
10. and 4703 more categories Wednesday, February 15, 12
11. different product attributes Wednesday, February 15, 12
12. • size • style • color • brand • megapixels • image stabilization • memory slot type • battery life Wednesday, February 15, 12
13. how do you model that? Wednesday, February 15, 12
14. EAV Entity Attribute Value Wednesday, February 15, 12
15. Entity Entity PK id type Attribute Attribute PK id name type Value PK id FK entity_id FK attribute_id value Wednesday, February 15, 12
16. id type author size 1 book Charles Dickens NULL 2 shirt NULL m Products Wednesday, February 15, 12
17. id type 1 book 2 shirt id name type 1 author string 2 size enum(‘s’,‘m’,‘l’) Entities Attributes id entity_id attribute_id value 1 1 1 Charles Dickens 2 2 2 m Values Wednesday, February 15, 12
18. SELECT `e`.`id` as id, `v`.`value` as author FROM `entities` e JOIN `values` v ON `v`.`entity_id` = `e`.`id` JOIN `attributes` a ON `v`.`attribute_id` = `a`.`id` WHERE `e`.`type`='book' AND `a`.`name` = 'author' id author 1 Charles Dickens Wednesday, February 15, 12
19. problem solved? Wednesday, February 15, 12
20. Configurable products Wednesday, February 15, 12
21. Product Option Product Option FK product_id FK option_id Entity Entity PK id type Attribute Attribute PK id name type Value PK id FK entity_id FK attribute_id value Wednesday, February 15, 12
22. id type 1 product 2 product 3 option id name type 1 author string 2 size enum(‘s’, ‘m’, ‘l’) 3 entry_type string 4 display_type string Entities Attributes product_id option_id 2 3 Product Options Wednesday, February 15, 12
23. id entity_id attribute_id value 1 1 1 Charles Dickens 5 1 3 book 6 2 3 shirt 7 1 4 simple 8 2 4 configurable 9 3 2 m 10 3 3 shirt 11 3 4 simple Values Wednesday, February 15, 12
24. SELECT `e`.`id` as id, `v1`.`value` as type `v2`.`value` as display_type FROM `entities` e JOIN `values` v1 ON `v1`.`entity_id` = `e`.`id` JOIN `attributes` a1 ON `v1`.`attribute_id` = `a1`.`id` JOIN `values` v2 ON `v2`.`entity_id` = `e`.`id` JOIN `attributes` a2 ON `v2`.`attribute_id` = `a2`.`id` WHERE `e`.`id` = 2 AND `a1`.`name` = 'type' AND `a2`.`name` = 'display_type' id type display_type 2 shirt configurable Wednesday, February 15, 12
25. SELECT `o`.`option_id` as id, `v1`.`value` as type `v2`.`value` as display_type `v3`.`value` as size FROM `options` o JOIN `entities` e ON `e`.`id` = `o`.`option_id` JOIN `values` v1 ON `v1`.`entity_id` = `o`.`option_id` JOIN `attributes` a1 ON `v1`.`attribute_id` = `a1`.`id` JOIN `values` v2 ON `v2`.`entity_id` = `o`.`option_id` JOIN `attributes` a2 ON `v2`.`attribute_id` = `a2`.`id` JOIN `values` v3 ON `v3`.`entity_id` = `o`.`option_id` JOIN `attributes` a3 ON `v3`.`attribute_id` = `a3`.`id` WHERE `o`.`product_id`=2 AND `a1`.`name` = 'type' AND `a2`.`name` = 'display_type' AND `a3`.`name` = 'size' id type display_type size 3 shirt simple m Wednesday, February 15, 12
26. more fun... Wednesday, February 15, 12
27. Grouped products Wednesday, February 15, 12
28. Product Group Product Group FK group_id FK product_id Product Option Product Option FK product_id FK option_id Entity Entity PK id type Attribute Attribute PK id name type Value PK id FK entity_id FK attribute_id value Wednesday, February 15, 12
29. id type 1 product 2 product 3 option 4 group 5 product 6 product id name type 1 author string 2 size enum(‘s’, ‘m’, ‘l’) 3 entry_type string 4 display_type string Entities Attributes product_id option_id 2 3 Product Options group_id product_id 4 5 4 6 Product Groups Wednesday, February 15, 12
30. id entity_id attribute_id value 1 1 1 Charles Dickens 10 4 3 knife 11 4 4 group 12 5 3 knife 13 5 4 simple 14 6 3 knife 15 6 4 simple Values Wednesday, February 15, 12
31. Almost done Wednesday, February 15, 12
32. Bundled product Wednesday, February 15, 12
33. six months later... Wednesday, February 15, 12
34. Wednesday, February 15, 12
35. schema that can store entries of any structure Wednesday, February 15, 12
36. wrong use of tables Wednesday, February 15, 12
37. Wednesday, February 15, 12
38. alternative Wednesday, February 15, 12
39. & Wednesday, February 15, 12
40. use document-store for unstructured data and RDBMS for structured, highly relational data Wednesday, February 15, 12
41. { "_id": ObjectId("4bffd798fdc2120019040000") , "type": "simple" , "name": "Small T-Shirt" , "price": 12.99 , "SKU": "TSHRT-1S" , "size": "S" , "brand": "Some cool brand" } Attributes Type Wednesday, February 15, 12
42. { "_id": ObjectId("4bffd798fdc2120019040000") , "name": "T-Shirt" , "type": "configurable" , "options": [ { "name": "small" , "price": 12.99 , "SKU": "TSHRT-1S" , "size": "S" } , { "name": "medium" , "price": 15.99 , "SKU": "TSHRT-1M" , "size": "M" } , { "name": "large" , "price": 17.99 , "SKU": "TSHRT-1L" , "size": "L" } ] , "brand": "Some cool brand" } Attributes Type Wednesday, February 15, 12
43. { "_id": ObjectId("4bffd798fdc2120019040000") , "name": "Chef's Knife" , "type": "grouped" , "products": [ { "name": "5\"" , "price": 29.95 , "SKU": "CHF-KNF5" , "length": "5\"" } , { "name": "6\"" , "price": 34.95 , "SKU": "CHF-KNF6" , "length": "6\"" } , { "name": "7\"" , "price": 39.95 , "SKU": "CHF-KNF7" , "length": "7\"" } ] } Attributes Type Wednesday, February 15, 12
44. Benefits Wednesday, February 15, 12
45. Performance Wednesday, February 15, 12
46. Flexibility Wednesday, February 15, 12
47. Drawbacks Wednesday, February 15, 12
48. Drawbacks • No relations and cascades • No structure can lead to chaos • Unique indexes index NULL values • Learning curve Wednesday, February 15, 12
49. MongoDB ODM We’ve fixed it http://www.doctrine-project.org/projects/mongodb_odm Wednesday, February 15, 12
50. MongoDB ODM • Relations and cascades • Structure through classes • In-memory change tracking and optimized single flush persistence • On-the-fly migrations • API identical to Doctrine ORM Wednesday, February 15, 12
51. References • Oil Drilling Platform in the Santa Barbara CA Channel - http://www.flickr.com/photos/ mikebaird/3898808431/ • Busy times on Main Street - http://www.flickr.com/photos/yourfavoritemartian/4867673963/ • Black Vector T-Shirt - http://skyje.com/gallery/black-vector-t-shirt/ • Black Vector T-Shirt - http://skyje.com/gallery/black-vector-t-shirt/ Wednesday, February 15, 12
52. Questions? Thank you! Wednesday, February 15, 12
53. come work for Wednesday, February 15, 12