Building your first mongoDB application

A rushed introduction of the features of MongoDB by walking through how one can building a simple location-based checkin application using MongoDB. The talk will cover the basics of MongoDB's document model, query language, map-reduce framework and deployment architecture.

Transcript

1. Ross Lawley - ross@10gen.com twitter: @RossC0 Building Your First MongoDB Application Tuesday, 20 March 12
2. Today's Talk • Quick introduction to mongoDB • Data modelling in mongoDB, queries, geospatial, updates and map reduce. • Using a location-based app as an example • Example works in mongoDB JS shell Tuesday, 20 March 12
3. What is mongoDB? MongoDB is a scalable, high-performance, open source NoSQL database. • Document-oriented storage • Full Index Support • Replication & High Availability • Auto-Sharding • Querying • Fast In-Place Updates • Map/Reduce • GridFS Tuesday, 20 March 12
4. Where can you use it? MongoDB is Implemented in C++ • Platforms 32/64 bit Windows, Linux, Mac OS-X, FreeBSD, Solaris Drivers are available in many languages 10gen supported • C, C# (.Net), C++, Erlang, Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, Scala, nodejs Community supported • Clojure, ColdFusion, F#, Go, Groovy, Lua, R ... http://www.mongodb.org/display/DOCS/Drivers Tuesday, 20 March 12
5. Terminology RDBMS MongoDB Table Collection Row(s) JSON Document Index Index Join Embedding & Linking Partition Shard Partition Key Shard Key Tuesday, 20 March 12
6. { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Ross", date : ISODate("2012-02-02T11:52:27.442Z"), text : "About MongoDB...", tags : [ "tech", "databases" ], comments : [{ author : "Fred", date : ISODate("2012-02-03T17:22:21.124Z"), text : "Best Post Ever!" }], comment_count : 1 } Example Document Tuesday, 20 March 12
7. BSON • JSON has powerful, but limited set of datatypes • Mongo extends datypes with Date, Int types, Id, … • MongoDB stores data in BSON • BSON is a binary representation of JSON • Optimized for performance and navigational abilities • Also compression See: bsonspec.org Tuesday, 20 March 12
8. Why use mongoDB? • Intrinsic support for agile development • Super low latency access to your data • Very little CPU overhead • No additional caching layer required • Built in replication and horizontal scaling support Tuesday, 20 March 12
9. Building Your First MongoDB App • Want to build an app where users can check in to a location • Leave notes or comments about that location Iterative Approach: • Decide requirements • Design documents • Rinse, repeat :-) Tuesday, 20 March 12
10. Requirements "As a user I want to be able to find other locations nearby" • Need to store locations (Offices, Restaurants etc) • name, address, tags • coordinates • Stretch Task! - User generated content e.g. tips / notes Tuesday, 20 March 12
11. Requirements "As a user I want to be able to 'checkin' to a location" Checkins • User should be able to 'check in' to a location • Want to be able to generate statistics: • Recent checkins • Popular locations Tuesday, 20 March 12
12. Collections users user1, user2 loc1, loc2, loc3 locations checkins checkin1, checkin2 Tuesday, 20 March 12
13. > location_1 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117 } Locations v1 Tuesday, 20 March 12
14. > location_1 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117 } > db.locations.find({name: "proArte Hotel Berlin"}) Locations v1 Tuesday, 20 March 12
15. > location_2 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"] } Locations v2 Tuesday, 20 March 12
16. > location_2 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"] } > db.locations.ensureIndex({tags: 1}) Locations v2 Tuesday, 20 March 12
17. > location_2 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"] } > db.locations.ensureIndex({tags: 1}) > db.locations.find({tags: "hotel"}) Locations v2 Tuesday, 20 March 12
18. > location_3 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"], lat_long: [52.5184, 13.387] } Locations v3 Tuesday, 20 March 12
19. > location_3 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"], lat_long: [52.5184, 13.387] } > db.locations.ensureIndex({lat_long: "2d"}) Locations v3 Tuesday, 20 March 12
20. > location_3 = { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"], lat_long: [52.5184, 13.387] } > db.locations.ensureIndex({lat_long: "2d"}) > db.locations.find({lat_long: {$near:[52.53, 13.4]}}) Locations v3 Tuesday, 20 March 12
21. // creating your indexes: > db.locations.ensureIndex({tags: 1}) > db.locations.ensureIndex({name: 1}) > db.locations.ensureIndex({lat_long: "2d"}) // finding places: > db.locations.find({lat_long: {$near:[52.53, 13.4]}}) // with regular expressions: > db.locations.find({name: /^proart/i}) // by tag: > db.locations.find({tag: "hotel"}) Finding locations Tuesday, 20 March 12
22. Updating Documents Atomic operators: $set, $unset, $inc, $push, $pushAll, $pull, $pullAll, $bit Tuesday, 20 March 12
23. // initial data load: > db.locations.insert(location_3) // adding a tip with update: > db.locations.update( {name: "proArte Hotel Berlin"}, {$push: { tips: { user: "Ross", date: "20/03/2010", tip: "Check out my replication talk later!"} }}) Inserting locations - adding tips Tuesday, 20 March 12
24. > db.locations.findOne() { name: "proArte Hotel Berlin", address: "Friedrichstrasse 151", city: "Berlin", post_code: 10117, tags: ["hotel", "conference", "mongodb"], lat_long: [52.5184, 13.387], tips:[{ user: "Ross", date: "20/03/2010", tip: "Check out my replication talk later!" }] } Stretch task - done Tuesday, 20 March 12
25. Requirements "As a user I want to be able to find other locations nearby" Need to store locations (Offices, Restaurants etc) • name, address, tags • coordinates • Stretch task! - User generated content e.g. tips / notes "As a user I want to be able to 'checkin' to a location" Checkins • User should be able to 'check in' to a location • Want to be able to generate statistics: • Recent checkins • Popular locations Tuesday, 20 March 12
26. > user_1 = { _id: "ross@10gen.com", name: "Ross", twitter: "RossC0", checkins: [ {location: "ProArte Hotel", ts: "20/03/2010"}, {location: "Tegel Airport", ts: "19/03/2010"} ] } > db.users.ensureIndex({checkins.location: 1}) > db.users.find({checkins.location: "ProArte Hotel"}) Users and Checkins Tuesday, 20 March 12
27. // find all users who've checked in here: > db.users.find({"checkins.location":"Hotel Berlin"}) Simple Stats Tuesday, 20 March 12
28. // find all users who've checked in here: > db.users.find({"checkins.location":"Hotel Berlin"}) // find the last 10 checkins here: > db.users.find({"checkins.location":"Hotel Berlin"}) .sort({"checkins.ts": -1}).limit(10) Simple Stats Tuesday, 20 March 12
29. // find all users who've checked in here: > db.users.find({"checkins.location":"Hotel Berlin"}) // find the last 10 checkins here: - Warning! > db.users.find({"checkins.location":"Hotel Berlin"}) .sort({"checkins.ts": -1}).limit(10) Simple Stats Tuesday, 20 March 12
30. // find all users who've checked in here: > db.users.find({"checkins.location":"Hotel Berlin"}) // find the last 10 checkins here: - Warning! > db.users.find({"checkins.location":"Hotel Berlin"}) .sort({"checkins.ts": -1}).limit(10) // count how many checked in today: - Warning! > db.users.find({"checkins.location":"Hotel Berlin", "checkins.ts": {$gt: midnight}} ).count() Simple Stats Tuesday, 20 March 12
31. // find all users who've checked in here: > db.users.find({"checkins.location":"Hotel Berlin"}) // find the last 10 checkins here: - Not as expected! > db.users.find({"checkins.location":"Hotel Berlin"}) .sort({"checkins.ts": -1}).limit(10) // count how many checked in today: - Warning! > db.users.find({"checkins.location":"Hotel Berlin", "checkins.ts": {$gt: midnight}} ).count() Simple Stats Tuesday, 20 March 12
32. > user_2 = { _id: "ross@10gen.com", name: "Ross", twitter: "RossC0", } > checkin_1 = { location: location_id, user: user_id, ts: "20/03/2010" } > db.checkins.ensureIndex({checkins.location: 1}) > db.checkins.find({user: user_id}) User and Checkins v2 Tuesday, 20 March 12
33. // find all users who've checked in here: > location_id = db.checkins.find({"name":"Hotel Berlin"}) > u_ids = db.checkins.find({location: location_id}, {_id: -1, user: 1}) > users = db.users.find({_id: {$in: u_ids}}) // find the last 10 checkins here: > db.checkins.find({location: location_id}) .sort({ts: -1}).limit(10) // count how many checked in today: > db.checkins.find({location: location_id, ts: {$gt: midnight}} ).count() Simple Stats Tuesday, 20 March 12
34. // Find most popular locations > map_func = function() { emit(this.location, 1); } > reduce_func = function(key, values) { return Array.sum(values); } > db.checkins.mapReduce(map_func, reduce_func, {query: {ts: {$gt: now_minus_3_hrs}}, out: "result"}) > db.result.findOne() {"_id": "Hotel Berlin", "value" : 17} Map Reduce Tuesday, 20 March 12
35. // Find most popular locations > agg = db.checkins.aggregate( {$match: {ts: {$gt: now_minus_3_hrs}}}, {$group: {_id: "$location", value: {$sum: 1}}} ) > agg.result [{"_id": "Hotel Berlin", "value" : 17}] Aggregation Tuesday, 20 March 12
36. P Deployment • Single server - need a strong backup plan Tuesday, 20 March 12
37. Deployment • Single server - need a strong backup plan • Replica sets - High availability - Automatic failover P P S S Tuesday, 20 March 12
38. Deployment • Single server - need a strong backup plan • Replica sets - High availability - Automatic failover • Sharded - Horizontally scale - Auto balancing P S S P S S P P S S Tuesday, 20 March 12
39. MongoDB Use Cases • Archiving • Content Management • Ecommerce • Finance • Gaming • Government • Metadata Storage • News & Media • Online Advertising • Online Collaboration • Real-time stats/analytics • Social Networks • Telecommunications Tuesday, 20 March 12
40. Tuesday, 20 March 12
41. @mongodb conferences, appearances, and meetups http://www.10gen.com/events http://bit.ly/mongofb Facebook | Twitter | LinkedIn http://linkd.in/joinmongo download at mongodb.org support, training, and this talk brought to you by Tuesday, 20 March 12