Welcome and Keynote Aaron Heckman, 10gen

Transcript

1. An Evening With MongoDB Wednesday, August 15, 12
2. An Evening With MongoDB Wednesday, August 15, 12
3. An Intro to MongoDB @aaronheckmann Evening with MongoDB Charlotte Wednesday, August 15, 12
4. Data model Wednesday, August 15, 12
5. Data model • Document oriented Wednesday, August 15, 12
6. Data model • vs Relational Wednesday, August 15, 12
7. Data model • Relational Product Product Product _id name created_at Wednesday, August 15, 12
8. Data model • Relational Product Product Product _id name created_at Product_Attribute Product_Attribute Product_Attribute Product_Attribute _id product_id key val Wednesday, August 15, 12
9. Data model • Relational • assembly required Product Product Product _id name created_at Product_Attribute Product_Attribute Product_Attribute Product_Attribute _id product_id key val Wednesday, August 15, 12
10. Data model • Relational • assembly required • app model != storage model Wednesday, August 15, 12
11. Data model • Document oriented Wednesday, August 15, 12
12. Data model • Document oriented • JSON Wednesday, August 15, 12
13. Data model • Document oriented • JSON-like Wednesday, August 15, 12
14. Data model • Document oriented • JSON-like • BSON Wednesday, August 15, 12
15. Data model • Document oriented • JSON-like • BSON • basically typed JSON • number, string, binary, array, etc Wednesday, August 15, 12
16. Data model Document oriented { _id: ObjectId(“..”) , name: “Panthers T-shirt” , created_at: ISODate("2012-08-15T15:42:09.195Z") } Wednesday, August 15, 12
17. Data model Document oriented { _id: ObjectId(“..”) , name: “Panthers T-shirt” , created_at: ISODate("2012-08-15T15:42:09.195Z") , props: [{ key: ‘string’, val: anything }] } Wednesday, August 15, 12
18. Data model • Document oriented • Ad-hoc query support Wednesday, August 15, 12
19. Data model Ad-hoc query support Wednesday, August 15, 12
20. Data model Ad-hoc query support • find, findOne Wednesday, August 15, 12
21. Data model Ad-hoc query support • find, findOne • accept a conditions object • regular expressions • numbers • strings • etc Wednesday, August 15, 12
22. Data model Ad-hoc query support • find, findOne • accept a conditions object • regular expressions • numbers • strings • etc • rich operators • $lt, $gt, $in, $ne, ... Wednesday, August 15, 12
23. Data model • Document oriented • Ad-hoc query support • Secondary indexing Wednesday, August 15, 12
24. Data model Secondary indexing Wednesday, August 15, 12
25. Data model Secondary indexing • createIndex() Wednesday, August 15, 12
26. Data model Secondary indexing • createIndex() • accepts an object Wednesday, August 15, 12
27. Data model Secondary indexing • createIndex() • accepts an object • options • unique, sparse, 2d, expiresAfterSeconds Wednesday, August 15, 12
28. Data model: gains Wednesday, August 15, 12
29. Data model: gains • Dynamic schemas Wednesday, August 15, 12
30. Data model: gains • Dynamic schemas • Data modeled directly to app Wednesday, August 15, 12
31. Data model: gains • Dynamic schemas • Data modeled directly to app • Retain ad-hoc queries Wednesday, August 15, 12
32. Data model: gains • Dynamic schemas • Data modeled directly to app • Retain ad-hoc queries • Retain secondary indexing Wednesday, August 15, 12
33. Data model: gains • Dynamic schemas • Data modeled directly to app • Retain ad-hoc queries • Retain secondary indexing • Productivity Wednesday, August 15, 12
34. Data model: losses Wednesday, August 15, 12
35. Data model: losses • Joins Wednesday, August 15, 12
36. Data model: losses • Joins • Multi-collection transactions Wednesday, August 15, 12
37. Data model: losses • Joins • Multi-collection transactions • use document level $atomics Wednesday, August 15, 12
38. Replication Wednesday, August 15, 12
39. Replication • distribute data across multiple machines Wednesday, August 15, 12
40. Replication • distribute data across multiple machines • redundancy Wednesday, August 15, 12
41. Replication • distribute data across multiple machines • redundancy • auto-failover Wednesday, August 15, 12
42. Replication • read from master or replicas Wednesday, August 15, 12
43. Replication • read from master or replicas • all writes go to master Wednesday, August 15, 12
44. Replication • read from master or replicas • all writes go to master • configurable • getLastError { w: ‘majority’ } Wednesday, August 15, 12
45. Sharding Wednesday, August 15, 12
46. Sharding • scale horizontally Wednesday, August 15, 12
47. Sharding • scale horizontally • range based partition mechanism Wednesday, August 15, 12
48. Sharding • scale horizontally • range based partition mechanism • shard key Wednesday, August 15, 12
49. Sharding • scale horizontally • range based partition mechanism • shard key • apps talk to shard set-ups the same way Wednesday, August 15, 12
50. MongoDB 2.2 overview @aaronheckmann Evening with MongoDB Charlotte Wednesday, August 15, 12
51. Mongo 2.2 Wednesday, August 15, 12
52. Mongo 2.2 • Concurrency improvements Wednesday, August 15, 12
53. Mongo 2.2 • Concurrency improvements • db level locking Wednesday, August 15, 12
54. Mongo 2.2 • Concurrency improvements • db level locking • more yielding Wednesday, August 15, 12
55. Mongo 2.2 • Concurrency improvements • Tag aware sharding Wednesday, August 15, 12
56. Mongo 2.2 • Concurrency improvements • Tag aware sharding • TTL collections Wednesday, August 15, 12
57. Mongo 2.2 • Concurrency improvements • Tag aware sharding • TTL collections • ensureIndex({ date: 1 }, { expiresAfterSeconds: 60*15 }) • cannot be compound • cannot be used on capped collections Wednesday, August 15, 12
58. Mongo 2.2 • Concurrency improvements • Tag aware sharding • TTL collections • Aggregation ... Wednesday, August 15, 12
59. MongoDB’s New Aggregation Framework @aaronheckmann Evening with MongoDB Charlotte Wednesday, August 15, 12
60. Aggregation in 2.0? Wednesday, August 15, 12
61. Map Reduce Wednesday, August 15, 12
62. Map Reduce A LARGE hammer Wednesday, August 15, 12
63. Map Reduce • Complex analytics on big data A LARGE hammer Wednesday, August 15, 12
64. Map Reduce • Complex analytics on big data • Distributed computing on clusters of machines A LARGE hammer Wednesday, August 15, 12
65. Problems Wednesday, August 15, 12
66. • Too complex Problems Wednesday, August 15, 12
67. • Too complex • Requires writing Javascript Problems Wednesday, August 15, 12
68. • Too complex • Requires writing Javascript • Expensive Problems Wednesday, August 15, 12
69. New Aggregation Framework Wednesday, August 15, 12
70. New Aggregation Framework • Declarative Wednesday, August 15, 12
71. New Aggregation Framework • Declarative • C++ implementation Wednesday, August 15, 12
72. New Aggregation Framework • Declarative • C++ implementation • Expression evaluation Wednesday, August 15, 12
73. New Aggregation Framework • Declarative • C++ implementation • Expression evaluation • Framework Wednesday, August 15, 12
74. Pipeline Wednesday, August 15, 12
75. Pipeline • Think Unix pipes Wednesday, August 15, 12
76. Pipeline • Think Unix pipes • Single collection Wednesday, August 15, 12
77. Pipeline • Think Unix pipes • Single collection • Series of operations Wednesday, August 15, 12
78. The Aggregation Command db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
79. The Aggregation Command • Takes two arguments db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
80. The Aggregation Command • Takes two arguments • aggregate: name of collection db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
81. The Aggregation Command • Takes two arguments • aggregate: name of collection • pipeline: array of operations db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
82. Aggregation helper db.article.aggregate( { $pipeline_op1 } , { $pipeline_op2 } , { $pipeline_op3 } , { $pipeline_op4 } , ... ); Wednesday, August 15, 12
83. Pipeline Operations Wednesday, August 15, 12
84. Pipeline Operations • $match • query predicate - coll.find(predicate) Wednesday, August 15, 12
85. Pipeline Operations • $match • query predicate - coll.find(predicate) • $project Wednesday, August 15, 12
86. Pipeline Operations • $match • query predicate - coll.find(predicate) • $project • reshapes results • include / exclude fields • computed fields Wednesday, August 15, 12
87. Pipeline Operations • $match • query predicate - coll.find(predicate) • $project • reshapes results • $unwind • hands out array elements one at a time in the context of their surrounding documents Wednesday, August 15, 12
88. Pipeline Operations • $match • query predicate - coll.find(predicate) • $project • reshapes results • $unwind • hands out array elements one at a time • $group • aggregates docs into buckets defined by a key Wednesday, August 15, 12
89. Pipeline Operations • $group aggregation expressions • _id is the group key • $sum • $avg • $push, $addToSet • more.. • $min, $max, $first, $last Wednesday, August 15, 12
90. Pipeline Operations • $sort • sorts documents • $limit • caps the number of documents • $skip • steps over the specified number of documents Wednesday, August 15, 12
91. Computed Expressions • Available in $project operations Wednesday, August 15, 12
92. Computed Expressions • Available in $project operations • Prefix expression language • Add two fields: • $add: [“$field1”, “$field2”] Wednesday, August 15, 12
93. Computed Expressions • Available in $project operations • Prefix expression language • Add two fields: • $add: [“$field1”, “$field2”] • Provide a value for a missing field: • $ifNull: [“$field1”, “$field2”] Wednesday, August 15, 12
94. Computed Expressions • Available in $project operations • Prefix expression language • Add two fields: • $add: [“$field1”, “$field2”] • Provide a value for a missing field: • $ifNull: [“$field1”, “$field2”] • Nesting: • $add: [“$field1”, $ifNull: [“$field2”, “$field3”]] Wednesday, August 15, 12
95. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp Wednesday, August 15, 12
96. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp • Date field extraction and arithmetic • Get year, month, day, hour, etc, from dates Wednesday, August 15, 12
97. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp • Date field extraction and arithmetic • Get year, month, day, hour, etc, from dates • Ternary conditional • Return one of two values based on a predicate Wednesday, August 15, 12
98. Demo Wednesday, August 15, 12
99. Usage Tips • Use $match as early as possible Wednesday, August 15, 12
100. Usage Tips • Use $match as early as possible • $sort (memory) Wednesday, August 15, 12
101. Usage Tips • Use $match as early as possible • $sort (memory) • $group • like $sort but not as much memory is needed Wednesday, August 15, 12
102. Sharding support • Mongos Wednesday, August 15, 12
103. Sharding support • Mongos • forwards ops up to first $group or $sort to shards Wednesday, August 15, 12
104. Sharding support • Mongos • forwards ops up to first $group or $sort to shards • combines shard server results and continues Wednesday, August 15, 12
105. http://docs.mongodb.org/manual/release-­‐notes/2.2/ http://docs.mongodb.org/manual/reference/aggregation/ http://www.mongodb.org/downloads @aaronheckmann @mongodb Questions? Wednesday, August 15, 12
106. http://docs.mongodb.org/manual/release-­‐notes/2.2/ http://docs.mongodb.org/manual/reference/aggregation/ http://www.mongodb.org/downloads @aaronheckmann @mongodb Thank  You! Wednesday, August 15, 12