Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MongoDB Intro + Performance Tuning & Monitoring

James Tan
March 19, 2014

MongoDB Intro + Performance Tuning & Monitoring

James Tan

March 19, 2014
Tweet

More Decks by James Tan

Other Decks in Technology

Transcript

  1. Senior Consultant MongoDB, Inc. London, UK James Tan #MongoDB User

    Group - Singapore Performance Tuning and Monitoring
  2. MongoDB is a ___________ database •  Document •  Open source

    •  High performance •  Horizontally scalable •  Full featured
  3. Document Database •  Not for .PDF & .DOC files • 

    A document is essentially an associative array •  Document = JSON object •  Document = PHP Array •  Document = Python Dict •  Document = Ruby Hash •  etc
  4. Open Source •  MongoDB is an open source project • 

    On GitHub •  Licensed under the AGPL •  Started & sponsored by MongoDB Inc (formerly 10gen) •  Commercial licenses available •  Contributions welcome
  5. High Performance •  Written in C++ •  Extensive use of

    memory-mapped files i.e. read-through write-through memory caching. •  Runs nearly everywhere •  Data serialized as BSON (fast parsing) •  Full support for primary & secondary indexes •  Document model = less work
  6. Full Featured •  Ad Hoc queries •  Real time aggregation

    •  Rich query capabilities •  Strongly consistent •  Geospatial features •  Support for most programming languages •  Flexible schema
  7. Node 1 Secondary Config Server Node 1 Secondary Config Server

    Node 1 Secondary Config Server Shard Shard Shard Mongos App Server Mongos App Server Mongos App Server Sharded Cluster
  8. What is performance tuning? 1.  Assess the problem and establish

    acceptable behavior 2.  Measure the current performance 3.  Find the bottleneck* 4.  Remove the bottleneck 5.  Re-test to confirm 6.  Lather, rinse, repeat * - (This is often the hard part) (Adapted from http://en.wikipedia.org/wiki/Performance_tuning )
  9. Pro-Tip: Know Thyself You have to recognize normal to know

    when it isn’t Source: http://www.flickr.com/photos/skippy/6853920/
  10. Metrics to Watch •  Memory usage •  Opcounters •  Lock

    % •  Queues •  Background flush average •  Replication stats
  11. What is MMS? The MongoDB Management Service: a free service

    (or software) for monitoring and backup
  12. Setting it up http://mms.mongodb.com/help/monitoring/tutorial/ •  Setup an account •  Install

    the agent •  Add your hosts •  Optional: hardware stats through munin-node •  Optional: enable logging and profiling •  More: mms.mongodb.com/help/monitoring/install/
  13. Notes •  Agent written in Go (was Python) •  Failover:

    run multiple agents (1 primary) •  Hosts: use CNAMEs, especially on AWS! •  You can use a group per env (each needs an agent) •  Connections are over SSL •  On-Premise solution for those that don’t want to use the hosted service
  14. Example: Replication Lag •  Secondaries under-specced vs primaries •  Access

    patterns between primary/ secondaries •  Insufficient bandwidth •  Foreground index builds on secondaries “…when you have eliminated the impossible, whatever remains, however improbable, must be the truth…” -- Sherlock Holmes Sir Arthur Conan Doyle, The Sign of the Four
  15. Example: Slow Performance Log analysis The best tools for analyzing

    MongoDB logs are included in mtools*: mlogfilter (filter logs for slow queries, table scans, etc…) mplotqueries (graph query response times and volumes) * https://github.com/rueckstiess/mtools
  16. Example: Slow Performance Log analysis (example syntax) Show me queries

    that took more than 1000 ms from 6 am to 6 pm: mlogfilter  mongodb.log  -­‐-­‐from  06:00  -­‐-­‐to   18:00  -­‐-­‐slow  1000  >  mongodb-­‐filtered.log   Now, graph those queries: mplotqueries  -­‐-­‐logscale  mongodb-­‐ filtered.log  
  17. Example: Slow query Wed  Jul  17  13:40:14  [conn28600]  query  x.y

     [snip]  ntoreturn:16   ntoskip:0  nscanned:16779  scanAndOrder:1  keyUpdates:0   numYields:  906  locks(micros)  r:46877422  nreturned:16  reslen: 6948  38172ms 38 seconds! Scanned 17k documents, returned 16.
  18. Example: Slow update Wed  Jul  17  14:11:59  [conn56541]  update  x.y

     query:  {  e:   ”[id1]"  }  update:  {  $addToSet:  {  fr:  ”[id2]"  }  }  nscanned:1   nmoved:1  nupdated:1  keyUpdates:0  locks(micros)  w:85145  11768ms Almost 12 seconds! This time, there’s “nmoved:1”, too. This means a document was moved on disk – it outgrew the space allocated for it.
  19. Monitoring: watch for warnings MMS warns you if your systems

    have startup warnings or if they are running outdated versions. Don’t ignore these!