MongoDB In the Cloud with Amazon EC2

From MongoDB LA, January 19, 2012

Transcript

1. In  the  Cloud  with  EC2  +  EBS Dan  Crosta,  10gen dcrosta@10gen.com @lazlofruvous
2. Agenda • MongoDB  deployment  components • Amazon  EC2  infrastructure • Deployment  scenarios  
3. MONGODB  COMPONENTS
4. MongoDB  Components • Actual data • Needs RAM + Disk IO • Stores sharding configuration • Stores small amounts of data • Infrequently queried/updated by MongoS • Stateless router • Typically run on App Servers • Can run as Arbiter • No data • Just votes to elect primary
5. Replica  Sets Primary Secondary Secondary Primary Secondary Arbiter Primary Secondary Secondary Secondary Secondary
6. Sharding MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD ConfigD ConfigD ConfigD MongoS MongoS MongoS
7. AMAZON  EC2
8. Amazon  EC2  Instance  Types 32-­‐bit  =  Don’t  Use ConfigD  /  Arbiter 32-­‐bit  =  Don’t  Use Typical  MongoD Big  MongoD High  CPU  not   necessary
9. OperaSng  System • Use  ext4,  xfs • Use  RAID – RAID10  on  MongoD – RAID1  on  ConfigbDB   • Turn  off  aSme • Raise  file  descriptor  limits cat >> /etc/security/limits.conf << EOF * hard nofile 65536 * soft nofile 65536 EOF
10. MONGODB  COMPONENTS  ON  EC2
11. MongoD  on  EC2 MongoD RAID  10 EBS Volumes 64-Bit EC2 Instance LVM and/or MDADM
12. MongoS  on  EC2 32- or 64-Bit EC2 Instance (Typically your app server) MongoS Your  App Language Specific Driver
13. Arbiter  on  EC2 Arbiter 64-Bit EC2 Instance (Micro Instance is Fine) Local Instance Storage
14. ConfigDB  on  EC2 ConfigDB 64-Bit EC2 Instance (Micro Instance is Fine) LVM or MDADM RAID  1 EBS Volumes
15. DEPLOYMENT  SCENARIOS
16. 3-­‐Node  Replica  Set   MongoD (Primary) RAID  10 MongoD (Secondary) RAID  10 MongoD (Secondary) RAID  10 Your  App Large Large Large
17. 2-­‐Nodes  +  Arbiter MongoD (Primary) RAID  10 MongoD (Secondary) RAID  10 Arbiter Your  App Large Large Micro
18. US  West US  East 3-­‐Node  Replica  Set   Your  App MongoD (Primary) RAID  10 Large MongoD (Secondary) RAID  10 Large MongoD (Secondary) Priority  =  0 RAID  10 Large
19. App  Tier Config  DB Shard  3 Shard  2 Shard  1 Sharding Amazon  ELB
20. RUNNING  IN  EC2
21. OperaSonal  ConsideraSons   • AutomaSng  Deployment   – Puppet   – Chef   – Amazon  CloudFormaSon  
22. Backup  /  Restore   • EBS  Snapshots – Also  for  rapid  replica  provisioning – Slightly  more  complicated  to  orchestrate  Sming   • LVM  Snapshot  (config+shards) • Use  secondaries  for  zero  downSme  backups • h_p://www.mongodb.org/pages/viewpage.acSon? pageId=19562846  
23. Security  Groups Component Default  Ports MongoD   27017,  28017  (Status  page) MongoD  Shard  Server 27018 MongoS 27017 ConfigDB 27019 • All nodes listen on all interfaces unless –bind_ip is specified • Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s) • Use security groups as access control lists • (e.g. allow 27017 from app servers, but not everybody)
24. QuesSons?   • Dan  Crosta – dcrosta@10gen.com – @lazlofruvous • Try  it  out!   – h_p://www.mongodb.org/downloads – h_p://www.mongodb.org/display/DOCS/Amazon +EC2