MongoDB London 2011 - MongoDB Command Line Tools

Transcript

1. mongo* Just what do those commands do? By Russell Smith Friday, 20 April 12
2. /usr/bin/whoami • Russell Smith • Consultant for UKD1 Limited • I Specialise in helping companies going through rapid growth; • Code, architecture, infrastructure, devops, sysops, capacity planning, etc • <3 MongoDB, Neo4j, MySQL, Riak, Gearman, Kohana, PHP, Debian, Puppet, etc... Friday, 20 April 12
3. Huh? • MongoDB ships with a variety of CLI tools • Some popular; mongodump, mongorestore, mongostat, mongo • Some less so; mongosniff, mongoexport, mongoimport, bsondump, etc Friday, 20 April 12
4. Core Friday, 20 April 12
5. mongo • Probably the most commonly used • ‘Hidden’ gems; • ~/.mongorc.js - will be executed by default • --eval “some js” - evaluates Javascript and exits Friday, 20 April 12
6. mongo (cont.) ~ mongo test --eval "printjson(db.getCollectionNames())" MongoDB shell version: 2.1.0-pre- connecting to: test [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ] ~ mongo test --quiet --eval "printjson(db.getCollectionNames())" [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ] Friday, 20 April 12
7. ~/.mongorc.js x = db.queue.count({state:4}); if (x > 0) { print('WARNING: ' + x + ' hung jobs'); } Friday, 20 April 12
8. ~/.mongorc.js ~ mongo MongoDB shell version: 2.1.0-pre- connecting to: test WARNING: 2 hung jobs > Friday, 20 April 12
9. mongod • The server itself • Useful: • --fastsync - allows (much) faster starting of a replset member by using a (recent) snapshot of a peer • --keyFile - replset / sharding authentication file Friday, 20 April 12
10. mongos • Sits in front of your shard servers and directs queries • --keyFile Friday, 20 April 12
11. Common Friday, 20 April 12
12. mongodump • Used to take backups • Common options: • --oplog • -d • -c Friday, 20 April 12
13. mongodump (cont.) ~ mongodump -d test -c queue -q "{state:4}" connected to: 127.0.0.1 DATABASE: test to dump/test test.queue to dump/test/queue.bson 2 objects Friday, 20 April 12
14. mongorestore • Restores backups taken with mongodump • Interesting options; • --filter • --oplogReplay • Possible performance issues with custom (non ObjectId) _id Friday, 20 April 12
15. mongorestore (cont.) mongorestore -d test -c queue --filter "{state:4}" dump/test/queue.bson connected to: 127.0.0.1 Mon Sep 19 07:34:03 dump/test/queue.bson Mon Sep 19 07:34:03 going into namespace [test.queue] 764 objects found 2 objects processed Friday, 20 April 12
16. mongostat • Produces periodically updated stats • Useful in watching server load & spotting general performance issues Friday, 20 April 12
17. Cursor paging Common operations fsync flushes Total data size locked active clients queues Friday, 20 April 12
18. mongostat (cont’d) • Watch for; • faults (page faults) • idx miss (index btree page misses) • locked (percentage of time globally locked) • repl (it should normally be M or SEC) Friday, 20 April 12
19. Others Friday, 20 April 12
20. mongoexport • Useful to export data for use with another database or program • You may specify down to the field level using a filter if needed • Can export as; • JSON (default) • CSV Friday, 20 April 12
21. mongoimport • Imports data in to mongodb • Currently supports importing; • JSON • CSV • TSV Friday, 20 April 12
22. mongoimport (cont’d) • Useful options include; • --upsert - allows updating objects that exist already (see --upsertFields) • --headerline - if your CSV / TSV contains headers • -f / --fields - specify field names Friday, 20 April 12
23. bsondump • Convert bson -> json bsondump test.bson --type json { "_id" : ObjectId( "4e72f51414345f302b0eea2e" ), "test" : 1 } 1 objects found Friday, 20 April 12
24. bsondump (cont.) • Debug info: bsondump test.bson --type debug --- new object --- size : 36 _id type: 7 size: 17 test type: 1 size: 14 etc Friday, 20 April 12
25. mongofiles ~ mongofiles put test.txt connected to: 127.0.0.1 added file: { _id: ObjectId('4e761b07c21062d06b349819'), filename: "test.txt", chunkSize: 262144, uploadDate: new Date(1316363015625), md5: "d41d8cd98f00b204e9800998ecf8427e", length: 12256 } done! Friday, 20 April 12
26. mongofiles (cont.) ~ mongofiles list connected to: 127.0.0.1 test.txt 12256 Friday, 20 April 12
27. mongotop • New in 1.9 • Collection level load stats Friday, 20 April 12
28. mongosniff • Used for sniffing the traffic between client / server • Can forward traffic • Only really useful for debugging... Friday, 20 April 12
29. Friday, 20 April 12
30. mongo-perf • Does not ship with mongodb See : https://github.com/mongodb/mongo-perf • I can’t compile it, but it probably works Friday, 20 April 12
31. Useful third party tools Friday, 20 April 12
32. tungsten replicator • http://code.google.com/p/tungsten-replicator/ • http://code.google.com/p/tungsten-replicator/wiki/ mysql> create table foo(id int primary key, msg varchar(35)); Query OK, 0 rows affected (0.05 sec) mysql> insert into foo values(1, 'hello from MySQL!'); Query OK, 1 row affected (0.00 sec) > db.foo.find() { "_id" : ObjectId("4dc77bacad9092bd1aef046d"), "id" : "1", "data" : "hello from MySQL!" } Friday, 20 April 12
33. automongobackup • A port of the popular AutoMySQLBackup • Rotates & compresses backups • https://github.com/micahwedemeyer/automongobackup Friday, 20 April 12
34. mongolog • Full query logging • Uses mongosniff • Will be on github soon Friday, 20 April 12
35. Other interesting projects • Eliots Munin plugins : https://github.com/erh/mongo-munin • gridfs-fuse : https://github.com/mikejs/gridfs-fuse Friday, 20 April 12
36. Further reading • Tungsten with Mongo details http://scale-out-blog.blogspot.com/2011/05/introducing-mysql-to-mongodb.html • Javascript shell API http://api.mongodb.org/js/current/ • Admin UI’s http://www.mongodb.org/display/DOCS/Admin+UIs • Hacking the shell http://bit.ly/r2vghH Friday, 20 April 12
37. Questions? Friday, 20 April 12