Upgrade to Pro — share decks privately, control downloads, hide ads and more …

NoSQL Forensics: What to do with (No)ARTIFACTS

NoSQL Forensics: What to do with (No)ARTIFACTS

MongoDB, Elasticsearch, and CouchDB. With the explosive
growth of these solutions, it’s only a matter of time before you
run into them on your next engagement. Wouldn’t you like to
know what to do with them?
Using a combination of log analytics and an understanding of
core functions of these databases, attendees will learn how to
examine NoSQL artifacts for user activity and data interaction.
We’ll begin with basic CRUD operations to understand how
information is logged, and then expand upon artifact analysis
to trace user sessions, attribute activity to particular users, and
even trace sessions to unique IPs. We’ll also examine artifacts
from RESTful interfaces of NoSQL databases, understanding
how users don’t need to access a machine to interact with data.
This session promises to offer attendees, both new and
experienced, a new perspective on NoSQL databases, artifacts,
and forensic analysis.

bromiley

July 08, 2015
Tweet

More Decks by bromiley

Other Decks in Technology

Transcript

  1. 1 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. NoSQL Forensics What to Do with (No)ARTIFACTS Matt Bromiley Senior Consultant, Mandiant
  2. 2 © Mandiant, a FireEye Company. All rights reserved. Agenda

    •  $ whoami •  Why Does this Matter? •  Examining MongoDB Artifacts •  Config File •  CRUD Artifacts •  User Sessions •  In-Row Data •  Searching Through Memory •  NoSQL Triage •  Q&A
  3. 3 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. $ whoami
  4. 4 © Mandiant, a FireEye Company. All rights reserved. $

    whoami •  Currently a Senior Consultant with Mandiant •  4+ years experience with a focus on data breaches, incident response, network security monitoring, and digital forensics •  Work with clients from small, regional shops to multinational Fortune 50s •  LOVE to develop & work with open source forensic tools •  LOVE to share, teach (woohoo FOR508!), learn, and help others improve (while improving myself!) Tweet/Git/Blog [@]505Forensics[.com]
  5. 5 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. Why Does This Matter?
  6. 6 © Mandiant, a FireEye Company. All rights reserved. Why

    Does This Matter? •  With Microsoft, Oracle, SAP, and IBM leading the pack, who cares about NoSQL?! •  NoSQL is on the rise! •  Easy to deploy •  Web app friendly •  APIs out the … •  Open-source, so lots and lots of players •  MongoDB •  Elastic(search) •  Couchbase •  Cassandra •  Hadoop Gartner’s Magic Quadrant
  7. 7 © Mandiant, a FireEye Company. All rights reserved. Why

    Does This Matter? (cont.) •  Who's Using This Stuff? •  MongoDB: ADP, The Weather Channel, MetLife, City of Chicago •  Elastic(search): Netflix, Target, LinkedIn, OpenTable, GitHub •  Couchbase: BMW, U.S. Senate, Comcast, Starbucks, eBay, AOL •  Hadoop: eBay, Expedia, Kayak, Samsung •  Even with all this support..the default sucks! •  Security is not a primary consideration •  Ease of coding, data accessibility > data security
  8. 8 © Mandiant, a FireEye Company. All rights reserved. Why

    Does This Matter? (cont.) •  Reports are being published on Internet-facing MongoDB servers with little or NO authentication •  Three students from University of Saarland in Germany published a report in January 2015 of nearly 40,000 MongoDB databases openly available on the Internet •  Shodan can be used to scan for NoSQL databases https://cispa.saarland/wp-content/uploads/2015/02/MongoDB_documentation.pdf
  9. 9 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. Examining MongoDB Artifacts
  10. 10 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  Config File •  /etc/mongodb.conf •  First place to look! •  Logging •  Where •  What •  How •  Security •  Authentication On/Off? •  Access •  IP Address •  Connection Ports •  Web Server
  11. 11 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations λ  CREATE $ mongo > use testdata switched to db testdata > db.blog.insert({“Title”:”My First Post”,”Author”:”505Forensics”})
  12. 12 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations (cont.) Log Output v2.x v3.x
  13. 13 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations λ  READ > db.blog.findOne() { "_id" : ObjectId("55879d3aeca5d08121fe1118"), "Title" : "My New Blog", "Author" : "505Forensics" }
  14. 14 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations (cont.) Log Output…
  15. 15 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations λ  UPDATE > post = db.blog.findOne() > post.comments = [{"Comment 1":"This is a comment!"},{"Comment 2":"This is another comment!"}] > db.blog.update({“Title”:”My New Blog”},post)
  16. 16 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations (cont.) Log Output…
  17. 17 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations λ  DELETE > db.blog.remove({“Title”:”My New Blog”})
  18. 18 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations (cont.) Log Output…
  19. 19 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  CRUD Operations λ  In summary: •  Default logging SUCKS! •  Importance of config file to find other artifacts of relevance •  External logs •  Default port: 27017 •  Authentication •  Turn to other artifacts within MongoDB to identify information around data •  User session IDs •  IP addresses/ports •  Timestamps within records •  Memory dump
  20. 20 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  User Sessions •  Upon connections, user sessions are assigned an ID # •  Use the conn# to track user activity •  Captures interactive user sessions •  Also captures API/script interactions
  21. 21 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  In-Row Data λ  By default, Mongo will add an '_id' field if not provided by data entry > db.blog.findOne() { "_id" : ObjectId("55879d3aeca5d08121fe1118”) ..snip.. } •  ID has a 12-byte structure: •  4-byte timestamp •  3-byte machine identifier •  2-byte process id •  3-byte random counter
  22. 22 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  Memory Dump •  We can also look to system memory dumps to carve MongoDB sessions/in-memory data •  Use what we know about data structure, commands, collections to build regex queries for memory strings •  Every entry has an “_id” field •  Mongo is stored in JSON •  CRUD commands have structure •  db.<collection>.insert({ •  db.<collection>.findOne() •  db.<collection>.remove({ •  etc… •  Dump memory strings, search for what we know
  23. 23 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  Memory Dump (cont.) $ strings mem.dump | grep '{ "_id"' Timestamps Machine ID Process ID
  24. 24 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  Memory Dump (cont.) $ strings mem.dump | grep ’db.testData' $ strings mem.dump | grep ’testData'
  25. 25 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. NoSQL Triage
  26. 26 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage •  Let’s say we encounter a MongoDB…what do we want to know? •  DB information •  Tables, collections, “schemas” •  Users •  Recent operations/changes •  Size of cluster (if > 1) •  Sources of data: •  Config file •  Other logs •  MongoDB itself! •  HTTP interface •  Mongo commands •  Javascript
  27. 27 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  MongoDB has a built-in web interface that is disabled by default (thankfully!), but may be turned on – look in the logs •  Good source of information
  28. 28 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  MongoDB has a range of built-in diagnostic commands that can be used to profile running instance •  mongostat – Provides quick overview of a running instance •  Can be used for remote servers
  29. 29 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  MongoDB has a range of built-in diagnostic commands that can be used to profile running instance •  mongotop – Provides time tracking of operations •  Great for DB/collection breakdown! •  Can be used for remote servers
  30. 30 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  MongoDB has a range of built-in diagnostic commands that can be used to profile running instance •  mongosniff – Basically tcpdump for MongoDB •  Not normally available in production environments; requires special build instructions •  However! Wireshark can also parse MongoDB traffic
  31. 31 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  PCAP: •  MongoDB Log:
  32. 32 © Mandiant, a FireEye Company. All rights reserved. NoSQL

    Triage •  NoSQL Triage (cont.) •  MongoDB also has a wide range of administration commands built-in •  JavaScript-function format $ mongo > db.serverStatus()
  33. 33 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  NoSQL Triage (cont.) •  We can also pass JavaScript commands directly through the terminal to pull back information •  For example: mongo --eval "printjson(db.serverStatus())" •  We can script it…we have the technology!
  34. 34 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  NoSQL Triage – Tool Time!
  35. 35 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  MongoDB Triage Script – v0.1
  36. 36 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  MongoDB Triage Script – v0.1
  37. 37 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  MongoDB Triage Script – v0.1 Example: serverstatus databases
  38. 38 © Mandiant, a FireEye Company. All rights reserved. Examining

    MongoDB Artifacts •  NoSQL Triage Scripts •  Free all the things! •  https://github.com/505Forensics/nosql_triage •  If this has any interest to you, I’d love feedback! •  Currently only MongoDB bash •  Elasticsearch, CouchDB, Hadoop, and Couchbase quickly on the way •  If you’ve encountered NoSQL, let me know how these can be modified to assist you
  39. 39 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. Conclusion
  40. 41 © Mandiant, a FireEye Company. All rights reserved. ©

    Mandiant, a FireEye Company. All rights reserved. THANK YOU!