Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Experiences using ReactiveMongo to build a JSO...

Experiences using ReactiveMongo to build a JSON API with Play

After ReactiveMongo was released late last year, I spent 2 months rewriting an application in Scala to use it. It is as close to the MongoDB wire-protocol as possible but unlike the official drivers, it is fully asynchronous & non-blocking, using the powerful Akka actors underneath the hood.

This talk will compare some of the idiomatic differences between programming with different MongoDB drivers, architectural patterns and also gives examples of the performance implications of these decisions.

After the talk, the attendees should have a clear understanding of why the concepts behind ReactiveMongo are important and also be able to reason about how they structure their application.

Alexander Jarvis

April 09, 2013
Tweet

Other Decks in Programming

Transcript

  1. ReactiveMongo “Experiences using ReactiveMongo to build a JSON API with

    Play” - Alexander Jarvis London, 9th April 2013 PANAXIOM
  2. Overview • A brief history & context • What is

    ReactiveMongo? • Asynchronous & Non-blocking I/O • Rewrite, reduce. • Vanity benchmarks • Typical usage & patterns
  3. ๏ Real-time location sharing ๏ Currently in alpha ๏ API

    available http://meep.io ๏ Uses ReactiveMongo
  4. Common project goals • Provide a fast JSON API over

    HTTP • High throughput, low latency • Scalability curl https://meep.io/users/UR6pdkqrFEgAG_H2 -H 'Authorization: Bearer xd97JMba+a*yM51ZsBEUUT)_fgYt10BO' -H 'Accept: application/json' { "id":"UR6pdkqrFEgAG_H2", "email":"[email protected]", "firstName":"Charles", "lastName":"Babbage", "image":"UR6qFEqrFBsA3OL9", "updated":"2013-02-15T22:32:38.410+01:00" }
  5. • Web framework for Java and Scala • 1.0 public

    2009 (mostly Java, Scala module) • 2.0 rewritten in Scala at end of 2011 • Gaining momentum, support from Typesafe • Open Source (Apache 2) playframework.com
  6. Enough Playing around, more mongo • Transition from relational databases

    and ORMs to MongoDB and documents makes sense for a JSON API • With de-normalised data a JSON API is even closer to representations in the database Web Client Web Service MongoDB BSON JSON
  7. Finding your MongoDB Nirvana Official Java driver (with mongo-jackson-mapper) public

    boolean updateAccessToken() { ! return db().update(DBQuery.is("_id", new org.bson.types.ObjectId(id)), ! ! ! new BasicDBObject("$set", new BasicDBObject("accessToken", accessToken)), ! ! ! false, ! ! ! false, ! ! ! WriteConcern.SAFE).getLastError().ok(); }
  8. Finding your MongoDB Nirvana Official Java driver (with mongo-jackson-mapper) public

    boolean updateAccessToken() { ! return db().update(DBQuery.is("_id", new org.bson.types.ObjectId(id)), ! ! ! new BasicDBObject("$set", new BasicDBObject("accessToken", accessToken)), ! ! ! false, ! ! ! false, ! ! ! WriteConcern.SAFE).getLastError().ok(); } public boolean updateAccessToken() { ! return users().update("{_id: #}", this._id).concern(WriteConcern.SAFE) ! ! ! .with("{$set: {accessToken: #}}", this.accessToken).getLastError().ok(); } Jongo after 1 month
  9. ReactiveMongo • Asynchronous & Non-Blocking Scala Driver for MongoDB •

    Written by Stephane Godbillion at Zenexity (original company behind Play) • Released 30th August 2012 • Open Source (Apache 2) reactivemongo.org
  10. Motivation • Classic synchronous DB drivers block threads until response

    is received • Threads can only scale up to a limit • Context switches are expensive • Made specifically for Play • The future is streaming data • Related to the “The C10K problem”
  11. Non-blocking I/O The fast food restaurant analogy: non-blocking: “When you

    place an order at the cashier, it’s sent off for someone else to fulfil and you step aside for the next customer to be served.” blocking: “When you place an order at the cashier, you wait for your order and no other customers can be served...”
  12. Scala Futures • A composable container for the result of

    a computation that may not exist yet. • ReactiveMongo’s basic Cursor functions return Futures: def headOption(): Future[Option[T]] def toList(upTo: Int): Future[List[T]] def toList(): Future[List[T]] (simplified)
  13. Rewrite, reduce • Strategic, long-term decision • Play’s Java API

    didn’t feel as fluid as 1.0 24th Sep -> 7th Nov 2012 (1 month and 14 days) 0 525 1,050 1,575 2,100 Lines of code 1,199 2,028 Java Scala
  14. Vanity benchmarks Downloading a GridFS file 0 375 750 1125

    1500 244 Kb file 1464.11 966.68 Requests / sec (mean) Official Reactive Apache Bench ab -n 5000 -c 10
  15. Vanity benchmarks Downloading a GridFS file 0 375 750 1125

    1500 244 Kb file 1464.11 966.68 Requests / sec (mean) Official Reactive 0 23 45 68 90 6.2 Mb file 86.12 57.12 Requests / sec (mean) Official Reactive Apache Bench ab -n 5000 -c 10
  16. Vanity benchmarks 33% increase in requests / second Official Java

    Driver ReactiveMongo • Noticed fans spinning when using the Java Driver • ReactiveMongo handles resources better
  17. Vanity benchmarks 33% increase in requests / second Official Java

    Driver ReactiveMongo • Noticed fans spinning when using the Java Driver • ReactiveMongo handles resources better
  18. Future is a Monad • Allows you to chain results

    together • map, flatMap, filter • flatMap will flatten the result e.g. two nested maps create a Future of a Future. Future[Future[Result]] Future[Result] flatMap
  19. Model • Manually define the reader/writer • Scala macros will

    reduce this to one line • Similar to Json reads writes combinators
  20. 1. Monolithic Controller • What we have seen thus far

    • Mongo collection as val in Controller • Choice of JSON to BSON implicit conversion • Will get messy when mixing more models and controllers
  21. 3. JSON Coast-to-Coast • Validate and transform JSON without static

    case class models • Data flow • JSON to BSON implicit conversions • Business logic needs to speak JSON github.com/mandubian/play2-json-demo example at
  22. Enumerators / Iteratees • Streaming data • Immutable Producer /

    Consumer Potentially huge list in memory Streamed from Mongo
  23. References 1. Play framework http://playframework.com/ 2. Mongo Jackson Mapper http://mongojack.org/

    3. Jongo http://jongo.org/ 4. ReactiveMongo http://reactivemongo.org/ 5. The C10K problem http://www.kegel.com/c10k.html 6. Scala Futures http://docs.scala-lang.org/overviews/core/futures.html 7. Scala Futures (akka) http://doc.akka.io/docs/akka/2.1.1/scala/futures.html 8. Reactive-json-demo https://github.com/alexanderjarvis/reactivemongo-json-demo 9. Linky https://github.com/alexanderjarvis/linky 10. JSON Coast-to-Coast http://mandubian.com/2013/01/13/JSON-Coast-to-Coast/