Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Reactive Programming with RxJava for Efficient...

Reactive Programming with RxJava for Efficient Data Access

This talk was given by Ben Christensen (https://twitter.com/benjchristensen) from Netflix and me at Couchbase Connect 2014 about RxJava, data access patterns and failure handling in reactive systems.

Video will be available through the Couchbase Resources channel

Michael Nitschinger

October 06, 2014
Tweet

More Decks by Michael Nitschinger

Other Decks in Programming

Transcript

  1. Reactive Programming with RxJava for Efficient Data Access Ben Christensen

    | Software Engineer, Netflix Michael Nitschinger | Software Engineer, Couchbase
  2. Reactive Extensions for Async Programming RxJava http://github.com/ReactiveX/RxJava http://reactivex.io Single Multiple

    Sync T getData() Iterable<T> getData() Async Future<T> getData() Observable<T> getData()
  3. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.create(subscriber -> { subscriber.onNext("Hello world!"); subscriber.onCompleted(); }).forEach(System.out::println); Synchronous single value
  4. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.create(subscriber -> { subscriber.onNext("Hello"); subscriber.onNext("world"); subscriber.onNext("!"); subscriber.onCompleted(); }).forEach(System.out::println); Synchronous multi-value
  5. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.create(subscriber -> { try { subscriber.onNext(doSomething()); subscriber.onCompleted(); } catch (Throwable e) { subscriber.onError(e); } }).subscribeOn(Schedulers.io()) .forEach(System.out::println); Asynchronous single value with error handling
  6. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.create(subscriber -> { int i = 0; while (!subscriber.isUnsubscribed()) { subscriber.onNext(i++); } }).take(10).forEach(System.out::println); Synchronous multi-value with unsubscribe
  7. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.create(subscriber -> { AtomicInteger i = new AtomicInteger(); AtomicLong requested = new AtomicLong(); subscriber.setProducer(r -> { if (requested.getAndAdd(r) == 0) { do { if (subscriber.isUnsubscribed()) { break; } subscriber.onNext(i.incrementAndGet()); } while (requested.decrementAndGet() > 0); } }); }).observeOn(Schedulers.newThread()) .take(10).forEach(System.out::println); Synchronous multi-valued source that thread-hops with “reactive pull” backpressure
  8. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable.from(iterable) .observeOn(Schedulers.newThread()) .take(10).forEach(System.out::println); Synchronous multi-valued source that thread-hops with “reactive pull” backpressure
  9. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  10. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  11. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  12.  Observable<R>  b  =  Observable<T>.flatMap({  T  t  -­‐>      

         Observable<R>  r  =  ...  transform  t  ...          return  r;    })   flatMap!
  13.  Observable<R>  b  =  Observable<T>.flatMap({  T  t  -­‐>      

         Observable<R>  r  =  ...  transform  t  ...          return  r;    })   flatMap!
  14. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  15. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  16. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  17. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  18. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  19. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  20. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  21. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  22. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  23.        Observable.zip(a,  b,  {  a,  b,  -­‐>  

                 ...  operate  on  values  from  both  a  &  b  ...              return  [a,  b];  //  i.e.  return  tuple          })  
  24. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  25. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  26. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  27. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  28. class  VideoService  {        def  VideoList  getPersonalizedListOfMovies(userId);  

         def  VideoBookmark  getBookmark(userId,  videoId);        def  VideoRating  getRating(userId,  videoId);        def  VideoMetadata  getMetadata(videoId);   }   class  VideoService  {        def  Observable<VideoList>  getPersonalizedListOfMovies(userId);        def  Observable<VideoBookmark>  getBookmark(userId,  videoId);        def  Observable<VideoRating>  getRating(userId,  videoId);        def  Observable<VideoMetadata>  getMetadata(videoId);   }   ... create an observable api: instead of a blocking api ...
  29. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); } 1 2 3 4 5 6
  30. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); } 1 2 3 4 5 6
  31. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); }
  32. public Observable<Void> handle(HttpServerRequest<ByteBuf> request, HttpServerResponse<ByteBuf> response) { // first request

    User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }).flatMap(data -> { // output as SSE as we get back the data (no waiting until all is done) return response.writeAndFlush(new ServerSentEvent(SimpleJson.mapToJson(data))); }); } 1 2 3 4 6
  33. // first request User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> {

    // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }) Decouples Consumption from Production
  34. // first request User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> {

    // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }) Decouples Consumption from Production
  35. // first request User object return new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> {

    // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }) Decouples Consumption from Production
  36. Decouples Consumption from Production // first request User object return

    new UserCommand(request.getQueryParameters().get("userId")).observe().flatMap(user -> { // then fetch personal catalog Observable<Map<String, Object>> catalog = new PersonalizedCatalogCommand(user).observe() .flatMap(catalogList -> { return catalogList.videos().<Map<String, Object>> flatMap(video -> { Observable<Bookmark> bookmark = new BookmarkCommand(video).observe(); Observable<Rating> rating = new RatingsCommand(video).observe(); Observable<VideoMetadata> metadata = new VideoMetadataCommand(video).observe(); return Observable.zip(bookmark, rating, metadata, (b, r, m) -> { return combineVideoData(video, b, r, m); }); }); }); // and fetch social data in parallel Observable<Map<String, Object>> social = new SocialCommand(user).observe().map(s -> { return s.getDataAsMap(); }); // merge the results return Observable.merge(catalog, social); }) 1 2 3 4 5
  37. class  VideoService  {        def  Observable<VideoList>  getPersonalizedListOfMovies(userId);  

         def  Observable<VideoBookmark>  getBookmark(userId,  videoId);        def  Observable<VideoRating>  getRating(userId,  videoId);        def  Observable<VideoMetadata>  getMetadata(videoId);   }   Clear API Communicates Potential Cost
  38. Implementation Can Differ class  VideoService  {        def

     Observable<VideoList>  getPersonalizedListOfMovies(userId);        def  Observable<VideoBookmark>  getBookmark(userId,  videoId);        def  Observable<VideoRating>  getRating(userId,  videoId);        def  Observable<VideoMetadata>  getMetadata(videoId);   }   BIO Network Call Local Cache Collapsed Network Call
  39. class  VideoService  {        def  Observable<VideoList>  getPersonalizedListOfMovies(userId);  

         def  Observable<VideoBookmark>  getBookmark(userId,  videoId);        def  Observable<VideoRating>  getRating(userId,  videoId);        def  Observable<VideoMetadata>  getMetadata(videoId);   }   Collapsed Network Call Collapsed Network Call BIO NIO Network Call Local Cache Implementation Can Differ and Change
  40. Java SDK 2.0 ©2014 Couchbase, Inc. — Proprietary and Confidential

    56 It is a complete rewrite compared to 1.* and provides asynchronous & synchronous document oriented APIs.
  41. Couchbase Core IO ©2014 Couchbase, Inc. — Proprietary and Confidential

    57 §  Common infrastructure & feature set for all language bindings §  Message oriented §  Asynchronous only §  Low overhead and performance focused §  Supports Java 6+ (including 8!) §  Disruptor RingBuffer for implicit batching and backpressure §  Netty for high performance IO
  42. Reacting to Failure ©2014 Couchbase, Inc. — Proprietary and Confidential

    67 §  Resiliency is key §  Things will go wrong, so better plan for it §  Do not aim for QA, aim for production §  Do not trust integration points §  Treat the Database (SDK) as an integration point The more you sweat in peace, the less you bleed in war.
  43. Useful Reading ©2014 Couchbase, Inc. — Proprietary and Confidential 68

    §  Release It! by Michael T. Nygard §  Stability Patterns & Antipatterns §  Capacity planning §  Operations
  44. Timeouts ©2014 Couchbase, Inc. — Proprietary and Confidential 69 § 

    The network is unreliable §  Servers fail §  The SDK contains bugs §  Always specify timeouts and deal with them! §  The synchronous wrapper defines them for you all the time.
  45. Coordinated Retry ©2014 Couchbase, Inc. — Proprietary and Confidential 73

    Fail fast §  Don’t let your system get stuck §  Fail and backpressure Retry §  immediately won’t help §  either linear or exponential backoff necessary §  have a strategy if retry also doesn’t work (Fallbacks!)
  46. More Patterns ©2014 Couchbase, Inc. — Proprietary and Confidential 76

    Circuit Breaker §  Closed if everything okay §  Opens once the integration point breaks §  Fail fast if open §  Rechecking with half-open Bulkheads §  Isolate failure where possible §  Don’t let your ship sink!