Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning - an Introduction for Ruby Develo...

Geoffrey Litt
September 13, 2016

Deep Learning - an Introduction for Ruby Developers

Deep learning is exploding in popularity these days, powering everything from photo tagging to self driving cars – and you don’t need to hire a data science team or learn C++ to use machine learning models in your application. We'll learn some modern deep learning techniques, targeted at Ruby developers with no prior experience in the field. We’ll also cover the various tools available in the Ruby machine learning ecosystem.

Geoffrey Litt

September 13, 2016
Tweet

More Decks by Geoffrey Litt

Other Decks in Programming

Transcript

  1. What we’ll cover today • Some fundamental intuition for how

    machine learning works • Recent developments in deep neural networks • Ruby tools for machine learning, including a new emerging project
  2. Apartment  size  (sq ft) Monthly rent  ($) 500 1000 650

    1200 800 1650 1000 2200 1050 2250 1200 2350 1300 2600
  3. 0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size
  4. 0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =    -­‐1 b      =    2500 Error = High
  5. 0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =    0 b      =    1750 Error = Lower
  6. 0 500 1000 1500 2000 2500 3000 400 600 800

    1000 1200 1400 Monthly  rent Apartment  size m    =      2.2 b =      -­‐200 Error = Low
  7. Image from http://cs231n.github.io/ 0.1    0.8    0.9    0.1

    0.0    0.2    0.8    0.1 0.0    0.4    0.3    0.0 0.1    0.5    0.1    0.0 P(cat) P(dog) Learning Weights Weight updates flow backwards through the network Error = High
  8. Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0

    0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights
  9. Image from http://cs231n.github.io/ 0.1    0.5    0.1    0.0

    0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
  10. Image from http://cs231n.github.io/ 0.0    0.3    0.5    0.1

    0.1    0.2    0.1    0.0 0.1    0.9    0.5    0.6 0.5    0.9    0.7    0.8 P(cat) P(dog) Learning Weights Error = Lower
  11. Image from http://cs231n.github.io/ P(cat) P(dog) Learning Weights Error = Low

    0.1    0.5    0.1    0.0 0.1    0.2    0.3    0.6 0.0    0.2    0.8    0.1 0.5    0.7    0.7    0.8
  12. “Deep Learning” = more layers = more data and compute

    needed to train = more parameters to train = things we have a ton of in 2016 neural nets with…
  13. Convolutional neural networks 0% 5% 10% 15% 20% 25% 30%

    2011 2012 2013 2014 2015 ImageNet Error  rate Human  benchmark AlexNet,  2012 GoogLeNet,  2014
  14. Recurrent neural networks Vinyals et  al  (2014).  Show  and  Tell:

     A  Neural   Image  Caption  Generator. CoRR,  abs/1411.4555,  .
  15. Deep Learning = Less Feature Engineering Early part of network

    learns features better than humans can create
  16. - + • Avoid new dependencies • Flexibility • Understand

    your solution • Lots of work • Testing/debugging • Performance
  17. - + • Quick results • No knowledge needed •

    Often basic algorithms • Only set use cases
  18. 0 20 40 60 80 100 120 140 160 180

    200 Number of machine learning libraries • scipy • numpy • NLTK • scikit-learn • keras • theano • caffe • tensorflow Source: https://github.com/josephmisiti/awesome-machine-learning
  19. 0 5000 10000 15000 20000 25000 30000 35000 fann theano

    caffe tensorflow Github Stars (9/5/16) ?
  20. C++ Core graph execution (C++) Python Ruby “The system includes

    front-ends for specifying TensorFlow computations in Python and C++, and we expect other front- ends to be added over time in response to the desires of both internal Google users and the broader open-source community.” SWIG
  21. Next steps… • Complete enough functionality to train basic neural

    networks • Gradient descent optimizer • Create easy install flow for dependencies • Add more test coverage • (Still far from production ready)
  22. To learn more… • Andrew Ng’s Machine Learning Coursera class

    • Google’s Udacity Tensorflow class • Contributors to tensorflow.rb welcome!: • github.com/somaticio/tensorflow.rb • I’ll tweet slides: @geoffreylitt