Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Predictive Modeling and Deep Learning

Predictive Modeling and Deep Learning

Entretiens du Nouveau Monde Industriel 2015, Paris.

Olivier Grisel

December 15, 2015
Tweet

More Decks by Olivier Grisel

Other Decks in Technology

Transcript

  1. Outline • Predictive Modeling & Artificial Intelligence • Deep Learning

    • Computer Vision • Natural Language Understanding and Machine Translation • Learning to Reason and Answer Questions
  2. type (category) # rooms (int) surface (float m2) public trans

    (boolean) Apartment 3 50 TRUE House 5 254 FALSE Duplex 4 68 TRUE Apartment 2 32 TRUE sold (float k€) 450 430 712 234 features target samples (train)
  3. type (category) # rooms (int) surface (float m2) public trans

    (boolean) Apartment 3 50 TRUE House 5 254 FALSE Duplex 4 68 TRUE Apartment 2 32 TRUE sold (float k€) 450 430 712 234 features target samples (train) Apartment 2 33 TRUE House 4 210 TRUE samples (test) ? ?
  4. Training text docs images sounds transactions Labels Machine Learning Algorithm

    Model Predictive Modeling Data Flow Feature vectors
  5. New text doc image sound transaction Model Expected Label Predictive

    Modeling Data Flow Feature vector Training text docs images sounds transactions Labels Machine Learning Algorithm Feature vectors
  6. Inventory forecasting & trends detection Predictive modeling examples Personalized radios

    Fraud detection Virality and readers engagement Predictive maintenance Personality matching
  7. Deep Learning • Neural Networks from the 90’s rebranded in

    2006+ • « Neuron » is a loose inspiration (not important) • Stacked architecture of modules that compute internal abstract representations from the data • Parameters are tuned from labeled examples
  8. Deep Learning in the 90’s • Yann LeCun invented Convolutional

    Networks • First NN successfully trained with many layers
  9. ImageNet Challenge 2012 • 1.2M images labeled with 1000 object

    categories • AlexNet from the deep learning team of U. of Toronto wins with 15% error rate vs 26% for the second (traditional CV pipeline)
  10. ImageNet Challenge 2013 • Clarifai ConvNet model wins at 11%

    error rate • Many other participants used ConvNets
  11. GoogLeNet vs Andrej • Andrej Karpathy evaluated human performance (himself):

    ~5% error rate • "It is clear that humans will soon only be able to outperform state of the art image classification models by use of significant effort, expertise, and time.” source: What I learned from competing against a ConvNet on ImageNet
  12. ImageNet Challenge 2015 • Microsoft Research Asia wins with networks

    with depths ranging from 34 to 152 layers • New record: 3.6% error rate
  13. Applications of RNNs • Natural Language Processing
 (e.g. Language Modeling,

    Sentiment Analysis) • Machine Translation
 (e.g. English to French) • Speech recognition: audio to text • Speech synthesis: text to audio • Biological sequence modeling (DNA, Proteins)
  14. Conclusion • ML and DL progress is fast paced •

    Many applications already in production (e.g. speech, image indexing, translation, face recognition) • Very promising results for QA and robot control • Machine Learning is now moving from pattern recognition to higher level reasoning