Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bayesian Inference & Neural Networks

Lukasz
March 01, 2017

Bayesian Inference & Neural Networks

Introduction to Bayesian Inference, Hierarchical Models & Bayesian Neural Networks.

Lukasz

March 01, 2017
Tweet

More Decks by Lukasz

Other Decks in Science

Transcript

  1. Bayesian Inference & Neural Networks Lukasz Krawczyk, 1st March 2017

    Homo Aprioriu s Homo Pragmatic us Homo Friquentist us Homo Sapiens Homo Bayesia nis
  2. 2017/07/03 2 / 26 Agenda • About me • The

    Problem • Bayesian Inference • Hierarchical Models • Bayesian Inference & Neural Networks
  3. 2017/07/03 3 / 26 About me • Data Scientist at

    Asurion Japan Holdings • Previous: Data Scientist at Abeja Inc. • MSc degree from Jagiellonian University, Poland • Contributor to several ML libraries
  4. 2017/07/03 5 / 26 „Missing Uncertainty“ Making a confident error

    is the worst thing we can do With DL models we generally only have point estimates of parameters and predictions Hard to make decisions when we‘re not able to tell whether a DL model is certain about its output or not Trust and adoption of DL is still low
  5. 2017/07/03 7 / 26 Bayesian Inference Inference Posterior Data Credibile

    Region Uncertainity Better Insights Prior μ σ Model Assumptions about data controlled by the prior
  6. 2017/07/03 8 / 26 Bayes formula P(θ true ∣D)= P(D∣θ

    true ) P(θ true ) P(D) • P(θ true | D): The posterior • the probability of the model parameters given the data: this is the result we want to compute. • P(D | θ true ): The likelihood • proportional to the likelihood estimation in the frequentist approach. • P(θ true ): The model prior • encodes what we knew about the model prior to the application of the data D. • P(D): The data probability • which in practice amounts to simply a normalization term.
  7. 2017/07/03 9 / 26 Bayesian Inference Bayesian Inference • General

    purpose framework • Generative models • Clarity of FS + Power of ML – White-box modelling – Black-box fitting (NUTS, ADVI) – Uncertainity → Intuitive insights • Learning from very small datasets • Probabilistic Programming
  8. 2017/07/03 10 / 26 Bayesian Inference • Bayesian Optimization (GP)

    • Hierarchical models (badass models) • Bonus points – Robust in high dimensions – Minibatches – Knowledge transfer Bayesian Inference
  9. 2017/07/03 13 / 26 Hierarchical Models – parameter pooling Pooled

    Unpooled Partial-pooling More accurate fitting Not enough data Generalization Small datasets Missing variations among groups
  10. 2017/07/03 14 / 26 Example – call duration model •

    Each advisor has his/her own distribution • Overall Call Center distribution is controlled by hyper parameter }
  11. 2017/07/03 15 / 26 Hierarchical Models - benefits • Modelling

    is very easy and intuitive • Natural hierarchical structure of observational data • Variation among individual groups • Knowledge transfer between groups
  12. 2017/07/03 18 / 26 Example – standard NN x 1

    x 2 y 0.1 1.0 0 0.1 -1.3 1 … 2 hidden layers sigmoid tanh Data Backpropagation
  13. 2017/07/03 19 / 26 Example – NN with Bayesian Backpropagation

    π n=2 Bayesian Backpropagation 2 hidden layers Data x 1 x 2 y 0.1 1.0 [0,1,...] 0.1 -1.3 [1,1,...] …
  14. 2017/07/03 21 / 26 Synergy – going deeper M S

    ~ G ~ Bayesian Hierarchical Model μ σ Weight regularization similar to L2
  15. 2017/07/03 22 / 26 Synergy – going deeper M S

    ~ G ~ Bayesian Hierarchical Model μ σ Regularization Weight regularization similar to L2
  16. 2017/07/03 25 / 26 Why is this important? Scientific perspective

    • NN models with small datasets • Complex hierarchical neural networks (Bayesian CNN) • Minibatches • Knowledge transfer Business perspective • Clear and intuitive models • Uncertainity in Finance & Insurance is extremely important • Better trust and adoption of Neural Network-based models