Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) - Budapest BI Forum, Budapest, Nov 2019

Transcript

1. Gradient Boosting Machines (GBM): From Zero to Hero (with R and Python Code) Szilard Pafka, PhD Chief Scientist, Epoch (USA) Budapest BI Forum Nov 2019
4. Disclaimer: I am not representing my employer (Epoch) in this talk I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
5. Source: Andrew Ng
6. Source: Andrew Ng
7. Source: Andrew Ng
13. Source: https://twitter.com/iamdevloper/
16. ...
23. y = f(x 1 ,x 2 ,...,x n ) “Learn” f from data
24. y = f(x 1 ,x 2 ,...,x n )
25. y = f(x 1 ,x 2 ,...,x n )
26. Supervised Learning Data: X (n obs, p features), y (labels) Regression, classification Train/learn/fit f from data (model) Score: for new x, get f(x) Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB… Goal: max acc/min err new data Metrics: MSE, AUC (ROC) Bad: measure on train set. Need: test set/cross-validation (CV) Hyperparameters, model capacity, overfitting Regularization Model selection Hyperparameter search (grid, random) Ensembles
27. Supervised Learning Data: X (n obs, p features), y (labels) Regression, classification Train/learn/fit f from data (model) Score: for new x, get f(x) Algos: LR, k-NN, DT, RF, GBM, NN/DL, SVM, NB… Goal: max acc/min err new data Metrics: MSE, AUC (ROC) Bad: measure on train set. Need: test set/cross-validation (CV) Hyperparameters, model capacity, overfitting Regularization Model selection Hyperparameter search (grid, random) Ensembles
30. Source: Hastie etal, ESL 2ed
31. Source: Hastie etal, ESL 2ed
51. no-one is using this crap
54. Live Demo Summary of the demo for those reading just the slides (e.g. those who did not attend the talk):
68. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
69. End of Demo