Confusion matrix

Confusion matrix 기초부터 머신러닝 응용까지 for dataitgirls3

Transcript

1. Evaluation for classification dataitgirls3 Instructor Sunmi Yoon
2. Confusion Matrix
3. https://sumniya.tistory.com/26
4. Evaluation Metrics from Confusion Matrix
5. https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62
6. Precision(੿޻ب), PPV(Positive Predictive Value) ݽ؛੉ TrueۄҊ ࠙ܨೠ Ѫ ઺ী, पઁ Trueੋ Ѫ੄ ࠺ਯ Recall(੤അਯ), Sensitivity, hit rate पઁ True ઺ী ݽ؛੉ True۽ ࠙ܨೠ ࠺ਯ “Precision݅ न҃ਸ ॳݶ ݽ؛੉ ੋ࢝೧૑Ҋ, Recall݅ न҃ॳݶ ݽ؛੉ ೻ಌ૓׮” ੄ ੄޷ܳ ࢤп೧ࠁࣁਃ.
7. Accuracy TP, TNਸ ݽف Ҋ۰ೞח ૑಴. Label ࠛӐഋ੉ बೡ ٸী ࢎਊਸ ઱੄೧ঠ ೤פ׮. F1 Score Precisionҗ Recall੄ ઑചಣӐ Label ࠛӐഋ੉ बೡ ٸী ݽ؛੄ ࢿמਸ ੿ഛೞѱ ಣоೡ ࣻ ੓णפ׮. Label ࠛӐഋ੉ बೡ ٸী, Accuracyח ૑಴۽ࢲ न܉ࢿਸ ੏णפ׮. ੉ਬܳ ࢤп ೧ ࠁࣁਃ.
8. https://sumniya.tistory.com/26 ৵ ࢑ࣿಣӐ੉ ইפҊ ઑചಣӐੋо?
9. ઑӘ݅ ؊ о ࠇद׮
10. https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ׮द ੉ Ӓܿਵ۽ جই৬ࢲ, ଘ ফܳ ઺बਵ۽ ࢤп೮਺
11. https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ׮द ੉ Ӓܿਵ۽ جই৬ࢲ, ੉ߣূ ফب э੉ ࢤпೞݶࢲ ࠇद׮ (૑Әࠗఠ ഁтܾ ࣻ ੓਺)
12. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN
13. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Precision Positive Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ ઺बਵ۽
14. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Negative Predictive Value ࠙ܨ Ѿҗ(ݽ؛)ਸ ઺बਵ۽
15. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN Recall Sensitivity True Positive Rate ੿׹ਸ ઺बਵ۽
16. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ False Positive Rate
17. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ Specificity True Negative Rate
18. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ੿׹ਸ ઺बਵ۽ Fall-out rate False Positive Rate
19. https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 ੉Ѧ ೞ੗Ҋ ೮঻ભ. ੉ߣূ ফب э੉ ࢤпೞݶࢲ ࠇद׮ (૑Әࠗఠ ഁтܾ ࣻ ੓਺)
20. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN ? TP ઺ब ૑಴ٜ ੿ܻೞ੗ݶ, ?
21. ੿׹ TRUE FALSE ࠙ܨѾҗ TRUE TP FP FALSE FN TN TN ઺ब ૑಴ٜ ? ੿ܻೞ੗ݶ, ?
22. ഁтܻભ? ਗې Ӓ۠Ѣ৘ਃ
23. ӝୡח ೮ਵפө ઑӘ݅ ؊ ೧ ࠇद׮.
24. Confusion Matrix with Histogram
25. https://www.medcalc.org/manual/roc-curves.php Criterion, Threshold য়ܲଃ Distribution਷ Actual True, ৽ଃ਷ Actual False. Threshold ਤ۽ח ݽف True۽ ৘ஏೞח ݽ؛੉ ੓׮Ҋ о੿೮ਸ ٸ,
26. https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױ੸ਵ۽ ਋ஏ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? Precision: Recall: Specificity: Fall-out:
27. https://www.medcalc.org/manual/roc-curves.php Thresholdܳ ӓױ੸ਵ۽ ਋ஏ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? True positive rate: True negative rate:
28. https://www.medcalc.org/manual/roc-curves.php ੉ߣূ ߈؀۽ ੉ز दெࠇद׮. যڃ ੌ੉ ੌযաաਃ? True positive rate: True negative rate:
29. Specificity৬ Sensitivity੄ ҙ҅ https://www.medcalc.org/manual/roc-curves.php
30. ROC(Receiver Operating Characteristic) curve
31. рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php AUC (Area Under Curve)
32. рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php Actual True৬ Actual False distribution੉ ৮߷ೞѱ эਸ ٸ (feature੄ class ߸߹מ۱ হ਺) ROC curveח 45ب пب ૒ࢶ
33. рױೞѱח, Sensitivity৬ 1-Specificityܳ п ୷ਵ۽ ೞח 2ରਗ Ӓې೐ https://www.medcalc.org/manual/roc-curves.php Actual True৬ Actual False distribution੉ Ҁ஖ח ৔৉ হ੉ ৮߷ೞѱ ܻ࠙ ؼ ٸ ROC ழ࠳ (feature੄ class ߸߹ מ۱੉ ৮߷) ROC ழ࠳о ઝ࢚ױী оө਎ࣻ۾ feature੄ class ߸߹ מ۱੉ જ׮Ҋ ೡ ࣻ ੓׮.
34. ROC(Receiver Operating Characteristic) curve with Machine Learning
35. Classifierܳ ݅ٚ׮ח Ѥ, ف ѐ੄ histogramਸ ӒܻҊ Thresholdܳ ੿੄ೞח Ѫ https://www.medcalc.org/manual/roc-curves.php
36. https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py Histogramਸ Ӓ۷׮ח Ѥ ROC ழ࠳ܳ Ӓܾ ࣻ ੓׮ח Ѫ!
37. https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html#sphx-glr-auto-examples-model-selection-plot-roc-py ROC ழ࠳ܳ Ӓܾ ࣻ ੓׮ח Ѥ ৈ۞ ROC ழ࠳ р ࠺Үܳ ా೧ જ਷ ࢿמ੄ ݽ؛ਸ ଺ইյ ࣻ ੓׮ח Ѫ!
38. AUCо ௼׮ = ݽ؛੉ ҅࢑ೠ probabilityܳ ߄ఔਵ۽ Ӓܽ histogramٜ੉ ੜ ܻ࠙غয ੓׮. = ݽ؛੉ Threshold(Decision BoundaryۄҊب ೠ׮)ী ؏ ޹хೞ׮. = উ੿੸ੋ ৘ஏਸ ೠ׮.
39. ݽ؛ ࢶఖী ROC ழ࠳ܳ ഝਊೠ׮ = Decision Boundaryী ࢚ҙহ੉ ؊ જ਷ ݽ؛ਸ ଺ח׮. = ganziо դ׮.
40. ૒੽ Ӓ۰ࠇद׮. ؘ੉ఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression - sklearn.linear_model.LogisticRegression - sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want ౲ - Tree ҅ৌ੄ ݽ؛੄ ҃਋ model੄ predict_proba() ݫࣗ٘ܳ ࢎਊೞݶ ഛܫ ҅࢑੉ ؾ פ׮. - ୊਺ীח Thresholdܳ a ݅ఀ ੉ز೧оݴ Sensitivity, Specificityܳ ҅࢑೧ ઝ಴ܳ ҳೞ ࣁਃ. - যڌѱ ೞݶ Thresholdܳ ੜ ੉زदఃݶࢲ ROC ઝ಴ܳ ନਸ ࣻ ੓ਸөਃ? - ઝ಴ٜਸ ಣݶ࢚ী ନযࠁࣁਃ.
41. sklearn.metrics.roc_curve ܳ ഝਊ ೧ ࠇद׮. ؘ੉ఠ: titanic ݽ؛ - sklearn.linear_model.LinearRegression - sklearn.linear_model.LogisticRegression - sklearn.tree.DecisionTreeClassifier - sklearn.ensemble.RandomForestClassifier ١ whatever you want ؊ աইоࢲ, - sklearnਸ ੉ਊ೧ AUCب ҅࢑ ೧ࠇद׮. - ৈ۞ ݽ؛ٜ੄ ࢿמਸ ࠺Ү ೧ ࠇद׮. - DecisionTreeClassifierܳ ࢎਊ೮؊ۄب, ࢎਊೠ featureо ׮ܰ׮ݶ ӒѤ ׮ܲ ݽ؛ੑפ ׮. - ఋ੉ఋץ ݈Ҋ, ׮ܲ classification ޙઁীب ഝਊ೧ ࠁࣁਃ.
42. ՘