Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINE Score: How to build Alternative Credit Sco...

LINE Score: How to build Alternative Credit Scoring Model

LINE DevDay 2019

November 21, 2019
Tweet

More Decks by LINE DevDay 2019

Other Decks in Technology

Transcript

  1. > Sangwoo Kim > Manager of Credit Scoring & Risk

    Management team > Responsible for developing LINE Score This session will benefit those who are interested in data analytics, especially credit score modeling Hello Developers and Data Scientists!
  2. > Developing a scoring model > Credit scoring system Credit

    scoring model Building a scoring model
  3. Agenda > Key Concept > LINE Score as an Alternative

    Credit Score > Overview of Credit Score > Techniques for Credit Score Modeling > Measurement methodology for performance > Achievements of LINE Score
  4. Predictive Model vs. Credit Scoring Model Predictive Model The general

    predictive model usually used in non-financial companies to achieve a specific purpose. > Mostly focusing on efficiency or effectiveness > Indirectly affects businesses or companies Credit Scoring Model The credit scoring model usually used in financial institutions or credit bureaus to evaluate a customer’s creditworthiness. > Mostly focusing on default > Influenced by various regulations > Directly affects the profit or losses of financial institutions
  5. Credit Score vs. Alternative Credit Score Credit Scoring Model The

    credit scoring model is a traditional evaluation method that usually uses financial data. > Uses financial data for scoring > Mainly used in countries with advanced financial industries > Only for those who have a history of financial transactions in the past Alternative Credit Scoring Model The alternative credit scoring model is an evaluation method that is in the limelight recently. It usually uses non-financial data. > Uses non-financial data for scoring > Available in all countries > Possible for those who don't have a history of financial transactions in the past
  6. Evaluate credit score by analyzing behavior data on the LINE

    platform LINE Score will only be calculated upon the user's consent
  7. An ecosystem in which data continuously flows into LINE scores

    can be utilized in various businesses at the same time Creating the LINE Score Platform Data source Financial Institutions Sharing Economy C2C Platforms Reservation Services Rental Businesses Score platform
  8. > A credit scoring model for evaluating customers who apply

    for loans Application Scoring System Behavior Scoring System > A credit scoring model for evaluating the behavior of existing loan customers Credit Scoring System > A statistical model that utilizes a customer's past history to predict creditworthiness in the future Key Terminology
  9. > Descriptive information used to predict defaults Feature or Variable

    (Input) ‘Build’ or ‘Develop’ credit scoring model > A process of developing a statistical model that predicts 'defaults' using 'features' Default (Target) > Mainly uses a loan delinquency history of more than 60 days or 90 days as target information Key Terminology
  10. Basic Concept of Credit Scoring Model How to Build a

    Credit Scoring Model Past default record (y) Past transaction history (x) Fitting the best algorithm to predict default y = f (x) How to Operate a Credit Scoring Model Funtion Loan applicant's future possibility of default Loan applicant's past transaction history Selected features (x) Probability of default (y)
  11. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition
  12. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Establishing the purpose of the model and selecting groups or subjects to be used for model development Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition
  13. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Selecting the target value that needs to be predicted
  14. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Creating and selecting features that are highly predictable and highly correlated with the target
  15. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Exploring the highly predictable and optimal algorithms suitable for the purpose of model development
  16. Process Overview The credit scoring model building process can be

    divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Converting to a score or grade form that is easy for customers to understand
  17. Several types of credit scoring models for different purposes >

    Behavior Scoring Model > Collection Scoring Model > Application Scoring Model Purpose of Models > Fraud Detection Model > Early Warning Model
  18. Set up a target that fits the model’s purpose Target

    Definition > Behavior Scoring Model > Collection Scoring Model > Application Scoring Model > Fraud Detection Model > Early Warning Model → Event of default → Event of default → Possibility of collection → Event of fraud → Event of issue related to credit risk
  19. Feature Creation and Selection Examples of candidate feature creation >

    Type of mobile phone > Number of received or sent calls in a day/week/month/year > The ratio between sending and receiving calls/messages > Main time zone
  20. Factors to consider when selecting features > Information value >

    Stability > Diversity > Correlation Feature Creation and Selection
  21. Traditionally, logistic regression has been widely used Recently, machine learning

    has also been used to increase predictability Model Fitting Logistic Regression A model usually can be derived in the form of a highly-explainable scorecard > Higher Interpretability > Simple > Relatively small amounts of data > Relatively low performance Machine Learning Algorithm A model usually can be derived in the form of a highly-complex and hard-to- interpret black box > Lower interpretability > Complicated > Requires a large amount of data > Relatively high performance
  22. Process for improving user utilization and understanding Scaling and Calibration

    Scaling Calibration Probability of Default Score 0.000% ~ 0.124% 1,000 ~ 901 0.125% ~ 0.249% 900 ~ 876 0.250% ~ 0.374% 875 ~ 851 0.375% ~ 0.499% 850 ~ 826 0.500% ~ 0.624% 825 ~ 801 0.625% ~ 0.749% 800 ~ 776 0.750% ~ 0.874% 775 ~ 751 … … Score Range Grade 820 ~ 1000 1 800 ~ 819 2 780 ~ 799 3 750 ~ 779 4 730 ~ 749 5 700 ~ 729 6 650 ~ 699 7 … …
  23. Measuring the performance of a model in terms of stability

    and discriminability How to Measure Model Performance Stability Indicator The difference in the distribution between the development data set and validation data set, which is used to verify the stability of the features, score, and grade > PSI Discriminative Power A numerical measure of how well rank ordering was done on predicting the target, which is used to show the discriminant power of the features, score, and grade > AUROC (AR) > K-S
  24. Discriminant Power Indicator: AUROC False Positive Rate True Positive Rate

    Perfect Model Developed Model Random Model Area Under the ROC = A+B A B
  25. Assume that there are 5 customers in default out of

    100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Random Model A
  26. Assume that there are 5 customers in default out of

    100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Developed Model Random Model A Perfect Model 5 10 20 25 30
  27. Assume that there are 5 customers in default out of

    100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Developed Model Area Under the ROC = A+B Random Model A Perfect Model 5 10 20 25 30 B
  28. Achievements of LINE Score Offer Save Increase many benefits based

    on LINE Score costs by providing more reasonable financial services accessibility to financial services