不動産webサービスを強くする機械学習の使い方

2016/07/28（木） 19:30〜
【不動産テック勉強会#1】人工知能時代に備えて不動産関連データについて色々語らう勉強会
※画像や物件名などについてはマスクしております。ご了承下さい。

Transcript

1. ෆಈ࢈XFCαʔϏεΛڧ͘͢Δ
   ػցֶशͷ࢖͍ํ ෆಈ࢈ςοΫษڧձ
   5XJUUFS
   !IJEEZZ
   
2. ͓໿ଋ
   ͜ͷ-5͸εϐʔΧʔݸਓͷ
   ݟղͰ͋Γɺॴଐ͢ΔاۀஂମΛ୅ද͢Δ΋ ͷͰ͸͋Γ·ͤΜɻ
   Disclaimer This LT expresses the viewpoints of ME and is not reviewed for correctness or accuracy by my company.
3. ୭ʁ
4. *%!IJEEZZ
   ৬ۀɿ1.ʢϓϩμΫτͷ΄͏ʣ
   ͓࢓ࣄɿΨνίʔσΟϯάҎ֎
   ಛٕɿXFCαʔϏεͮ͘Γ
   
   ɹ
   
   
   ͦΕඞཁʁͱݴ͍์ͭ
   ݴޠɿ3
   MFWFM
   
   ͪΐͬ͜ͱSVCZ
   
   SBJMT
   ઐ߈ɿܭྔܦࡁֶʢ541ʣ
5. ͜Ε·Ͱ࡞ͬͨܞΘͬͨαʔϏε ohmy!Ո௞
6. ௞ି෺݅৘ใαΠτΛ΍͍ͬͯ·͢
7. σʔλ෼ੳɺϏδϡΞϥΠθʔγϣϯʹΑΓɺ Ϣʔβʔ͕෺݅બͼΛ͠΍͘͢͢Δ
8. ࠓ೔͸-3௞ିͰͷ
   ػցֶशͷ
   ࢖͍ํΛ
   ͝঺հ͍ͨ͠
9. ڪΔ΂͖෺͕݅ଘࡏ͢Δ
14. ͜ͷ΁Μ͸Θ͔Γ΍͍͢
15. ͜ͷ΁Μ͸Θ͔Γ΍͍͢
    ೖྗϛε
17. ͜Ε͸ϗϯϞϊ
19. ͜Ε͸χηϞϊ
20. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ
    ͍Ζ͍Ζؔ࿈͍ͯ͠Δ
21. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ
    ͍Ζ͍Ζؔ࿈͍ͯ͠Δ
    ͸͘͡ͷ͕ΊΜͲ͍͘͞
22. Ո௞΍ؒऔΓɺ޿͞ͳͲͱ
    ͍Ζ͍Ζؔ࿈͍ͯ͠Δ
    ͸͘͡ͷ͕ΊΜͲ͍͘͞
    Ϟσϧͱ͔ߟ͑ͨ͘ͳ͍
23. 0OFDMBTT
    47.
    
    TWN͞Μ͕ద౰ʹ֎Ε஋൑ఆͯ͘͠ΕΔ
    
    ڭࢣσʔλΛ༻ҙ͠ͳͯ͘ΠΠ
    
    Βͪ͘Μ
24. One-class svm library(kernlab) # make data to one-class svm model DF <- data.frame(DF, class=1) outlier.svm <- ksvm(x=class ~bukken_shubetsu +struct +struct_all +madori +history_TOTAL +log_price_with_kanrihi, data=DF, type="one-svc",C=1000,scaled=TRUE,nu=0.01, kernel="rbfdot") #judge outlier DF$outlier <- predict(outlier.svm, DF, type = "response") ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ங೥਺ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ
27. Կ΋ߟ͑ͣ
    ߦ͘Β͍Ͱ
    ֎Ε஋͸͚ͨ͡ʂ
    0 ʾ˜ʽ 0
    ƂŖŘ̇
28. ʢ໨తʣ
    ೖྗϛεσʔλͷ࡟ݮ
    ʢख๏ʣ
    "OPNBMZ
    %FUFDUJPO
    ʢํ๏ʣ
    0OFDMBTT
    47.
29. ͞ΒͳΔ໰୊
31. ࢛৞൒ɺ෩࿊ແɺτΠϨڞಉɺτΩϫ૳ͷΑ͏ͳ෺݅
32. χʔζ͸
    ͋Δ͔΋͠Εͳ͍
33. χʔζ͸
    ͋Δ͔΋͠Εͳ͍
    ʢ͕ͩʣ
34. ͋·Γʹଟ͘
    ϦετϖʔδʹͰΔͱ
    ݟͨ໨͕ѱ͍ɻɻ
35. ΑΖ͍͠ɺ
    ͳΒ͹ɺ
    ൑ఆثΛͭ͘Ζ͏
36.  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू ਓྗͰ໨ݟʂ
    ʢϋʔτΛڧ͘΋ͭ͜ͱʣ
    ˞෺݅ࣗମ͸ѱ͋͘Γ·ͤΜ
37.  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू
     ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़ ͜͜͸Կ΋ߟ͑ͳͯ͘΋ΠΠ
38. Boro detection svm library(kernlab) boro.svm <- ksvm( boro~bukken_shubetsu +struct +struct_all +madori_num +madori_type_num +history_TOTAL +log_price_with_kanrihi, data=DF, C=1.584893, scaled=TRUE,nu=0.01, kernel="rbfdot",kpar=list(sigma=10),cross=13)  ಛघ෺݅ΛਓྗͰ݅ఔ౓ऩू
     ಛघ෺݅Ͱ͸ͳ͍΋ͷΛ݅நग़
     47.ʹͯɺʮಛघ෺݅൑ఆثʯΛੜ੒ ෺݅छผ ݐஙλΠϓ ઐ༗໘ੵ ؒऔΓ ؒऔΓλΠϓ ؅ཧඅࠐΈՈ௞ʢର਺Խʣ ங೥਺ http://d.hatena.ne.jp/sleepy_yoshi/20120624/p1 Λνϡʔχϯάͷࢀߟʹ͠·ͨ͠
40. ಛघͳ෺͕݅
    Ϧετ্Ґʹग़ͳͬͨ͘ʂ
    0 ʾ˜ʽ 0
    ƂŖŘ̇
41. ʢ໨తʣ
    Ϧετϖʔδͷ69վળ
    ʢख๏ʣ
    ̎஋෼ྨث
    ʢํ๏ʣ
    47.
42. ਺ࣈ΋͍͋͛ͨͰ͢
    ઌੜɻ
43. $73͕͋Βͳ͍͔ͳʔ
    ʢ͕͋Βͳ͍ʣ
44. ͦ͏͔ɺ
    $7ʢ໰߹ͤʣ͞ΕΔ ͨ
    ෺݅ʹ͸ͳʹ͔ಛ௃͕͋Δ ͷͰ͸ʁ
45. 8FCϚʔέతΞϓϩʔνͩͱɺ ϢʔβʔηάϝϯτΛ͖Γɺ
    ରԠ͢Δ෺݅Λஸೡʹਫ਼౓ߴ͘
    Ϩίϝϯυ͢Δͱ͜Ζ͕ͩɺɺɺ
46. ࡶʹ
    σʔλυϦϒϯͰ
    $73վળ͍ͤͨ͞
47. σʔλαΠΤϯςΟετ
    Ϩϕϧද
    IUUQEIBUFOBOFKQTIBLF[P
48. Ϩϕϧ
    Ϩϕϧͷਓୡ͸ूܭ෼ੳʹՃ͑ͯɺ࠷௿ݶͷػցֶश ΍౷ܭֶͷख๏Λ஌͍ͬͯ·͢ɻ47.΍ϥϯμϜϑΥϨ ετͳͲͷϝδϟʔͳख๏Λ֮͑ɺσʔλ෼ੳ͕໘ന͘ ͳͬͯ͘ΔࠒͰ͢ɻ͔͠͠ͳ͕Β3΍4144ͳͲͷઐ༻ ιϑτΛৗʹσϑΥϧτઃఆͷύϥϝʔλͰ෼ੳ͍ͯ͠ ͨΓɺಛ௃ྔબ୒΍લॲཧͷॏཁੑΛ؁͘ݟΔ܏޲͕͋ Γ·͢ɻ
    ͍ۙ͏ͪʹݱ࣮ͷσʔλ͸JSJTͷΑ͏ʹ؁͘ͳ͍͜ͱΛ ஌Δ͜ͱʹͳΔͰ͠ΐ͏ɻ
    
 ͋Γ͕ͪͳൃݴ
    ʮϥϯμϜϑΥϨετ࠷ڧʯ
49. ͋Γ͕ͪͳൃݴ
    ʮϥϯμϜϑΥϨετ࠷ڧʯ
50. ͋Γ͕ͪͳൃݴ
    ʮϥϯμϜϑΥϨετ࠷ڧʯ
51. ϥϯμϜϑΥϨετͰ
    $7͞Ε΍͍͢෺݅Λ༧ଌ
52. ϥϯμϜϑΥϨετͰ
    $7͞Ε΍͍͢෺݅Λ༧ଌ http://nakhirot.hatenablog.com/entry/20130704/1372874761 ΑΓൈਮ
53. ϥϯμϜϑΥϨετͰ
    $7͞Ε΍͍͢෺݅Λ༧ଌ CVͨ͠ ෺݅σʔλ CV͠ͳ͔ͬͨ ෺݅σʔλ 3BEPN
    'PSFTU
    ʢύλʔϯೝࣝʣ ࠓ೔ͷ ෺݅σʔλ $7ͦ͠͏ͳ ෺݅σʔλʂ Πϝʔδ ֶश ֶश ༧ଌ
54. 3ͷSBOHFS
    QBDLBHFͳΒ QSPCBCJMJUZ͕ग़ྗՄೳ install.packages('Rcpp') install.packages('ranger') # make CV model CV.ranger <- ranger(formula = CV ~ walk_time1+struct+struct_all +level3+direction+madori_num+madori_type_num+price_with_kanrihi +history_TOTAL+station1+bukken_shubetsu+gyosha_no, data = DFtrain, num.trees=300, write.forest =TRUE, probability =TRUE, always.split.variables= "station1") # prediction of CV model cv.predict <- predict(CVmodel,DF) # draw probabilities cv.predict$predictions[,2]
55. $7
    1SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ
56. $7
    1SPCBCJMJUZͷߴ͍ ॱʹϦετදࣔ ޲্
57. ʢ໨తʣ
    $73վળ
    ʢख๏ʣ
    ̎஋෼ྨʢ$7֬཰Λܭࢉʣ
    ʢํ๏ʣ
    3BOEPN'PSFTU RͩͱrangerͳͲ৽͍࣮͠૷͕Φεεϝʂʂ
58. ƅƁƅ Űŕ
59. ʮਓ޻஌ೳ࣌୅ʹඋ͑ͯ ෆಈ࢈ؔ࿈σʔλʹ͍ͭ ͯ৭ʑޠΒ͏ษڧձʯ
60. ਓ޻஌ೳͷఆٛ
61. ਓ޻஌ೳͷఆٛ %FFQ-FBSOJOHͰ
    ͳΜ͔Ͱ͖ͳ͍͔ͳʔ
62. %FFQ-FBSOJOH
    ͱ͍͑͹ը૾ղੳ
63. ͜Ε·Ͱͷ෺݅αΠτʹ ͸ͳ͍ըظతػೳʂ
64. ࿨ࣜτΠϨ൑ఆث
65. ࿨ࣜτΠϨ൑ఆث ࿨ࣜτΠϨ͚ͩ͸ઈରʹݏͩʂ
    ͱݴ͏ਓ͸͖ͬͱ͍Δʹ͕͍ͪͳ͍
66. H2O Deeplearning library(h2o) # Deep learningͰֶशͤ͞Δ localH2O <- h2o.init(ip = "localhost", port = 54321, startH2O = TRUE, nthreads=-1) res.dl <- h2o.deeplearning(x = 2:10001, y = 1, training_frame = as.h2o(target), activation = "TanhWithDropout", hidden=rep(160,5), epochs = 20) pred.dl <- h2o.predict(object=res.dl, newdata = as.h2o(target)) pred <- as.data.frame(pred.dl) # ਖ਼ղ཰Λ֬ೝ print(1-sum(abs(round(pred[,1]) - target[,1]))/length(target[,1]))  ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू
     ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵
     %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒
67. H2O Deeplearning ਖ਼ղ཰ɿ  ࿨ࣜτΠϨը૾ΛਓྗͰ݅ఔ౓ऩू
     ը૾αΠζΛἧ͑ͯɺάϨʔεέʔϧʹม׵
     %FFQMFBSOJOHͰ࿨ࣜτΠϨ൑ఆثΛ࡞੒
68. ʢࠓ೔ͷ͓࿩ʣ
    ೖྗϛεσʔλ࡟আ
    ಛघ෺݅൑ఆ
    $73վળϑΟϧλ
    ࿨ࣜτΠϨ൑ఆث
69. ΋ͬͱ෺݅બͼΛ
    ָ͍͠΋ͷʹͯ͠
    ͍͖·͠ΐ͏ʂ
70. ͋Γ͕ͱ͏͍͟͝·ͨ͠