ひたすら楽してディープラーニング

Transcript

1. Yuji Isobe ͻͨ͢Βָͯ͠
 σΟʔϓϥʔχϯά NodeֶԂ20࣌ݶ໨
2. [ “Node.js”, “MongoDB”, “AngularJS”, “socket.io”, “Emotion Intelligence “
 ] @yujiosaka +BWB4DSJQU
3. emin = Emotion Intelligence ؾ࣋ͪΛղ͢ΔςΫϊϩδʔͷ୳ڀ Emotion Intelligence͸ɺʮແҙࣝͷߦಈ͔Βɺ ਓͷؾ࣋ͪͷػඍΛղ͢Δ஌ੑʯΛɺਓ޻஌ೳ͓Αͼػցֶ शͷԠ༻ٕज़Λ༻͍ͯ ։ൃ͠ɺϏδωεʹԠ༻͍ͯ͠·͢ɻ
4. ZenClerk Series lϢʔβʔͷߪೖҙཉͷߴ·ΓzΛ࡯஌͢ΔγεςϜͰɺ
 ΋ͬͱαΠτͷച্Λ৳͹ͯ͠Έ·ͤΜ͔ʁ [FODMFSL
   MJUF ײ৘Λղੳ͢Δਓ޻஌ೳʮ&NPUJPO
   *0ʯ͕
 &$αΠτ্Ͱʮ࠷దͳλΠϛϯάͰͷൢଅʯΛ࣮ݱ͠ɺച্Λ޲্ͤ͞·͢ɻ ࠓ·͞ʹߪೖΛ໎͍ͬͯΔ͓٬༷ʹ
 ͏Ε͍͠ʮ࠷ޙͷͻͱԡ͠ʯΛఏڙ͠·͢ɻ ;FODMFSL
5. ػցֶशͱݴ͑͹…
6. Lee Sedol vs. AlphaGo
7. Google Trends
8. ࠔͬͨ(´ɾωɾʆ)
9. ͜ͷձࣾʹೖΔ·ͰҰ౓΋
 ػցֶश΍ͬͯ͜ͳ͔ͬͨorz
10. ͱΓ͋͑ͣೖ໳ॻΛಡΉ ✓ ϐϯΫͷബ͍ຊ ✓ ௨শʮ͸͡ύλຊʯ ✓ ݟͨ໨ΑΓࠎଠͳ಺༰ ✓ ͪΌΜͱ਺͕ࣜࡌͬͯΔ
 ʢͦͯͭ͠·ͮ͘ʣ
11. ୈ3ষɿϕΠζ ୈ5ষɿkNN๏ ୈ6ষɿҰൠԽઢܗϞσϧ ୈ7ষɿχϡʔϥϧωοτϫʔΫ ୈ8ষɿαϙʔτϕΫλʔϚγϯ ӽ͑ΒΕͳ͍น ← ෼͔Δ ← ෼͔Δ ← ෼͔Δ ← ͓ɺ͓͏… ← (ɾ㱼ɾ) ŜŠŘŘ!!
12. σʔλՄࢹԽ ػցֶश ਺ֶ ౷ܭֶ ܭࢉػՊֶ ίϛϡχέʔγϣϯ υϝΠϯ஌ࣝ ࣗ෼ͷεΩϧηοτ ͜͜Λ৳͹͍ͨ͠
13. Kaggle͸اۀ΍ݚڀऀ͕σʔλΛ౤ߘ͠ɺੈքதͷ ౷ܭՈ΍σʔλ෼ੳՈ͕ͦͷ࠷దϞσϧΛڝ͍߹͏ɺ ༧ଌϞσϦϯάٴͼ෼ੳख๏ؔ࿈ϓϥοτϑΥʔϜٴ ͼͦͷӡӦձࣾͰ͋Δɻ ΢ΟΩϖσΟΞ ૑ཱऀɿ ΞϯιχʔɾΰʔϧυϒϧʔϜ ઃཱɿ 2010೥4݄
15. ࠓճ͸͜Εʹ
 νϟϨϯδ
16. ✓ MNISTʢ਺ࣈͷखॻ͖σʔλʣͷ෼ྨ ✓ 28 x 28ϐΫηϧ ✓ 6ສ݅ͷֶशσʔλ ✓ 1ສ݅ͷςετσʔλ νϡʔτϦΞϧ՝୊
17. ਖ਼ղ཰99%Λ໨ࢦ͢ http://yann.lecun.com/exdb/mnist/
18. Ͱ΋ͳΜ͔೉ͦ͠͏…
19. Ͱ͖Ε͹ͻͨ͢Βָ͍ͨ͠
21. ͻͨ͢Βָͯ͠FF6
22. 1.࠷খউརճ਺ͰΫϦΞ 2.ඞਢઓಆҎ֎Ͱ౪·ͳ͍ 3.ಓதͷΞΠςϜ͸ճऩ͠ͳ͍ 4.ฤू͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠FF6
23. 1.࠷ۙͷϒʔϜʹ৐͔ͬΔ 2.ۤखͳݴޠ͸࢖Θͳ͍ 3.ߴ౓ͳϥΠϒϥϦ͸࢖Θͳ͍ 4.ϓϨθϯࢿྉ͸ؤுΒͳ͍ ͻͨ͢Βָͯ͠Kaggle ͱΓ͋͑ͣσΟʔϓϥʔχϯά
 ࢖ͬͯΈΔ +BWB4DSJQU͔͠࢖Θͳ͍ MPEBTIͱઢܗ୅਺ϥΠϒϥϦ
 ͑͋͞Ε͹͍͍ ΤϯδχΞͩ͠ίʔυͰউෛ
24. ·ͣ͸৘ใऩू͔Β
25. IUUQOFVSBMOFUXPSLTBOEEFFQMFBSOJOHDPNJOEFYIUNM
26. ✓ ӳޠͷΦϯϥΠϯϒοΫʢ೔ຊޠԽ΋ਐߦதʣ ✓ χϡʔϥϧωοτϫʔΫ͔Β
 σΟʔϓϥʔχϯά·ͰͷྲྀΕΛৄࡉʹղઆ ✓ Pythonͷαϯϓϧ࣮૷͕ಡΈ΍͍͢ Neural Networks and Deep Learning Φεεϝʂ
27. ઓུΛཱͯΔ
28. Python→CoffeeScript→ES2015 sed + ؾ߹ Decaf JS ੈքॳʁES2015ͰσΟʔϓϥʔχϯά npmௐ΂ JavaScript Babel
29. PythonͱCoffeeScriptͬͯࣅͯͶʁ
30. Python def update_mini_batch(self, mini_batch, eta): nabla_b = [np.zeros(b.shape) for b in self.biases] nabla_w = [np.zeros(w.shape) for w in self.weights] for x, y in mini_batch: delta_nabla_b, delta_nabla_w = self.backprop(x, y) nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)] nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)] self.weights = [w-(eta/len(mini_batch))*nw for w, nw in zip(self.weights, nabla_w)] self.biases = [b-(eta/len(mini_batch))*nb for b, nb in zip(self.biases, nabla_b)]
31. CoffeeScript updateMiniBatch: (miniBatch, eta) -> nablaB = (Matrix.zeros(b.rows, b.cols) for b in @biases) nablaW = (Matrix.zeros(w.rows, w.cols) for w in @weights) for [x, y] in miniBatch [deltaNablaB, deltaNablaW] = @backprop(x, y) nablaB = (nb.plus(dnb) for [nb, dnb] in _.zip(nablaB, deltaNablaB)) nablaW = (nw.plus(dnw) for [nw, dnw] in _.zip(nablaW, deltaNablaW)) @weights = (w.minus(nw.mulEach(eta / miniBatch.length))
 for [w, nw] in _.zip(@weights, nablaW)) @biases = (b.minus(nb.mulEach(eta / miniBatch.length))
 for [b, nb] in _.zip(@biases, nablaB))
32. PythonϥΠϒϥϦͷAPIΛ࣮૷͢Δ
33. numpy.nan_to_num nanToNum() { let thisData = this.data, rows = this.rows, cols = this.cols; let row, col, result = new Array(rows); for (row=0; row<rows; ++row) { result[row] = new Array(cols); for (col=0; col<cols; ++col) { result[row][col] = n2n(thisData[row][col]); } } return new Matrix(result); };
34. numpy.ravel ravel() { let thisData = this.data, rows = this.rows, cols = this.cols; let a = new Array(rows * cols); for (let i = 0, jBase = 0; i<rows; ++i, jBase += cols) { for (let j = 0; j<cols; ++j) { a[jBase + j] = thisData[i][j]; } } return a; };
35. CoffeeScript͕ڐ͞ΕΔͷ͸2015೥·ͰͩΑͶʔ https://github.com/juliankrispel/decaf
36. …
37. ಈ͍ͨস
38. ͦΖͦΖͪΌΜͱษڧ͢Δ
39. χϡʔϥϧωοτϫʔΫ ਆܦճ࿏໢ɺӳ
    OFVSBM
    OFUXPSL
    // ͸ɺ೴ػೳʹݟΒΕΔ͍͔ͭ͘ͷಛੑΛܭࢉػ ্ͷγϛϡϨʔγϣϯʹΑͬͯදݱ͢Δ͜ͱΛ໨ࢦͨ͠਺ֶϞσϧͰ͋Δɻ
    χϡʔϥϧωοτϫʔΫ
    
    8JLJQFEJB
    IUUQTKBXJLJQFEJBPSHXJLJχϡʔϥϧωοτϫʔΫ χϡʔϥϧωοτϫʔΫͱ͸ʁ
40. b ύʔηϓτϩϯϞσϧ x1 x2 x3 output w1 w2 w3 PVUQVU
    
    
    JG
    ЄKXKYK
    
    C
    ≤
    
 
    JG
    ЄKXKYK
    
    C
    
    
41. 5 ύʔηϓτϩϯϞσϧ ఱؾ͸ྑ͍͔ʁ ൴ঁ͸ߦ͖͍͔ͨʁ ձ৔͸Ӻͷ͔ۙ͘ʁ ͓ࡇΓʹߦ͘ʁ 6 2 2 No Yes Yes No 
    
    
    
    
    
    
    ≤
    
    
42. b γάϞΠυχϡʔϩϯϞσϧ x1 x2 x3 w1 w2 w3 PVUQVU
    
    
    
    
    
    
    
    
    
    
    
    
    
    
 
    
    FYQ ЄKXKYK
    
    C output
43. εςοϓؔ਺ʢύʔηϓτϩϯʣ         
44. γάϞΠυؔ਺         
45. ✓ 0ͱ1ͷؒͷ஋ΛදݱͰ͖ΔΑ͏ʹͳͬͨ ✓ খ͞ͳೖྗͷมԽ͕খ͞ͳग़ྗͷมԽʹରԠ͢Δ ✓ ཁ͢Δʹඍ෼Ͱ͖Δ͜ͱʹͳͬͨ Կ͕خ͍͠ͷ͔ʁ
46. χϡʔϥϧωοτϫʔΫͷߏ଄ w + Δw
 b + Δb output + Δoutput
47. ✓ ֤χϡʔϩϯʢɹʣͷॏΈʢwʣͱόΠΞεʢbʣ Λௐ੔͢Δ͜ͱͰ༧ଌਫ਼౓Λ޲্͢Δ ✓ ͦͷͨΊͷςΫχοΫͱͯ͠ɺ
 όοΫϓϩύήʔγϣϯͳͲ͕ੜΈग़͞Εͨ χϡʔϥϧωοτϫʔΫͷֶश
48. σΟʔϓϥʔχϯάͱ͸ʁ
49. χϡʔϥϧωοτϫʔΫͷߏ଄
50. σΟʔϓϥʔχϯάͷߏ଄
51. ࠷ۙͷϒʔϜͷഎܠ ✓ ࠷ۙςΫχοΫ͕͍ͭ͘΋ݟ͔ͭͬͨ ✓ ૚ΛॏͶͯ΋͏·ֶ͘शͰ͖ΔΑ͏ʹͳͬͨ ✓ ૚ΛॏͶΔ͜ͱͰදݱೳྗͱਫ਼౓͕޲্ͨ͠
52. ͍࣮͟૷΁
53. ৞ΈࠐΈχϡʔϥϧωοτϫʔΫ
54. ໰୊ ͨͬͨ1pxͣΕΔ͚ͩͰผͷը૾ͱͯ͠ೝࣝ͞Εͯ͠·͏ 1px
55. ղܾΠϝʔδ
56. ߏ଄ convolutional layer pooling layer
57. ✓ γάϞΠυؔ਺Ҏ֎ͷ׆ੑԽؔ਺ʢReLUʣ ✓ ਖ਼نԽʢL1ਖ਼نԽ/ਖ਼نԽ/υϩοϓΞ΢τʣ ✓ ΫϩεΤϯτϩϐʔίετؔ਺ ✓ ιϑτϚοΫεؔ਺ ✓ ॏΈॳظԽͷվળ ͦͷଞͷٕज़
58. σΟʔϓϥʔχϯά͸ٕज़ͷू߹ମ ʮσΟʔϓϥʔχϯάʯͱ͍͏ΞϧΰϦζϜ͸ଘࡏ͠ͳ͍ ໨తʹԠͯ͡Ϟσϧ΍ٕज़ΛύζϧͷΑ͏ʹ૊Έཱͯͯɺ
 ΑΓߴ͍ਫ਼౓ΛੜΈग़͢͜ͱ͕Ͱ͖Δ
59. ϋϚͬͨ͜ͱ
60. ϋϚͬͨᶃ
 ਺͕ࣜ಄ͷதʹೖͬͯ͜ͳ͍໰୊ ීஈ࢖͍׳ΕͨϓϩάϥϜʹͯ͠ΈΕ͹
 ҙ֎ͱʮͳΜͩͦΜͳ͜ͱ͔ʯͰࡁΉ͜ͱ΋͋Δ function sigmoid(z) { return 1 / (1 + Math.exp(-z)); } let output = sigmoid(w.dot(a).plus(b));
61. ܭࢉํ๏͸෼͔Βͳ͍ͷͰɺภඍ෼ͨ݁͠Ռͷ਺ࣜΛ
 stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ costDelta(y) { this.outputDropout.minus(y); } ϋϚͬͨᶄ
 ภඍ෼ͷܭࢉํ๏͕෼͔Βͳ͍໰୊
62. ڭՊॻ௨Γʹ࣮૷͢ΔͱιϑτϚοΫεؔ਺͕ܻ͋;Ε͢Δ ·ͨ΋΍stackoverflow͔ΒҾͬு͖ͬͯͨΒಈ͍ͨ ϋϚͬͨᶅ
 ڭՊॻʹ͸ॻ͍͍ͯͳ͍໰୊ let max = _.max(vector), tmp = _.map(vector, (v) => { return Math.exp(v - max); }), sum = _.sum(tmp); return _.map(tmp, (v) => { return v / sum; });
63. PythonͰ1࣌ؒͷͱ͜Ζ͕24͔͔࣌ؒΔ ཪͰNumpy͕ੌ͍͜ͱ΍ͬͯ͘ΕͯΔΒ͍͠ ·ͩΫϥελϦϯά΋Ͱ͖ͯͳ͍͠ɺϝϞϦޮ཰΋ѱ͍ ϋϚͬͨᶆ
 Զͷઢܗ୅਺ͷܭࢉͦ͘஗͍໰୊ ։ൃ࣌͸খ͞ͳσʔληοτͰରԠ
64. ࢀߟ࣮૷͕Theano΍TensorFlowͳͲͷߴػೳͳ
 ϥΠϒϥϦΛ࢖༻͍ͯ͠ΔͱԿ΍ͬͯΔͷ͔෼͔Βͳ͍ ࣗಈඍ෼ͬͯͳΜ΍ͶΜ ϋϚͬͨᶇ
 ϥΠϒϥϦߴػೳա͗໰୊ ؾ͍ͮͨΒPythonʹৄ͘͠ͳͬͯͨ
65. …
66. WIP
67. IUUQTHJUIVCDPNZVKJPTBLBEFFQMFBSOJOHKT ZVKJPTBLBEFFQMFBSOJOHKT
    %FFQ
    -FBSOJOH
    XSJUUFO
    JO
    &4
68. σϞ
69. ਖ਼ղ཰99.1% ※ ࣮ͨͩ͠ߦ࣌ؒ24࣌ؒҎ্
70. ຊ൪؀ڥͰ͸
 PythonΛ࢖͍·͠ΐ͏
71. ͻͨ͢Βָͯ͠
 σΟʔϓϥʔχϯά ָ͠ΜͰ
72. $BSFFST
    ࠾༻৘ใ
    &NPUJPO
    *0ͷςΫϊϩδʔͰɺ
    ʮҰาઌͷະདྷΛΑΓ๛͔ʹ͢Δ࢓૊ΈΛ࡞ΔʯϝϯόʔΛืूதͰ͢ɻ
    ืूதͷ৬छΛݟΔ WE ARE HIRING! https://www.emin.co.jp/careers/