Share
Embed
パターン認識と機械学習 第1章 #PRML学ぼう PRML輪講 #2 / PRML Seminar 2 go to introduction in machine learning
筑波大学で開催される、パターン認識と機械学習についての勉強会の資料です。
PRMLと呼ばれるパターン認識機械学習という本についての輪講を行っています。
このスライドは第1章の内容をまとめたものです。
https://github.com/hurutoriya/prml-seminar/tree/master/chapter1
Githubで管理しています。
Shunya Ueta
April 10, 2015
Transcript
43 slides
-
PRML Seminar #1 PRML Seminar #1 1.1-1.6.1 #PRML ֶ〱⿸ Shunya
Ueta Graduate School of SIE, Univ. of Tsukuba Department of Computer Science April 13, 2015 1 / 43 -
PRML Seminar #1 Introduction 〈〣ษڧձ〠〙⿶〛 PRML ྠߨ #2 ༰ ୈ
1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ 1.2 ֬ 1.4 ࣍ݩ〣ढ⿶ 1.5 ܾఆཧ 1.6 ใཧ 2 / 43 -
PRML Seminar #1 Introduction ࣗݾհ ▶ ໊લ:্ా൏ (@hurutoriya) ▶ ஜେֶେֶӃ
1 Go to Doctor course :) ▶ ใཧݚڀࣨॴଐ ▶ ݚڀ : ը૾ೝࣝɾػցֶश 3 / 43 -
PRML Seminar #1 Introduction 〈〣ษڧձ〠〙⿶〛 〈〣ษڧձ〠〙⿶〛 ▶ むのがアೝࣝ〝ػցֶश〠〙⿶〛〣ྠߨ〜『 ػցֶश〝むのがアೝࣝ〣جૅぇཧղɺ࣮༻゛よ゚〜⿶〈 〟『ࣄぇత〠なゎべがぇ։࠵「〛⿶ 〳『
▶ 2015 ぇॲ〠Ұप༧ఆ ▶ डߨऀ〠〤جૅత〟ඍੵɾઢܗɾ֬౷ܭ〣ࣝぇલ ఏ〝「〛⿶〳『 ▶ ࢿྉத〣つアゆ゚ぢがへ〤 Python ぇ࠾༻「〛⿶〳『ɻ ▶ ษڧձ〠ؔ『ใ〠〙⿶〛〤 Hashtag: #PRML ֶ〱⿸ ぇ 〘〛ൃ৴「〛⿶ 〳『 4 / 43 -
PRML Seminar #1 Introduction PRML ྠߨ #2 ༰ ࠓճ〣୲ 1
→ 1.6 5 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ػցֶश〝〤? むのがアೝࣝ (Pattern
Recognition): ܭࢉػぎ゚っ゙どわぇ௨」〛ɺぶがの〣த〣نଇぇࣗಈత〠ݟ〙 々ग़『ɻߋ〠〒〣نଇੑぇ༻⿶〛ぶがのぇҟ〟じふっ゙〠ྨ 『ɻ ྫ) खॻ ࣈ〣ೝࣝ ೖྗ〝「〛 28 × 28 〣େ 《〣खॻ ࣈ〣ը૾⿴ɻೖྗ ぶがの〤 784 ࣍ݩ〣࣮よぜぷ゚ x 〜දݱ〜 ɻよぜぷ゚ x ぇೖྗ〝「〛ड々औ〿ɺ〒ぁ 0 . . . 9 〣〞〣ࣈぇද「〛⿶ ぇग़ྗ『ػցぇ࡞ɻ ਤ 1: खॻ ࣈ〣ը૾ྫ 6 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ࣮ݱํ๏ (1) ਓྗ〠〽゚が゚〣࡞
→ ゚が゚〣ൃࢄ (࣮ݱෆՄ) ػցֶशతぎゆ゜がば ▶ ܇࿅ू߹ (training set): N ݸ〣खॻ จࣈ〣େ 〟ू߹ぇ༻ҙ『 {x1, . . . , xN } ▶ ඪよぜぷ゚ (target vector): t Ұ〙Ұ〙〣ࣈ〠ରԠ『じふっ゙ぇද『よぜぷ゚ ࠷ऴత〠ಘ〾ぁ〣〤 y(x) 〜⿴ɻ 〈〣ؔ〠ࣈ〣ը૾ x ぇೖྗ『〝ɺඪよぜぷ゚〝ը૾ぶが の〠ූ߸ԽՄೳ〟ぶがの߹《ぁ〔よぜぷ゚ y(ඪよぜぷ゚) よ゙アそ《ぁ〔ը૾ฦ〘〛。ɻ 7 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ࣮ݱํ๏ (2) ▶
܇࿅ (training), ֶश (leaning) ஈ֊ : training set 〣〴〜ゑぶ゚Խ《ぁ〛⿶ঢ়ଶ ▶ ふとぷू߹ (test set) : ܇࿅ू߹Ҏ֎〣ぶがの (ະ〣ぶがの) ▶ ൚Խ (generalization) : ܇࿅ू߹Ҏ֎〣ぶがの (ະ〣ぶがの) 〠ର「〛దԠՄೳ〠《 【〈〝 ࣮〝「〛ೖྗぶがの〤େ 〟ଟ༷ੑぇ࣋〙ɻ→ ൚Խத৺త〟՝〝〟ɻ 8 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ྫ)Ajax ぇ〘〔खॻ จࣈೝࣝ ਤ
2: Ajax ぇ〘〔खॻ จࣈೝࣝ ࢀߟ: http://chasen.org/ taku/software/ajax/hwr/ 9 / 43 -
PRML Seminar #1 ୈ 1 ষ ং લॲཧ (Preprocessing) ࣮ੈք〜〤ɺೖྗม〤લॲཧ
(Preprocessing) 〠〽〿ぇղ 〹『。「〛。ɻ ྫ) खॻ ࣈ ▶ ࣈը૾〠มܗ (ぎやくアม)ɺ֦େɾॖখぇߦ⿶ ಉҰ〣େ 《〠ม → ೖྗぶがの〣ଟ༷ੑ〣ݮগ લॲཧ〣ஈ֊〤ಛநग़ (feature extraction) 〝ݺ〥ぁɻ ଟ༷ੑݮগ〣తҎ֎〠〷ɺܭࢉ〣ߴԽ〣〔〶〠〷༻⿶〾ぁ ࣄଟ⿶ɻ 10 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ػցֶश〣ྨ 1. ڭࢣ⿴〿ֶश
(supervised learning) : ܇࿅ぶがのよ゙ア そ《ぁ〛⿶ঢ়ଶ〜〣 ▶ ぜとྨ (classification) : ֤ೖྗよぜぷ゚ぇ༗ݶݸ〣 ࢄじふっ゙〠ྨ『 ▶ ճؼ (regression) : ٻ〶ग़ྗҰ〙〟⿶「〤〒ぁҎ্〣࿈ଓ ม〜⿴〽⿸〟 2. ڭࢣ〟「ֶश (unsupervised learning) : ܇࿅ぶがのよ゙ アそ《ぁ〛⿶ঢ়ଶ〜〣 ▶ ぜとの゙アそ (clustering) : ྨࣅ「〔ࣄྫ〣そ゚がゆぇݟ〙 々 ▶ ີਪఆ (density estimation) : ೖྗۭؒ〠々ぶがの〣 ぇݟ〙々 3. ڭࢣ⿴〿ֶश (semis-upervised learning) : ܇࿅ぶがの よ゙アそ《ぁ〛⿶〷〣〝ඇよ゙アそঢ়ଶ〣ࠞࡏ「〛 ⿶ঢ়ଶ〜〣 11 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ڧԽֶश (reinforcement learning)(1)
ڭՊֶश (reinforcement learning) : ⿴༩⿺〾ぁ〔݅Լ〜ɺใ ुぇ࠷େԽ『〽⿸〟ద〟ߦಈぇݟ〙々ɻ ঢ়ଶ〝ߦಈ〣ܥྻ〾ڥ〝〣૬ޓ࡞༻ぇ௨」〛ֶशぇߦ⿸ (ߦ ಈج४〤ۙ〣ใु〕々〜〤〟。ɺաڈ〣ߦಈ〷ࢀߟ〠《ぁ)ɻ ڭࢣ⿴〿ֶश〝〣ҧ⿶ : ࠷ద〟⿺〤༩⿺〾ぁ』〠ࢼߦࡨޡぇ௨ 」〛ֶशぎ゚っ゙どわࣗ〾࠷దղぇൃݟ『 みひぜせをゑア〠ର『ڧԽֶश〣ద༻ (Tesauro 1994) ぺゔが゚ぼひぷゞがぜ (ୈ 5 ষ) 〠〽〿ɺࣗࣗ〣ぢゃが〝Կ ඦສ〷〣だがわぇ〈〟『ඞཁ⿴ɻ બࢶ〤ແ〠ଘࡏ『ɺউར〝⿶⿸ܗ〜「ใुぇ༩⿺〈 〝〜 〟⿶ɻ〒〣〔〶ɺউར〠ؔ『ख〠ର「〛〤ਖ਼֬〠ใ ुぇׂ〿〛ඞཁ⿴ (৴པׂ〿〛)ɻ 12 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ڧԽֶश (reinforcement learning)(2)
Լه〣̎〙ぇߦ⿶ڧԽֶशぇߦ⿸ (ぷ゛がへざや〣ؔ)ɻ ▶ ୳ࠪ (exploration) : ৽ن〣ख〞ぁ〰〞༗ޮ〟〣ぇ୳『 ▶ ར༻ (exploitation) : ߴ⿶ใुಘ〾ぁ〈〝い〘〛⿶ ߦಈぇऔ 13 / 43 -
PRML Seminar #1 ୈ 1 ষ ং ୈ 1 ষ〜ಋೖ『
3 〙〣ॏཁ〟ಓ۩ 1. ֬ 2. ܾఆཧ 3. ใཧ 14 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ 1.1
ଟ߲ࣜۂઢやくひふくアそ ܇࿅ぶがの ೖྗ : N ݸ〣؍ଌ x ぇฒ〮〔 x = (x1, . . . , xN )T ग़ྗ : 〒ぁ〓ぁ〠ରԠ『؍ଌ t = (t1, . . . , tN )T ະ〣ೖྗม x 〠ର「〛ඪมぇ༧ଌ「〔⿶ t (൚Խ) ඪ〝『ゑぶ゚〤 sin 2πxɺ܇࿅ぶがの〤ぽぐどぇ【〛 N = 10 〜༩⿺〾ぁɻ ਤ 3: ex)N=10 〜܇࿅ぶがの༩⿺〾ぁ〛⿶ 15 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ 1.1
ଟ߲ࣜۂઢやくひふくアそ y(x, w) = w0x0 + w1x1 + · · · + wM xM = M ∑ j=0 wjxj M:ଟ߲ࣜ〣࣍ (order) ω:ଟ߲ࣜ〣ɺ〳〝〶〛 ω 〜ද『 E(w) = 1 2 N ∑ n=1 {y(xn, w) − tn}2 ܇࿅ぶがの〠ଟ߲ࣜぇ⿴〛〤〶〈〝〜ɺ〣ぇٻ〶〔⿶ → ޡࠩؔ (Error Function) 〣࠷খԽぇࢦ『ɻ 16 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ Fitting
Order Case:M=0,1 0.0 0.2 0.4 0.6 0.8 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 ਤ 4: M=0 0.0 0.2 0.4 0.6 0.8 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 ਤ 5: M=1 17 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ Fitting
Order Case:M=3,9 0.0 0.2 0.4 0.6 0.8 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 ਤ 6: M=3 0.0 0.2 0.4 0.6 0.8 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 ਤ 7: M=9 18 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ աֶश〝ฏۉೋฏํࠜޡࠩ
9 ࣍〜やくひふくアそ「〔߹ɺաֶश (over-fitting) ػցֶश〣ඪ:ະ〣ぶがの〠ର「〛ਫ਼〣ߴ⿶༧ଌ (൚Խ) ERMS = √ 2E(w)/N ฏۉೋฏํࠜޡࠩ (root-mean-square error,RMS error) ▶ N 〜ׂ〈〝〜つアゆ゚〣せをひゆぇফ『 ▶ ฏํࠜぇऔ〈〝〜ɺݩ〣ई〠『 19 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ ぶがのなひぷ〣つぐど〠Ԡ」〔աֶश〣༷ࢠ
0.0 0.2 0.4 0.6 0.8 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 ਤ 8: M=9,N=1000 20 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ ޡࠩؔ〣ਖ਼ଇԽ
(regularization) աֶशぇ੍ޚ E(w) = 1 2 N ∑ n=1 {y(xn, ω) − tn}2 + λ 2 ||w||2 ̎࣍〣ਖ਼ଇԽ〣߹ɺ゙ひでճؼ (ridge regression) 21 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.1 ଟ߲ࣜۂઢやくひふくアそ ݕূ༻ू߹
(Validation set):w ぇܾఆ『〔〶〠ぶがのなひぷ りが゚へɾぎげぷू߹ (hold-out set) 〝〷ݺ〥ぁɻ ܽ: وॏ〟ぶがのぇແବ〠『 22 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ ֬
ぶがの〠〤ぽぐどඞ』ਵ「ɺぶがのなひぷ〣つぐど〷༗ݶ〜 ⿴ɻෆ࣮֬ੑॏཁ〟֓೦〝〟ɻ ֬〣֓೦ぇ؆୯〠આ໌ ਤ 9: 〝੨ɺ̎〙〣ശ⿴ɻ〣ശ〠〤〿え〉 2 ݸɺざ゛アで 6 ݸɺ੨〣ശ〠〤〿え〉 3 ݸɺざ゛アでҰݸೖ〘〛⿶ɻ〣ശぇ 40%, ੨〣ശぇ 60%〜બ〨ɺՌ〤ಉ」֬〾「《〜બ〫ɻ 23 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ ֬〣جຊత๏ଇ
֬〣Ճ๏ఆཧ (Sum rule) p(X) = ∑ Y p(X, Y ) ֬〣๏ఆཧ (Prduct rule) p(X, Y ) = p(Y |X)p(X) よぐど〣ఆཧ (Bay’s theorem) p(Y |X) = p(X|Y )p(Y ) p(X) 24 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ よぐど〣ఆཧ〣ײత〟આ໌
〞〣ശぇબ〨〳「〔? ࣄલ֬ (prior probability) ࣄલ〠ಘ〾ぁ֬ p(Box) ࣄޙ֬ (posterior probability) Ռぇબえ〕ޙ֬ఆ『֬ p(Box|Fruit) Ұ୴Ռざ゛アで〕〝いぁ〥ɺ⿶ശ〤ざ゛アで〣ଟ⿶ → ⿶ശ〜⿴֬ߴ。〟ɻ 25 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.1
࿈ଓม〭〣֦ு ֬ີ (probability density) ର:࿈ଓ (continuous value) ࿈ଓ〜⿴ม x ۠ؒ (x, x + δx) 〠ೖ֬ δx → 0 〜༩⿺〾ぁ〔࣌〣 x ্〣 p(x) 1. p(x) ≥ 0 2. ∫ ∞ −∞ p(x)dx = 1 ࣭֬ྔ (probability mass) ର:ࢄू߹ (discrete set) ࢄม〜⿴ x ⿴ x 〠〟֬ p(x) 26 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.2
ظ〝ࢄ〠〙⿶〛 ظ (expectation) ⿴ؔ f(x) 〣֬ p(x) Լ〜〣ฏۉ ࢄ E[f] = ∑ x p(x)f(x) ࿈ଓม E[f] = ∫ p(x)f(x)dx ࢄ (variance) var[f] = E[(f(x) − E[f(x)])2] = E[f(x)2 − 2f(x)(E)[f(x)] + E[f(x)]2] = E[f(x)2 − 2f(x)(E)[f(x)] + E[f(x)]2] = E[f(x)2] − 2E[f(x)](E)[f(x)] + E[f(x)]2 = E[f(x)2] − E[f(x)]2 seminar1.5 27 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ ڞࢄ
(covariance) cov[x, y] = Ex,y[(x − E[x])(y − E[y])] = Ex,y[xy + E[x]E[y] − xE[y] − yE[x]] = Ex,y[xy] + Ex,y[E[x]E[y]] − Ex,y[xE[y]] − Ex,y[yE[x]] = Ex,y[xy] + E[x]E[y] − E[x]E[y]] − E[y]E[x] = Ex,y[xy] − E[x]E[y] = Ex,y[xy] − E[x]E[y](x, y: independ) = E[x]E[y] − E[x]E[y](x, y: independ) = 0 seminar1.6 28 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.3
Frequentist VS Bayesian Frequentist ֬ぇアはわ〟܁〿ฦ「తࢼߦ〣ස〝〴〟『 Bayesian ෆ࣮֬ੑ〣߹⿶ぇ֬〝『 ྫ) 〈〣ੈք〠Կ〷܁〿ฦ「ߦ⿸〈〝〜 ࣄ 〞ぁ〕々⿴ߟ⿺〛〴〛ཉ「⿶ɻ ೆۃ〣ණࣦ『〟〞ෆ֬〟ࣄى 〔〝「 〽⿸ɻؒ〞〣ఔ༹々〛⿶〣ใぇಘ〈〝 〜ɺ〒〣ใぇదԠ『〈〝〜ೆۃ〣ණࣦ『 ෆ֬《ぇ༧ଌ『ɻ 29 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.3
Likelifood function working on Frequentist, Bayesian ؔ (likelihood function) p(Data|param) ぶがの〠ର『ධՁɻむゐが の〣ؔ〝〴〟【ɻ Frequentist むゐがの〤ݻఆ《ぁ〛⿶〷〣〝ߟ⿺〾ぁ〛⿶ɻ ぶがの〣ぇߟྀ「〛ɺむゐがの〤ܾఆ《ぁɻ Bayesian ぶがの〤།Ұ〠ఆ〳〿ɺむゐがの〠ؔ『ෆ࣮֬ ੑ〤 w 〣֬〝「〛දݱ《ぁɻ 30 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ Bayesian
〣ར ࣄલࣝぇࣗવ〠ಋೖ〜 〈〝 ྫ) Frequentist 〠々؍ଌ〠ภ〿ൃੜ「〔߹〣֬ ެฏ〠දɾཪ〜ぢぐアぇ 3 ճ〆〛ຖճද〜〔ɻݹయత〟 ࠷ਪఆ〜〤ɺදग़֬〤 1 〠〟〘〛「〳⿸ɻ 〤֬〝త〠ಉ」〜⿴ɻ ྫ) つぐぢ゜ぇৼ〘〛 1 ࡾճ࿈ଓಉ」ग़ಉ ࣌֬〝〤ಉ〜⿴ɻ ҧ⿶〤ɺ֬〤ʮࣄ〣֬ʯ〜⿴〿ɺ〤ʮ؍ ଌぶがのԼ〜〣Ծઆ〣ʯ〜⿴ɻ (likelihood for a hypothesis given a set of observations) 31 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.4
Gaussian distribution N(x|µ, σ2) = 1 √ 2πσ2 exp{− 1 2σ2 (x − µ)2} params: µ, σ2 −3 −2 −1 0 1 2 3 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 ਤ 10: µ = 0, σ = 1 32 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ 1.2.5
Re:Fitting curve Bayesian 〠々ۂઢやくひふくアそ p(t|x, w, β) = N(t|y(x, w), β−1) 〣ٯࢄ〠૬『むゐがの β ぇఆٛɻ ܇࿅ぶがの x, t ぇ〘〛ɺະ〣むゐがの w, β ぇٻ〶〣〠 ࠷ਪఆぇ༻⿶ɻ p(t|x, w, β) = N ∏ n=1 N(tn|y(xn, w), β−1) 33 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ Maximum
likelifood function ؔぇ࠷େԽ『《⿶〠〤ɺରぇ༻⿶ɻར〝「〛 1. ࢉՃࢉ〠มԽ 2. ֬〤গ〜දݱ《ぁ〣〜ɺࢉぇߦ⿸〝ぎアはがや゜が සൃ p(t|x, w, β) = N ∏ n=1 N(tn|y(xn, w), β−1) ln p(t|x, w, β) = − β 2 N ∑ n=1 {y(xn, w) − tn}2 + N 2 , ln β − N 2 ln2π 34 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.2 ֬ ༧ଌ
(predictive distribution) むゐがのよぜぷ゚ wML ぇ〳』ܾఆ「ɺβML ぇਪఆ『ɻ 1 βML = 1 n N ∑ n=1 {y(xn, wML) − tn}2 βML ܾఆ《ぁ〔〈〝〠〽〿༧ଌ〝⿶⿸ܗ〜 t 〣֬ぇ ߟ⿺〈〝〜 ɻ p(t|x, w, β) = N ∏ n=1 N(tn|y(xn, w), β−1) 35 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.4 ࣍ݩ〣ढ⿶ ࣮ੈք〭〣Ԡ༻
ۂઢやくひふくアそ〜〤ɺೖྗむゐがの〤Ұ〙ɻݱ࣮〜〤 ೖྗむゐがの〤ෳݸଘࡏ『〣〔〿લ〜⿴ɻ ࣗ〔〖〤 3 ࣍ݩ〣ଘࡏɻ4 ࣍ݩҎ্〣ۭؒ〠ؔ「〛〤زԿత ײ〤ಇ 〚〾⿶ɻ 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 ਤ 11: 3 ぜとଘࡏ『ぶがのなひぷ〣ࢄਤɺい〿〹『⿶〽⿸〠 ̎࣍ݩ〣෦ۭ〭ؒ 36 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.4 ࣍ݩ〣ढ⿶ ׂ〿〛ํ๏〣ఏҊ
10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 ਤ 12: ಉぞひへ〜࠷〷ଟ⿶ぜとぇׂ〿〛 ߴ࣍ݩ〠༗ޮ〟̎〙〣ੑ࣭ 1. ࣮ぶがの〜〷༗ޮ〟ぶがの〤࣍ݩ〠ूத「〛⿶ 2. ぶがの〤〾〟ࣄଟ⿶ (ہॴత〠) 〣〜ɺૠ〠〽〿ରԠ 『 ֶशゑぶ゚ぇద༻『ඞཁ〤〟。ɺૠ〜ࣄ〿 37 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.5 ܾఆཧ About
decision theory ೖྗよぜぷ゚ x 〝ඪม t ⿴〿ɺx 〣৽〔〟〠ର「〛ରԠ 『 t ぇٻ〶〔⿶ɻ ճؼ t 〤࿈ଓม ぜとྨ t 〤ぜとよ゚ ਪ (inference) ܇࿅ぶがの x, t 〾ಉ࣌֬ p(x, t) ぇٻ〶 〈〝 ܾఆཧ〣ओ〤ਪ〜ಘ〾ぁ〔֬ t ぇ༧ଌ「ܾఆ『 〈〝ɻ 38 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.5 ܾఆཧ key
of decision theory ▶ Berger, James O. Statistical decision theory and Bayesian analysis. Springer Science & Business Media, 1985. ▶ Bather, John. ”Decision Theory.A n Introduction to Dynamic Programming and Sequential Decisions.” (2000). APA ྫ) ױऀ〣 X ઢը૾ぇよぜぷ゚Խ「〔 x ぇ༻「〛ɺ〒〣ױऀ ؞〞⿸ぇぜと C1(؞ױऀ),C2(ඇ؞ױऀ) 〠い々〔⿶ɻ (p(Ck|x ぇٻ〶〔⿶) p(Ck|x) = p(x|Ck)p(Ck) p(x) p(Ck) 〤 X ઢը૾ぇద༻『લ〠ਓؒ؞〠ࣄલ֬ɺ p(Ck|x) 〤 X ઢը૾〾ಘ〾ぁ〔ใぇ༻「〛よぐど〣ఆཧぇ ༻⿶〛मਖ਼「〔ࣄޙ֬〜⿴ɻ 39 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.5 ܾఆཧ some
decision rule ଛࣦؔ (loss function) ܾఆɾߦಈ〠ਵ『ଛࣦぇද『ؔ ؞ױऀ〣ぜとྨ〜ɺ〞〖〾ױऀ〠〽〿େ ⿶ ଛࣦぇ༩⿺〕あ⿸? ▶ ؞〜〟⿶ਓぇ؞〝அ『 ▶ ؞〣ਓぇ؞〝அ『 غ٫ざゆてゖア (reject option) ܾఆࠔ〟߹〠〤ɺܾఆぇආ 々બ 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Probabability θ 棄却領域 p(C2|x) p(C1|x) 40 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.5 ܾఆཧ 1.5.4
inferense and decision ਪஈ֊ → ܾఆஈ֊〭〣̏〙〣ぎゆ゜がば ੜゑぶ゚ (generative model) ぜと〉〝〠ぜと〣݅ ີ p(x|Ck) ぇܾఆ 『ਪぇղ。ɻ〒「〛 p(Ck) 〷ٻ〶〛ɺࣄޙ ֬ p(Ck|x) ぇٻ〶ɻ〒「〛ܾఆཧ〠〽〿ぜ と〠ׂ〿〛ɻग़ྗ〕々〜〟〟。ೖྗ〷ゑぶ゚Խ 《ぁ〣〜ɺぶがの〣ੜ〷Մೳ〠〟ɻ ࣝผゑぶ゚ (discriminative model) ぜとࣄޙ֬ p(Ck|x) ぇܾఆ『ਪぇղ ɺ ܾఆཧぇ༻⿶〛৽〔〟 x ぇぜと〠ׂ〿〛ɻ ࣝผؔ (discriminative function) ਪ【』〠ܾఆ『ɻ֤ ೖྗ x 〾ぜとよ゚〭〝ࣸ૾『ɻ 41 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.6 ใཧ ใྔ〝ごアぷ゜ゃが
⿴ࢄ֬ม x ⿴〝『ɻ h(x) = − log2 p(x) : ใྔぇද『 H[x] = − ∑ x p(x) log2 p(x) : ֬ม x 〣ごアぷ゜ゃが 42 / 43 -
PRML Seminar #1 ୈ 1 ষ ং 1.6 ใཧ ֬ม
x ぇ༩⿺ɺ8 ݸ〣Մೳ〟ঢ়ଶぇ֬〜औɻ 〒〣ぇड৴ऀ〠 3bit 〣《〠「〛ૹɻ 〈〣ม〣ごアぷ゜ゃが〤ɺ H[x] = −8 × 1 8 log2 1 8 = 3bit ࣍〠 x 〤 a, b, c, d, e, f, g 〣 8 〈〣Մೳ〟ঢ়ଶぇ〝〿ɺ {1 2 , 1 4 , 1 8 , 1 16 , 1 64 , 1 64 , 1 64 , 1 64 } 〣֬〜༩⿺〾ぁɻ〒〣ࡍ〣ごアぷ ゜ゃが〤 H[x] = − 1 2 log2 1 2 − 1 4 log2 1 4 − 1 8 log2 1 8 − 1 16 log2 1 16 − 4 64 log2 1 64 = 2bit ֬〣ඇҰ༷〟〷〣〽〿〷ɺҰ༷〟〣〰⿸ごアぷ゜ ゃが〤ߴ⿶ࣄいɻごアぷ゜ゃが〤֬〝ີ〠ؔ ⿴ɻ 43 / 43