Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[読み会] Individually Fair Gradient Boosting
Search
mei28
April 13, 2021
0
26
[読み会] Individually Fair Gradient Boosting
読み会資料
Individually Fair Gradient Boosting (ICLR 2021)
mei28
April 13, 2021
Tweet
Share
More Decks by mei28
See All by mei28
[読み会] “Are You Really Sure?” Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making
mei28
0
49
[JSAI'24] 人間の判断根拠は文脈によって異なるのか?〜信頼されるXAIに向けた人間の判断根拠理解〜
mei28
1
390
[CHI'24] Fair Machine Guidance to Enhance Fair Decision Making in Biased People
mei28
0
34
[DEIM2024] 卓球の得点予測における重要要素の分析
mei28
0
25
[Human-AI Decision Making勉強会] 意思決定 with AIは個人vsグループで変わるの?
mei28
0
180
[読み会] Words are All You Need? Language as an Approximation for Human Similality Judgements
mei28
0
28
[参加報告] AAAI'23
mei28
0
78
[計算機構論] Learning Models of Individual Behavior in Chess
mei28
0
67
[計算機構論] Why do tree-based models still outperform deep learning on tabular data?
mei28
0
43
Featured
See All Featured
Measuring & Analyzing Core Web Vitals
bluesmoon
1
39
Facilitating Awesome Meetings
lara
49
6k
GitHub's CSS Performance
jonrohan
1030
460k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
26
2k
Code Review Best Practice
trishagee
64
17k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5k
Fireside Chat
paigeccino
32
3k
Writing Fast Ruby
sferik
626
61k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
370
Embracing the Ebb and Flow
colly
84
4.4k
The Cult of Friendly URLs
andyhume
78
6k
Transcript
Individual Fair Gradient Boosting 2021/04/13 @ಡΈձ ༶໌
•ஶऀใ •Alexander Vargo, Fan Zhang, Mikhail Yurochkin, Yuekai Sun
•ϛγΨϯେֶɼ্ւՊٕେֶɼMIT-IBM Watson AI Lab •ग़య: ICLR2021 •ͳΜͰબΜ͔ͩʁ •ݸผެฏੑ+ܾఆΛߟ͍͑ͯΔͷҙ֎ʹগͳ͍ɽˠ͜Ε͕ॳΊͯΒ͍͠ จใ ݸผެฏੑ + GBDTʹͨ͠ݚڀ
•ػցֶश(ML)͕ҙࢥܾఆͷͰ͘ΘΕ࢝Ί͍ͯΔ •ಛఆͷάϧʔϓ(ਓ)ʹରͯ͠ෆެฏͳධՁΛ͍͚ͯ͠ͳ͍ •Amazonͷཤྺॻ৹ࠪγεςϜͰࠩผ͕ߦΘΕ͍ͯͨ͜ͱ͕໌Β͔ʹͳͬ ͨɽ ΠϯτϩμΫγϣϯ ެฏੑΛߟྀ͍͔ͯ͠ͳ͍ͱ͍͚ͳ͍
•MLք۾Ͱେ͖͘ೋछྨͷެฏੑΛߟ͑Δ •ݸผެฏੑ: ࣅ͍ͯΔݸਓಉ͡ධՁΛड͚Δ͖ •ूஂެฏੑ: ूஂ͝ͱʹධՁͷࠩผ͕ͳ͍Α͏ʹ͢Δ͖ •ूஂެฏੑ͕Α͘औΓ্͛ΒΕ͍ͯΔ •ݸਓͷྨࣅΛ͖ͪΜͱఆٛ͢Δ͜ͱ͕ࠔ͔ͩͬͨΒ ΠϯτϩμΫγϣϯ ࠓճݸผެฏੑΛରͱ͍ͯ͘͠
•දσʔλʹGBDTΛ༻͍Δͷ͕ओྲྀʹͳ͖͍ͬͯͯΔɽ •ैདྷͷFair-awarness MLͰnon-smoothͳϞσϧ ϊϯύϥϝτϦοΫMLͰ͋·Γྑ͍ޮՌ͕ಘΒΕͯ ͍ͳ͔ͬͨɽ ΠϯτϩμΫγϣϯ ޯϒʔεςΟϯάܾఆ(GBDT)Λରͱ͢Δ
•ݸผެฏੑΛରʹͨ͠GBDTʹΑΔख๏ΛఏҊͨ͠ɽ •ϞσϧͷͦΕͧΕͷެฏੑΛূ໌͢Δ͜ͱ͕Մೳɽ •ݸผެฏੑ͚ͩͰͳ͘ूஂެฏੑΛ্ͤͭͭ͞ɼਫ਼Λҡ࣋ ͢Δख๏ʹͳ͍ͬͯΔ͜ͱΛ࣮ݧతʹࣔͨ͠ɽ ΠϯτϩμΫγϣϯ ߩݙ
•ೖྗ: , ग़ྗ: •อޢ͢Δଐੑ: ͍ΘΏΔηϯγςΟϒଐੑ •αϯϓϧ͝ͱެฏࢦඪ: ͜Εαϯϓϧ͕͍ۙ΄Ͳࣅ͍ͯΔ •ඪ:
ɹαϯϓϧ͝ͱʹެฏͳϞσϧ Λ֫ಘ͢Δ͜ͱ 𝒳 ∈ ℝd 𝒴 = {0,1} 𝒵 = 𝒳 × {0,1} dx f : 𝒳 → {0,1} ४උ ͏ه߸Λఆٛ͢Δ
•ఢରֶशʹΑͬͯୡ͢Δํ๏ଘࡏ͍ͯ͠Δ • ֶश͕ೖྗʹରͯ͠Β͔Ͱ͋Δ͜ͱ͕લఏʹͳ͍ͬͯΔ •Β͔Ͱͳ͍Ϟσϧʢܾఆͱ͔ʣʹରͯ͠ఢରֶशΛߦ͑ ΔΑ͏ʹ͍ͨ͠ʂ • ੍ݶ͖ఢରతίετؔΛఆٛͨ͠Αʂ طଘख๏Ͳ͏ͩͬͨͷʁ Non-smoothͳϞσϧͰ͏·͍͔͘ͳ͔ͬͨɽ
•Transport cost function: ݸผͷαϯϓϧ͕͍ۙ΄Ͳখ͍͞ •Zͷ্֬ͷ࠷ద༌ૹڑ : ͷۙ͞Λߟ͍͑ͯΔ c ((x1
, y1), (x2 , y2)) ≜ d2 x (x1 , x2) + ∞ ⋅ 1 {y1 ≠y2} W W (P1 , P2) ≜ inf Π∈C(P1 , P2) ∫ 𝒵×𝒵 c (z1 , z2) dΠ (z1 , z2) ४උ αϯϓϧ͝ͱʹެฏͳϞσϧΛֶश͍ͨ͠
• σʔλੜɼ ͷඍখͳڐ༰ύϥϝʔλ •ඪຊ্ۭؒͰ1) σʔλੜ͕͍ۙ ɹɹɹɹɹ ɹ2) MLϞσϧͷଛࣦΛେ͖͘ͳΔͷ Λ୳͍ͨ͠
Lr (f) ≜ sup P:W(P, P* )≤ϵ 𝔼P [ℓ(f(X), Y)] P⋆ ϵ > 0 ४උ ఢରతϦεΫؔΛఆٛ͢Δɽ
•ྨࣅͨ͠αϯϓϧʹରͯ͠ϞσϧͷੑೳࠩΛݟ͚ͭΒΕΔ •ੑೳࠩΛ୳ࡧ͢Δ͜ͱͰʹରͯ͠ؤ݈ͳެฏੑͩͱଊ͑ ΒΕΔɽ •ݱঢ়ͩͱ·ͩsmoothͳϞσϧͷޯ͔͠ಘΒΕͳ͍ɽ ४උ ϩόετͰެฏͳΛಘ͍ͨʂ
•σʔληοτΛ֦ு͢Δ: •࠷ద༌ૹؔʹ੍ݶΛՃ͑Δ: ҧ্͍ͷσʔληοτ͔Ͳ͏͔ 𝒟0 ≜ {(xi , yi), (xi
,1 − yi)} n i=1 W𝒟 (P1 , P2) ≜ inf Π∈C0(P1 , P2) ∫ 𝒵×𝒵 c (z1 , z2) dΠ (z1 , z2) ఏҊख๏ ੍ݶΛՃ͑ͯnon-smoothͷͨΊʹ͢Δɽ
• σʔληοτΛՃ͑Δ͜ͱͰ্ք ʹࢦࣔ͞Εͨʹ੍ݶ ͞ΕΔ •͜ΕʹΑͬͯ༗ݶ࣍ݩઢܗܭը๏ʹΑͬͯղ͚ΔΑ͏ʹͳΔɽ •ଛࣦ ʹ͔͠ґଘͯ͠ͳ͍ ͔ΒඇฏͳϞσϧͰద༻Ͱ͖Δɽ D0
ℓ (f (xi), yi) and ℓ (f (xi) ,1 − yi) ఏҊख๏ ͬͱඇฏʹద༻Ͱ͖ΔΑ
ޯϒʔεςΟϯάͰ ΛٻΊΔඞཁ͕͋Δɽ μϯεΩϯͷఆཧΛ༻͍Δͱޯɼ ∂L ∂ ̂ y ∂L ∂
̂ yi = ∂ ∂f (xi) [ sup P:W𝒟(P, Pn)≤ϵ 𝔼P [ℓ (f (xi), yi)]] = ∑ y∈𝒴 ∂ ∂f (xi) [ℓ (f (xi), y)) P* (xi , y) ఏҊख๏ ޯϒʔεςΟϯάͰ͑ΔΑ͏ʹ͢Δ
•ઌड़ͷޯͰɼϞσϧΛඍ͢Δඞཁ͕ͳ͍͔ΒඇฏͳϞ σϧͰؔޯΛධՁ͢Δ͜ͱ͕Ͱ͖Δʂ •͋ͱ ΛٻΊΕྑ͍ɽ •ઢܗܭը๏ʹΑͬͯ ΛٻΊΔํ๏ΛఏҊ͢Δɽ P⋆ P⋆ ఏҊख๏
ؔޯΛߟ͑Δ
• ʹΑΔҙͷ ʹରͯ͠ɼ ͱ͢Δͱ ࣍ͷΑ͏ͳߦྻ ͰදͤΔɽ 1. 2. D0
P Pi,k = P({(xi , k}), k ∈ {0,1} WD (P, Pn ) ≤ ϵ Π Π ∈ Γ with Γ = {Π ∣ Π ∈ ℝn×n + , ⟨C, Π⟩ ≤ ϵ, ΠT ⋅ 1n = 1 n 1n} Π ⋅ y1 = (P1,1 , …, Pn,1), and Π ⋅ y0 = (P1,0 , …, Pn,0) ఏҊख๏ Λઢܗܭը๏ͰٻΊΔ P⋆
•ߦྻ ɹˠ ϥϕϧjͰ͋Δαϯϓϧj͕αϯϓϧiʹ ͳͬͨͱ͖ͷଛࣦ •ٻΊ͍ͨߦྻ ࣍ͷΑ͏ʹͳΔ Ri,j = l(f(xi
), yj ) Π⋆ Π⋆ ∈ arg max Π∈Γ ⟨R, Π⟩ ఏҊख๏ ͞Βʹఆ͍ٛͯ͘͠Α
•݁ہ࠷ޙͷ ΛٻΊΔ͜ͱ͕Ͱ͖Εྑ͍ɽ •ٻΊΔʹ͋ͨͬͯɼؔFʹԿԾఆΛஔ͍͍ͯͳ͍ͷͰɼඇ ฏͳؔʹద༻Ͱ͖Δɽ Π⋆ ఏҊख๏- ·ͱΊ ͜ΕͰඇฏͳؔʹద༻Ͱ͖Δʂ
•3ͭͷσʔληοτ(German Credit, Adult, COMPASS)Ͱݕূ •ఏҊख๏Ͱ༻͍ΔܾఆΞϧΰϦζϜɼXGBoostͱ͢Δɽ •ଛࣦؔϩδεςΟοΫଛࣦΛ༻͍Δɽ ࣮ݧ
•YurochikinΒͷΛར༻͢Δ: •QηϯγςΟϒ෦ۭؒͱߦ͢ΔࣹӨߦྻ •อޢ͞ΕΔηϯγςΟϒଐੑҎ֎ͷใ͕ಉ͡ͳΒಉʹѻΘ ΕΔ͖Ͱ͋Δͱ͍͏ߟ͔͑Β࡞ΒΕͨɽ d2 x = (x1 −
x2 , Q(x1 − x2 )) ࣮ݧ ެฏੑࢦඪʹ͍ͭͯ(ݸผͷαϯϓϧʹؔͯ͠)
•ܾఆख๏ʹؔͯ͠ɼର߅͕ͳ͍ͨΊόχϥΛ༻͍Δɽ •σʔλͷલॲཧΛ༻͍Δख๏ͱൺֱ͢Δ •อޢଐੑΛͳ͘͠ɼ෦ۭؒʹӨ͢Δ(Yurochkin et al., 2020) •ݸਓʹҟͳΔॏΈΛద༻ͯ͠όϥϯεΛͱΔ(Kamiran & Calders,
2011) ࣮ݧ ର߅ख๏ʹ͍ͭͯ
•อޢ͞ΕΔ͍ͯΔଐੑͱ૬͕ؔ͋Δଐੑ(e.g. ͔?࠺͔?)ΛͣΒ ͢͜ͱͰࣄ࣮ͷਓΛ࡞ɽ •→΄΅ಉ͡ਓ͔ͩΒಉ͡ධՁΛ͞ΕΔ͖ •อޢଐੑ͝ͱͷTPR,TNRͷࠩ(GAPMax)→Ϟσϧͷެฏੑࢦඪ •อޢଐੑ͝ͱͷRMSEͷࠩ(GAPRMSE) →Ϟσϧͷ༧ଌੑೳ ࣮ݧ ධՁʹ͍ͭͯ(طଘख๏ʹର͠༏ྼ͕ͳ͍Α͏ʹՃΛ͢Δ)
•ྸΛηϯγςΟϒଐੑʹઃఆ → ถࠃͰྸΛ͚ͭͯ༩৴ அ͢Δͷҧݑ •ࣹӨʹΑΔલॲཧఏҊ΄ͲݸਓͷެฏੑΛ্ͤ͞ͳ͔ͬͨɽ ࣮ݧ݁Ռ ᶃ German Credit
•ఏҊख๏GBDTͷੑೳͷྑ͞ΛҾ͖ܧ͗ͭͭɼެฏͳϞσϧʹ ͳ͍ͬͯͨʂ ࣮ݧ݁Ռ ᶄ Adult
•NNϞσϧͷํ͕ਫ਼جຊతʹྑ͔ͬͨɽ •͔͠͠ެฏੑʹ͍ͭͯɼఏҊͷํ͕ྑ͔ͬͨɽ ࣮ݧ݁Ռ ᶅCOMPASS
•ݸผެฏੑΛୡ͢Δ՝ΛMLϞσϧͷੑೳࠩΛ୳ࡧͰ͖ͳ͍ ͜ͱ → ୳ࡧۭؒΛ༗ݶ۠ؒʹ੍ݶ͢Δ͜ͱͰࠀͨ͠ɽ •ࠓճઃఆ੍ͨ͠ݶ͖ఢରଛࣦؔଞͷnon-smoothख๏(ϥϯ μϜϑΥϨετ)ͳͲʹద༻Ͱ͖Δ͔͠Εͳ͍ɽ •࣮ײͱͯ͠ɼNNϞσϧΑΓܾఆϕʔεͷ΄͏͕ਫ਼ʴެฏ ੑΛୡͰ͖ͦ͏. ·ͱΊ
ݸผެฏੑʴܾఆͷख๏ΛఏҊͨͧ͠
•࡞ऀ͕͍ࣔͯ͠ΔཧΛͪΌΜͱཧղͰ͖ͳ͍ͯ͘͘͠ɽ •ݸผެฏੑΛߟ͍͑ͯΔจΛಡΊͯྑ͔ͬͨɽ ײ ͖ͪΜͱཧΛ͑Δֶྗ͕ཉ͍͠