Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[読み会] Can I Trust My Fairness Metric? Assessing...

mei28
October 12, 2021
63

[読み会] Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference

読み会資料
Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference (NeurIPS 2020)

mei28

October 12, 2021
Tweet

More Decks by mei28

Transcript

  1. Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled

    Data and Bayesian Inference ಡΈձ@2021/10/1 2 ༶ ໌఩
  2.  ஶऀ: Disi Ji1, Padhraic Smyth1, Mark Steyvers 2 ॴଐ:

    University of California, 1 Department of Computer Science
 2 Department of Cognitive Science s બΜͩཧ༝ : ެฏੑʹؔ͢ΔධՁͱϕΠζతΞϓϩʔνΛֶͿͨΊ ࿦จ৘ใ
  3.  : ֶशͨ͠܇࿅Ϟσϧ, : ೖྗ, : Ϋϥεϥϕϧ Ϟσϧੜ੒είΞ: →Ϟσϧͷ༧ଌ֬཰ :

    Ϟσϧͷ༧ଌϥϕϧ ʹԠͯ͡ϥϕϧ͕ܾఆ͢Δ ෼ྨث͕ΩϟϦϒϨʔγϣϯ͞ΕΔ → είΞsͷ஋ͷ֬཰Ͱ༧ଌ͕߹͍ͬͯΔͱߟ͑ΒΕΔ M x y ∈ {0,1} s = PM (y = 1|x) ∈ [0,1] ̂ y s P( ̂ y = y|s) = s ४උ දه
  4.  : ର৅ͷूஂ (e.g. ਓछɼੑผ… ) : ूஂgʹ͓͚ΔԿ͔͠Βͷࢦඪ (e.g. Accuracy,

    TPR, FPR… ) : ެฏੑࢦඪ, ࠓճ͸ Ͱߟ͍͑ͯΔ : ͦΕͧΕϥϕϧ͋Γɼϥϕϧͳ͠σʔληοτ
 Ͱ͋Δঢ়گΛߟ͍͑ͯΔ g ∈ {0,1,...,G − 1} θg Δ = θ0 − θ1 g ∈ {0,1} nL , nU nL ≪ nU ४උ දه (ެฏੑ)
  5.  ϥϕϧͳ͠σʔληοτͷϥϕϧ͸ɼείΞ Λ༻͍ٖͯࣅతʹར ༻ αϯϓϧ( )͸฼ूஂ ΋͘͠͸ ͔ΒIIDʹαϯϓϦ ϯά͞Ε͍ͯΔͱߟ͑Δɽ ·ͨ

    ͷ΋ͷ͸୯ʹ ΍ ͔Βੜ੒͞Ε͍ͯΔͱߟ͑Δ s x, s, y P(x, y) P(s, y) nU P(x) P(s) ४උ ฼ूஂ
  6.  άϧʔϓ͝ͱͷࢦඪ ɹ
 Ͱߟ͑Δ ༧ଌϞσϧͷਖ਼ޡ ެฏੑࢦඪ ͱͨ͋͠ͱMCMCαϯϓϦϯάʹΑͬͯࣄޙ෼෍ Λ֫ಘ ҰԠਪఆͰ͖Δ͕ɼσʔλ਺ʹਫ਼౓͕ґଘ͢Δͷ͕໰୊ θg

    = P( ̂ y = 1|y = 1,g) θg ∼ Beta(αg , βg ) αg = βg = 1 Ii = I( ̂ (yi = yi ),1 ≤ i ≤ nL Ii ∼ Bernoulli(θg ) Δ = θ1 − θ0 P(Δ|DL ) ఏҊख๏: ४උ Beta-Binomial Estimation
  7.  ϥϕϧͳ͠σʔληοτ ʹରͯ͠είΞ ΋͠Ϟσϧ͕׬શʹΩϟϦϒϨʔγϣϯ͞Ε͍ͯΔͳΒείΞΛ ͦͷ··༧ଌʹ༻͍Δ͜ͱ͕Ͱ͖Δ Ͱάϧʔϓ͝ͱͷධՁࢦඪΛఆٛͰ͖Δ nU sj = PM

    (yj = 1|xj ) ̂ θg = (1/nU,g)∑ j∈g sj I (sj ≥ 0.5) + (1 − sj) I (sj < 0.5) ఏҊख๏: ४උ Leveraging Unlabeled Data with a Bayesian Calibration Model
  8.  ϥϕϧͷͳ͍αϯϓϧͷείΞͱϥ ϕϧ෇͖ͷσʔληοτΛ૊Έ߹Θ ͤΔ͜ͱͰ, ͷਪఆΛߦ͏͜ͱ͕Ͱ ͖Δɽ θt g θt g

    = 1 nL,g + nU,g ∑ i:i∈a I ( ̂ yi = yi) + ∑ i⋅j∈a zt j ఏҊख๏ άϥϑΟΧϧϞσϧ
  9.  Ϟσϧʹద༻͢ΔͨΊʹ࣍ࣜͰਖ਼ղϥϕϧ͕ੜ੒͞ΕΔͱԾఆ άϧʔϓ͝ͱͷύϥϝʔλ͸ͦΕͧΕڞ௨ͷ෼෍͔Βੜ੒͞ΕΔ ϋΠύϥ ͸੾அਖ਼ن෼෍͔Βੜ੒ yi ∼ Bernoulli (f (si

    ; agj , bqi , cgi )) logag ∼ N(μa , σa ), logbg ∼ N(μb , σb ), logcg ∼ N(μc , σc )whereπ = {μa,b,c , σa,b,c } π ఏҊख๏ ΩϟϦϒϨʔγϣϯؔ਺ͷϞσϧԽ