Impossibility between Fairness Criteria in Machine Learning

- Many formal fairness criteria have been developed to fit for various kinds of contexts
- The impossibilities between fairness criteria means that almost all these criteria cannot be satisfied simultaneously
- Some AI regulations mistreat the impossibilities

Transcript

1. Impossibility between Fairness Criteria in Machine Learning Toshihiro Kamishima Independent Researcher, Japan NRC Canada/AMII - JST “Trusted AI Quality Systems” ICML co-located session Seoul, South Korea, 2026-07-05 1
2. Who am I? 2 Toshihiro Kamishima 1994 — 2024 Advanced Industrial Science and Technology, Japan 2022 ECMLPKDD Test of Time Award Research Interests 1994 — Recommender Systems, Ranking 2010 — Fairness in Machine Learning 2012 firstly developed “fairness-aware recommender systems” [Moritz Hardt’s homepage]
3. Outline 3 Fairness-Aware Machine Learning Data analysis taking into account potential issues of fairness. It maintains the influence of these types of sensitive information: to enhance social fairness (gender, race,…) restricted by law or contracts (insider or private information) Many formal fairness criteria have been developed to fit for various kinds of contexts The impossibilities between fairness criteria means that almost all these criteria cannot be satisfied simultaneously Some AI regulations mistreat the impossibilities
4. Formal Fairness 4
5. Formal Fairness 5 In fairness-aware machine learning, we maintain the influence: sensitive information target / objective gender race other socially sensitive info university admission credit scoring crick-through rate Formal Fairness The desired condition defined by a formal relation between sensitive feature, target variable, and other variables in a model Influence How to related these variables Which set of variables to be considered What states of sensitives or targets should be maintained
6. Notations of Variables 6 target variable / object variable An objective of decision making, or what to predict Ex: loan approval, university admission, what to recommend = observed / true, = predicted sensitive feature To ignore the influence to the sensitive feature from a target Ex: socially sensitive information (gender, race), items’ brand Specified by a user or an analyst depending on his/her purpose It may depend on a target or other features non-sensitive feature vector All features other than a sensitive feature Y Y ̂ Y S X
7. Type of Formal Fairness 7 association-based fairness fairness-criteria defined based on statistical association, namely correlation and independence counterfactual fairness causal effect of the sensitive information to the outcome economics-based fairness using a notion of a fairness in game theory or econometrics, such as Gini index In this talk, we concentrate on association-based fairness, and discuss the impossibility between formal fairness criteria
8. Fairness Criteria of Association-based Fairness 8
9. Association-Based Fairness 9 fairness through unawareness statistical parity equalized odds sufficiency Ŷ ⫫ S | X Ŷ ⫫ S Ŷ ⫫ S | Y Y ⫫ S | Ŷ awareness unaware aware unit individual group wordview WAE WYSIWYG comments treat like cases alike alias: situation testing equality of outcomes alias: demographic parity, independence equality of false positive and false negative rates alias: separation equality of positive and negative predictive values
10. Fairness through Unawareness 10 Fairness through Unawareness: Prohibiting to access individuals' sensitive information during the process of learning and inference In a sense that sensitive information is not collected, this framework protects the privacy of individuals Pr[ Ŷ | X, S ] A unfair model is trained from a dataset including sensitive and non-sensitive information Pr[ Ŷ | X ] A fair model is trained from a dataset eliminating sensitive information An unfair model, Pr[ Ŷ | X, S], is replaced with a fair model, Pr[ Ŷ | X ] Pr[ Ŷ, X, S ] = Pr[ Ŷ | X, S] Pr[ S | X ] P[ X ] Pr[ Ŷ | X ] Pr[ S | X ] Pr[ X ] 'BJSOFTT
    UISPVHI
    6OBXBSFOFTT Ŷ ⫫ S | X
11. Individual Fairness 11 Individual Fairness: Implementation of the principle of “Treat like cases alike” such as by Aristotle Distributions of a target variable are equal for all possible sensitive groups given a specific non-sensitive values Pr[ Ŷ | S, X=x ] = Pr[Ŷ | X=x], ∀x ∈ Dom(X) Ŷ ⫫ S | X Conditioning fairness criteria by can be considered as individual fairness X Individual fairness happens to coincide with fairness through unawareness = fairness through unawareness (privacy protection) Ŷ ⫫ S | X individual fairness Ŷ ⫫ S | X
12. Pr[Ŷ=1 | S=s, X=0] Pr[Ŷ=0 | S=s, X=0] Pr[X=0] Pr[X=1] Pr[Ŷ=1 | S=s, X=1] Pr[Ŷ=0 | S=s, X=1] S=0 S=1 S=0 S=1 Ŷ=1 S=0, X=0 Ŷ=1 S=1, X=0 Ŷ=1 S=0, X=1 Ŷ=1 S=1, X=1 Ŷ=0 S=0, X=0 Ŷ=0 S=1, X=0 Ŷ=0 S=0, X=1 Ŷ=0 S=1, X=1 a kind of procedural fairness Fairness through Unawareness Fairness through Unawareness 12 Ŷ ⫫ S ∣ X Pr[Ŷ, S ∣ X] = Pr[Ŷ ∣ X] Pr[S ∣ X] These gaps indicate unfair decision A learned model directly access sensitive information
13. Statistical Parity / Independence 13 Ratio of predictions must be proportional to the sizes of sensitive groups Statistical Parity / Independence: ̂ Y ⫫ S equality of outcome: Goods are distributed by following pre- specified procedure In a context of FAML, the predictions are distributed so as to be proportional to the sizes of sensitive groups [Calders+ 10, Dwork+ 12] Hazelwood School District v. United States, 433 U.S. 299 (1977) Gross Statistical Disparity: Discrimination in employment is determined whether the ratio of protected and non-protected groups of employees is diverged from the corresponding ratio in general population
14. equality of outcome Statistical Parity / Independence Pr[Ŷ=1 | S=0] Pr[Ŷ=0 | S=0] Pr[S=0] Pr[S=1] Pr[Ŷ=1 | S=1] Pr[Ŷ=0 | S=1] Ŷ=1 S=1 Ŷ=0 S=1 Ŷ=0 S=0 Ŷ=1 S=0 Statistical Parity / Independence 14 [Calders+ 10, Dwork+ 12, Barocas+ 19] Ŷ ⫫ S Pr[Ŷ, S] = Pr[Ŷ] Pr[S] This gap indicates unfair decision Ratios between positives and negatives in prediction should be matched among all sensitive groups
15. Impossibility between Fairness Criteria 15
16. Satisfiablity between Fairness Criteria 16 statistical parity Ŷ ⫫ S equality of outcome sufficiency Y ⫫ S | Ŷ Calibrating predictive values fairness through unawareness Ŷ ⫫ S | X unaware sensitive info equalized odds Ŷ ⫫ S | Y Calibrating prediction errors to observation group fairness mutually exclusive criteria simultaneously satisfiable criteria [Žliobaitė+ 16] [Kleinberg+ 16, Chouldechova 17]
17. Fairness through Unawareness & Statistical Parity 17 Ŷ X S S ⫫ X Ŷ ⫫ X It is impossible for AI & humans to satisfy fairness through unawareness and statistical parity simultaneously We must select one of these criteria to satisfy [Žliobaitė+ 16] Satisfying fairness through unawareness, S ⫫ Ŷ | X To simultaneously satisfy statistical parity, S ⫫ Ŷ, a condition of S ⫫ X OR Ŷ ⫫ X must be satisfied S ⫫ X: a sensitive feature and non-sensitive features are independent unrealistic X and are uncontrollable, and X is high-dimensional Ŷ ⫫ X: a sensitive feature and a target variable are independent meaningless Ŷ must be random guess S
18. How the impossibility influences on Laws and Regulations 18
19. Privacy can conflict with Fairness 19 Some privacy regulations prohibit to collect sensitive information Fairness criterion that can be satisfied by hiding sensitive information is only fairness through unawareness To satisfy all other types of fairness conditions, sensitive information must be collected Unfortunately, some regulations mistreat the impossibility
20. EU Regulation 20 The new AI Act overrides the GDPR in the Article 10, 5(f), and the sensitive can be collected for removing biases Article 10, 5 (f) The records of processing activities pursuant to Regulations (EU) 2016/679 and (EU) 2018/1725 and Directive (EU) 2016/680 include the reasons why the processing of special categories of personal data was strictly necessary to detect and correct biases, and why that objective could not be achieved by processing other data. The conflict was resolved The EU GDPR demands to protect the privacy, and prohibits the collection of sensitive information This regulation conflicts with fairness enhancement because it is mathematically impossible to satisfy these constraints simultaneously
21. Regulations in Japan 21 Japanese AI lawʰਓ޻஌ೳؔ࿈ٕज़ͷݚڀ։ൃٴͼ׆༻ͷਪਐʹؔ͢Δ๏ ཯ʱKVTU
    outlines the basic policy AI providers’ guideline『AI 事業者ガイドライン ver. . 』is provided note : 潜在的なバイアスの例として、⼈種や性別等の明らかなセンシティブ属性 を取り除いたデータであっても、関連する情報からセンシティブ属性が類推されて しまう場合や、複数の属性の組み合わせによってバイアスが⽣じてしまう場合（交 差バイアス）などがある。 removing sensitive information satisfying fairness through unawareness It is impossible to satisfy the condition without observing sensitive information bias under statistical parity This note seems to request satisfying statistical parity without referring sensitive information
22. Canadian Regulations 22 In the bill C-36, Protecting Privacy and Consumer Data Act (PPCDA) is currently discussed, and it regulates the data privacy It is allowed to recover sensitive information from anonymized data for testing and checking fairness, but in the inference phase, it is not allowed It is impossible to satisfy fairness criteria except for fairness through awareness. https://www.legal500.com/developments/thought-leadership/canada-tables-bill-c-36-the-protecting-privacy-and-consumer- data-act/ I’m not familiar with Canadian AI regulations, so I asked Gemini. So the information in this page might not be correct
23. Conclusion 23 Several types of formal fairness criteria have been proposed to fit for various contexts It is impossible to satisfy these criteria simultaneously in general Some regulations mistreats the impossibility My tutorial slide about fairness-aware machine learning https://www.kamishima.net/faml/