Google's New Causal Framework Sheds Light on Subgroup Fairness in Machine Learning

Understanding Subgroup Fairness in Machine Learning

Evaluating fairness in machine learning involves analyzing model performance across different subgroups defined by attributes such as race, gender, or socioeconomic status. This is particularly critical in sensitive domains like healthcare, where biased model outcomes can cause disparities in treatment or diagnosis. Subgroup-level analysis helps identify hidden biases in data or model design. Fairness is not just about equal statistics but about ensuring equitable real-world outcomes.

Challenges with Data Distribution and Structural Bias

Model performance differences across subgroups often reflect underlying disparities in data distributions rather than intrinsic model bias. These distribution differences may stem from broader social and structural inequities influencing training and evaluation data. Demanding equal performance across subgroups without considering these factors can lead to misinterpretations. Moreover, non-representative training data due to sampling bias or structural exclusions can cause poor generalization and increased disparities, especially when bias structures are unknown.

Limitations of Traditional Fairness Metrics

Common fairness metrics include disaggregated statistics and conditional independence tests like demographic parity, equalized odds, and sufficiency. For example, equalized odds aim to equalize true and false positive rates across groups. However, these metrics can be misleading under distribution shifts. Variations in label prevalence across subgroups may cause accurate models to fail fairness criteria, incorrectly suggesting bias.

Introducing a Causal Framework for Fairness Evaluation

Researchers from Google and partner institutions proposed a causal graphical model-based framework that explicitly represents data generation processes, subgroup differences, and sampling biases. This approach moves beyond assumptions of uniform distributions and helps interpret subgroup performance variations more rigorously. The framework combines traditional disaggregated metrics with causal reasoning, encouraging deeper understanding of disparity sources rather than relying solely on metric comparisons.

Modeling Different Types of Distribution Shifts

The framework uses causal directed acyclic graphs to categorize shifts such as covariate shift, outcome shift, and presentation shift, incorporating variables like subgroup membership, outcomes, and covariates. Covariate shift occurs when feature distributions differ but the outcome-feature relationship remains stable. Outcome shift happens when this relationship varies by subgroup. The model also accounts for label shift and selection biases affecting subgroup data. These distinctions help predict when subgroup-aware modeling improves fairness and when it might be unnecessary. This systematic categorization clarifies when standard fairness evaluations are valid or misleading.

Empirical Findings

The researchers tested Bayes-optimal models under various causal conditions to see when fairness criteria like sufficiency and separation hold. Sufficiency (Y independent of subgroup A given model output) held under covariate shift but not under more complex shifts. Separation (model output independent of subgroup given outcome) was valid only under label shift without subgroup input. These results emphasize the importance of subgroup-aware models in practice. Additionally, when selection bias depends solely on variables like features or subgroup, fairness criteria can still be met, but bias related to outcomes complicates fairness maintenance.

Implications for Fairness Assessment

Fairness evaluations based solely on subgroup metrics can be misleading due to the influence of data structure. The proposed causal framework provides tools to detect and interpret these complexities, combining statistical fairness with real-world considerations. While it does not guarantee perfect equity, it offers a transparent foundation for understanding how algorithmic decisions impact diverse populations.

For more details, check out the original paper and GitHub repository. Follow related updates on Twitter and join communities like the 100k+ ML SubReddit and newsletters.