A recent study by Yale researchers has shed light on how biases in artificial intelligence (AI) systems used in healthcare can exacerbate existing health disparities among different races and ethnicities in the United States. The review, published in PLOS Digital Health, provides a comprehensive analysis of how biases at various stages of AI development can lead to poor clinical outcomes, particularly for minority communities.
John Onofrey, Ph.D., assistant professor at Yale School of Medicine and senior author of the study, emphasizes the concept of “bias in, bias out” in AI systems. This principle underscores how biases present in the data used to train AI models can result in biased outputs, potentially perpetuating or even amplifying existing health inequities.
The Yale team identified sources of bias at each stage of medical AI development, including training data, model development, publication, and implementation. They provided examples to illustrate the impact of these biases on healthcare outcomes and suggested strategies for mitigation.
One striking example highlighted in the study is the use of race as a factor in estimating kidney function. Previous research has shown that this practice can lead to longer wait times for Black patients to get onto transplant lists. The researchers recommend using more precise measures, such as socioeconomic factors and zip codes, in future algorithms to address this disparity.
James L. Cross, a first-year medical student at Yale School of Medicine and the study’s first author, stresses the importance of incorporating social determinants of health in medical AI models for clinical risk prediction. This approach could help create more equitable and accurate predictive tools.
The study serves as a wake-up call for the medical community, emphasizing the need for vigilance in developing and implementing AI systems in healthcare. As AI continues to play an increasingly significant role in medical decision-making, addressing these biases is crucial to ensure that technological advancements benefit all patients equally, regardless of their racial or ethnic background.
See: “‘Bias in, bias out’: Study identifies bias in medical AI” (November 25, 2024)
