Balancing Bias Mitigation and Data Protection in AI-Driven Healthcare

Insights from the European Health Data Space, AI Act and GDPR

Abstract

This paper examines the regulatory tensions between algorithmic bias mitigation and data protection in AI-driven healthcare within the European Union’s legal framework. Through analysis of the European Health Data Space Regulation, AI Act, and General Data Protection Regulation, the study reveals a fundamental paradox: while the EHDS promotes data anonymization for secondary use, effective bias detection in high-risk AI systems often requires access to the very demographic data that anonymization obscures. The research highlights documented cases of algorithmic bias in healthcare, including racial disparities in skin cancer diagnosis and gender biases in heart attack prediction systems, demonstrating the practical importance of this regulatory challenge. The findings illustrate how the EHDS’s opt-out mechanism may disproportionately exclude vulnerable populations from datasets, further compromising representativeness. This study contributes to the discourse by identifying an “identification paradox” where data protection measures may inadvertently perpetuate algorithmic discrimination. The paper concludes by proposing potential regulatory and technical approaches to reconcile privacy protection with algorithmic fairness, ensuring healthcare AI systems can deliver equitable outcomes while respecting fundamental rights to data protection.