Unconscious Bias: Is AI dividing us?

Bad AI is becoming a major headache for organizations. Tech is a male-dominated sphere, which means that it produces, inherently, male-skewed AI driven by unconsciously biased datasets. The effects of this can be measurable. Run through the same AI, women can receive worse credit or loan agreements than their male counterparts, be pushed out from job openings, receive worse medical treatment, or even receive performance penalties for doing the same work as men to the same standard. So how has this situation emerged and, more importantly, what can be done about it?


In this episode, we speak to Erin Young, research scientist from the Alan Turing Institute, who are dedicated to solving societal problems using technology. Their research has found deep structural inequalities in the field of AI, including higher attrition rates for women, who are generally filling lower paid, less prestigious jobs than their male counterparts. 


That's having a tangible, real-world effect. Anjana Susarla is a professor in Responsible AI from the University of Michigan. She's been tracking instances of biased AI finding its way into society, including documented cases of women in common-property states where spouses incomes and assets are joined being given lower credit limits on cards than their male counterparts. She also documents several cases of poor AI decision making in AI-assisted hiring and HR systems.


So should these systems be using AI at all? Well, Ivana Bartoletti argues that sometimes, AI isn't the answer. She's the Global Chief Privacy Officer at WiPro, and an expert on bias in AI. She notes several cases where institutional bias has been backed up by AIs which reflected existing societal pre-conceptions, for example in AI giving lower exam scores to pupils from poorer backgrounds in the UK, and lower state benefits to migrants in the Netherlands.


So what should be done? HPE's Chief Technology Officer Fidelma Russo argues that, as project leaders and managers, a lack of diversity in AI and the creeping problems it's causing should have been identified by the industry some time ago. She says drastic change is now needed to fix the problem. Fortunately, it's one the industry is rapidly becoming aware of and is now at pains to fix.