A recent study conducted by researchers at Stanford Law School has revealed significant disparities in the responses of chatbots based on the racial and gender connotations of user names. For example, a chatbot may suggest a lower salary for a job candidate with a name that sounds African American compared to a candidate with a name that sounds Caucasian. The study examined various scenarios, including purchasing decisions, chess matches, public office predictions, sports rankings, and hiring advice, and found biases that were generally disadvantageous to Black individuals and women. The researchers concluded that these biases are a systemic issue within AI chatbots and reflect the stereotypes encoded in the training data.

The study highlights the importance of addressing these biases and raises concerns as businesses increasingly rely on artificial intelligence in daily operations. The research suggests that AI companies should recognize the existence of biases and continuously test their models to mitigate harmful outputs. However, the study also acknowledges the potential need for tailored advice based on socio-economic factors, as long as it is approached in a desirable and informed manner.