Insurers are required to ensure that their utilization of consumer data and artificial intelligence (AI) is non-discriminatory, as outlined in a memo from the state Department of Financial Services dated July 11. The memo asserts that insurers should refrain from incorporating external data or AI systems without conducting a comprehensive assessment to confirm that they do not unfairly discriminate between individuals in similar situations. In the event that the assessment reveals a disproportionate impact on a particular demographic group, insurers must strive to find a less biased alternative. This directive, issued through a circular letter, is applicable to approximately 1, 960 companies regulated by the DFS, including life, property and casualty insurers, as well as certain health insurers. DFS Superintendent Adrienne Harris stated that this guidance aims to prevent the perpetuation or amplification of systemic biases, which have resulted in unlawful or unfair discrimination, while also safeguarding the market's stability. Consumer Reports, a consumer rights group, pointed out that traditional analysis methods can also be discriminatory. However, the risks become more pronounced when companies employ opaque algorithms to analyze vast amounts of data and automate decisions. AI systems may be fed with incomplete, inaccurate, or unrepresentative data, or data derived from biased practices that are no longer valid. For example, life insurers have acknowledged using customers' body mass index (BMI), a measure of body weight and height ratio.

However, the American Medical Association has advised against relying solely on BMI since its development predominantly relied on data from white patients. Hence, insurers may need to reassess how they incorporate BMI into their practices, according to Chuck Bell, Advocacy Programs Director at Consumer Reports. The DFS' approach seems to be appropriately flexible, according to Eric Linzer, President and CEO of the New York Health Plan Association, which represents health insurers. While AI is not commonly employed for underwriting and pricing health plans, it has been useful in detecting fraudulent claims and identifying gaps in care. Linzer expressed support for a framework that enables health plans to develop suitable tools that enhance, rather than replace, human decision-making and expertise. He also highlighted that AI holds the potential to improve the efficiency and affordability of the healthcare system.