Before the Algorithm, What’s in the Imagination?

Algorithms are becoming pervasive in our daily lives, from recommending what movies we should watch to determining who should be approved for loans or certain medical care.

But algorithms suffer from inherent bias due to human imagination, or lack thereof. Algorithmic systems are representative of the person creating them and based on presumed relationships. As a result, University of Florida College of Journalism and Communications Associate Professor Jasmine McNealy explains that it’s critical to mitigate the potential harm human intervention can have on developing algorithms by asking essential questions.

Algorithm designers and the model they use can cause serious repercussions. At least one lawsuit for wrongful arrest and imprisonment has been filed partly because of Detroit’s facial recognition technology used to identify individuals. Another example is California’s facial recognition system to verify unemployment claims during the COVID-19 pandemic, which left 1.1 million people scrambling, having been given about 30 days to verify their identity or lose their benefits. The algorithm, and its developers, failed to adequately consider darker-skinned individuals during facial recognition, a failure to exhibit imagination.

McNealy explains that whether intentional or not, implicit systemic racism and sexism influence algorithm creators. This is especially true in healthcare settings where race has been used as a critical factor in algorithmic decisions such as placement on the transplant list.

Moving forward, algorithm designers must ask themselves:

• What is this supposed to solve?
• What or who is this product/service supposed to look like?
• Who is/is not supposed to be here?
• What is the human impact of these kinds of tools?

According to McNealy, future research should include continuous evaluation and auditing of algorithmic or decision systems. Additionally, state and federal legislators are becoming involved in discussions regarding facial recognition and AI as the government acknowledges the significant and harmful impact algorithm designs can have and who needs to be held accountable.

Other articles by Jasmine McNealy on inherent algorithmic bias

• There is No Trolley Problem
• Amplifying “Otherness”

The original article, “Before the Algorithm, What’s in the Imagination?”, was published in the May – June 2022 edition of Interactions.

This summary was written by Dana Hackley, Ph.D.

Posted: October 4, 2022
Tagged as: AIatUF, Algorithms, Artificial Intelligence, Jasmine McNealy