The Algorithm Thinks You're Suspicious: How AI Perpetuates Racial Profiling

---

Let me tell you about Robert Williams.

In January 2020, Robert was standing in his front yard in Farmington Hills, Michigan, when police pulled up and arrested him. In front of his wife. In front of his two young daughters. In plain view of his neighbors.

His crime? Looking like someone he wasn't.

A facial recognition system had matched his expired driver's license photo to grainy surveillance footage of a man stealing watches from a store in Detroit. The algorithm said it was him. The police believed the algorithm. Robert spent 30 hours in a dirty, overcrowded jail cell for a crime he didn't commit.​

Here's the kicker: Robert Williams was at his desk at an automotive supply company when the theft occurred. He was nowhere near the store. The algorithm was just... wrong.
And Robert wasn't the last.

Welcome to Algorithm Ethics Wednesday, where we're talking about something that should terrify every single one of us: AI systems are automating racial profiling at an unprecedented scale, and they're hiding behind the myth of technological objectivity to do it.

---

"Objective" Algorithms Are Anything But

Here's the lie we've been sold: computers are neutral. Algorithms don't have feelings. AI can't be racist because it's just math.

This is, to put it technically, complete and utter garbage.

AI systems learn from data. And that data? It's created by humans. In a society with centuries of documented racial bias. Using information collected by systems with well-established patterns of discrimination.

The NAACP puts it plainly: "Communities of color, and the Black community in particular, are disproportionately affected by law enforcement. We face higher rates of surveillance, stops, and arrests, which will only increase due to biased algorithmic predictions."​

The algorithm isn't neutral. It's just laundering human bias through code and calling it science.

---

Predictive Policing: Pre-Crime Is Here (And It's Racist)

Remember that movie Minority Report, where police arrested people BEFORE they committed crimes? We're doing that now. It's called predictive policing. And it's exactly as dystopian as it sounds.

What Is Predictive Policing?

Predictive policing uses algorithms to analyze historical crime data and predict where crimes will happen or who will commit them. Police then deploy resources to those "high-risk" areas or surveil those "high-risk" individuals.​

Sounds scientific, right? Sounds efficient. Sounds like the future of law enforcement.

Except it's built on a foundation of garbage data that perpetuates exactly the discrimination it claims to eliminate.

The Feedback Loop From Hell

Here's how the cycle works, explained by the National Academies:

1. Historically, Black and brown neighborhoods have been over-policed

2. More police presence = more arrests (even for minor things white neighborhoods get away with)

3. More arrests = more crime data

4. Algorithm sees more crime data in these neighborhoods

5. The algorithm predicts more crime in these neighborhoods

6. Police deploy MORE resources to these neighborhoods

7. More police presence = more arrests

8. Repeat forever​

The algorithm isn't predicting where crime happens. It's predicting where police have historically LOOKED for crime. Those are very different things.

Chicago's Strategic Subject List: A Case Study in Algorithmic Racism

Chicago's Strategic Subject List (SSL), also called the "Heat List," was supposed to be the future of violence prevention. The algorithm analyzed historical arrest records to identify individuals at risk of being involved in gun violence, either as victims or offenders.​

Here's what actually happened:

The list exploded in size. What was supposed to be a targeted list of high-risk individuals eventually included every single person arrested or fingerprinted in Chicago since 2013.​

It overwhelmingly targeted Black and Latino men. A study found the SSL disproportionately flagged young Black men under 20 as "high-risk," many of whom had never committed a violent crime.​

It didn't reduce crime. Research found the SSL had no significant impact on reducing crime in high-crime districts.​

It increased unconstitutional stops. Officers were told they could stop and interview individuals solely because they were on the list, even without reasonable suspicion of a crime. That's unconstitutional.​

It made things worse. People on the list were 2.07 times more likely to be found NOT GUILTY of all charges. The algorithm wasn't identifying criminals. It was identifying people who would be wrongly accused.​

The program was shelved in January 2020. But the damage was done, and similar programs continue operating across the country.​

The Real Impact

According to research cited by the National Academies, place-based predictive policing algorithms consistently recommended greater police scrutiny of Black and Latino residents than White residents. The number of predictions increased as the Black and Latino proportion of the population increased, and in areas where low-income households or public housing were concentrated.​

This isn't preventing crime. This is automating redlining.

---

Facial Recognition: The Technology That Can't See Black People (Correctly)

If predictive policing is about where crime happens, facial recognition is about who did it. And it turns out, this technology has a massive accuracy problem that falls almost entirely on Black faces.

The NIST Study That Should Have Changed Everything

In 2019, the National Institute of Standards and Technology (NIST) conducted the most comprehensive study of facial recognition bias ever done. They tested 189 algorithms from 99 developers, representing a majority of the industry.​

The findings were damning:

False positives were 10 to 100 times higher for Asian and African American faces compared to White faces.​

The highest rates of false positives were found for African American women. NIST specifically noted this is "particularly important because the consequences could include false accusations."​

Many algorithms picked incorrect images among Black women at significantly higher rates than any other demographic.​

Native Americans experienced the highest rates of false positives among algorithms developed in the US.​

False positives were 2 to 5 times higher in women than men across most algorithms.​

Let me translate that: if you're a Black woman, facial recognition is up to 100 times more likely to incorrectly identify you as someone else. And that "someone else" might be a criminal suspect.

Why Does This Happen?

The algorithms are trained on datasets that are predominantly white and male. Digital cameras can also fail to provide the color contrast needed for accurate face prints from photos of darker-skinned people.​

As the Michigan Law School's Civil Rights Litigation Initiative explains: "The flaws in the technology are especially pronounced when it is used to identify people of color because most of its algorithms are built by analyzing a data set consisting primarily of white faces."​

The AI literally learned what faces look like from white people, and now it struggles to tell Black people apart.

The Human Cost: Real People, Real Arrests, Real Trauma

Let's go back to Robert Williams for a moment.

When police arrested him, his photo from an expired driver's license was identified as the ninth-most-likely match to the surveillance footage. Not the first. The NINTH. And his updated license photo didn't even come up at all.​

But police decided to focus on Williams anyway. They put his photo in a lineup shown to a loss-prevention contractor who wasn't even at the store on the day of the theft. She identified Williams as the closest resemblance based on the same poor-quality footage that generated the wrong match in the first place.​

Robert Williams' case was the first publicly reported wrongful arrest due to facial recognition. But it wasn't the last:

Porcha Woodruff (2023): A 32-year-old, heavily pregnant Black woman in Detroit was arrested for allegedly participating in a carjacking. She was eight months pregnant. The actual suspect was never described as visibly pregnant. She spent 11 hours in jail.​

Nijeer Parks (2019): An innocent Black man in New Jersey spent TEN DAYS in jail because facial recognition matched him to a crime. Any adequate investigation would have revealed he had no connection to the crime.

Michael Oliver: Another Black man wrongfully arrested by Detroit police based on faulty facial recognition.​

At least eight Americans have been wrongfully arrested after facial recognition misidentifications. All of them were Black.

What The Wrongfully Arrested Say

Robert Williams testified: "In my case, as in others, the police did exactly what the law would require them to do, but it didn't help. Once the facial recognition software told them I was the suspect, it poisoned the investigation. This technology is racially biased and unreliable and should be prohibited."​

The technology doesn't just fail. It actively leads investigators AWAY from the truth by giving them false confidence in the wrong suspect.

---

ShotSpotter: Surveilling Black Neighborhoods With Microphones

If you haven't heard of ShotSpotter, here's the elevator pitch: it's a network of microphones installed throughout a city that supposedly detects gunshots and alerts police to the location.

Sounds useful, right? Faster response times to shootings could save lives.

Here's the problem: where those microphones are placed, how accurate they actually are, and what happens when police respond to alerts.

Where The Microphones Are

A data leak in 2024 exposed the locations of ShotSpotter's nationwide network of more than 25,000 microphones. And the results confirmed exactly what civil rights advocates had been saying: ShotSpotter sensors are disproportionately located in communities of color.​

In Milwaukee, sensors are present exclusively on the North and South sides of the city, in predominantly Black and Latinx communities. ShotSpotter is nowhere to be found in majority-white neighborhoods.​

In New York City, Black and Latine residents make up two-thirds of the people who live in areas surveilled by ShotSpotter.​

Black residents in NYC live in the same police precinct as 93% more unconfirmed alerts than the citywide average. Neighborhoods with predominantly Black residents are 3.5 times more likely to have an officer deployed based on an unconfirmed alert than a neighborhood with predominantly white residents.

The Accuracy Problem

SoundThinking (the company that makes ShotSpotter) claims 97% accuracy and a false positive rate of only 0.5%.​

But here's what the Chicago Office of Inspector General found when they actually analyzed the data:

Of 50,176 confirmed and dispatched ShotSpotter alerts, only 4,556 (9.1%) resulted in documented evidence of a gun-related criminal offense.​

Let me say that again: NINETY PERCENT of ShotSpotter alerts in Chicago did not result in any evidence of actual gunfire.

Brooklyn Defenders' analysis of NYPD data found "extremely low confirmation rates throughout its history."bds​

The ACLU of Massachusetts found that nearly 70% of ShotSpotter alerts in Boston went unconfirmed.​

So we have a technology that:

• It is deployed almost exclusively in Black and brown neighborhoods

• Has a 90% rate of alerts that produce no evidence of gunfire

• Sends armed police into those communities dozens of times per day, expecting to confront a dangerous situation

What could possibly go wrong?

What Actually Goes Wrong

In Chicago, police responded to more than 60 ShotSpotter alerts per day. Most of them for nothing.​

But those "nothing" responses aren't actually nothing. They're armed officers entering Black and brown neighborhoods on high alert, expecting gunfire. In communities that already have traumatic relationships with police. In a country where unarmed Black people are killed during routine police encounters.

And sometimes, the consequences are deadly.

The Chicago Office of Police Accountability confirmed that an erroneous ShotSpotter report led police to shoot at a boy who was lighting fireworks in a backyard.​

A child. Lighting fireworks. Shot at by police. Because a microphone thought it heard something.

Brooklyn Defenders put it plainly: "Like so many of NYPD's other massively expensive and invasive technologies, ShotSpotter is an engine for over-policing that leads to an influx of police in Black and Latine neighborhoods based on false gunshot alerts."

---

The Pattern: Technology As A Shield For Discrimination

Let's zoom out for a second.

Predictive policing uses historical crime data that reflects decades of discriminatory policing to predict where to deploy MORE police, creating a feedback loop that compounds discrimination.

Facial recognition uses algorithms trained primarily on white faces, making them dramatically worse at identifying Black faces, leading to wrongful arrests of innocent Black people.

ShotSpotter places surveillance microphones almost exclusively in Black and brown neighborhoods, sending armed police on high alert into those communities dozens of times daily based on alerts that are wrong 90% of the time.

Do you see the pattern?

These technologies don't reduce bias. They AUTOMATE it. They give racist outcomes the veneer of scientific objectivity. They allow institutions to say "the algorithm decided" instead of taking responsibility for discriminatory decisions.

As the UN's Special Rapporteur on racism noted: "Predictive policing can exacerbate the historical over-policing of communities along racial and ethnic lines. Because law enforcement data reflects historical patterns of discriminatory policing, predictive tools trained on this data will project those patterns into the future."​

The computer didn't decide. The computer reflected the biases of everyone who built it, trained it, and chose where to deploy it.

---

Why This Matters More Than You Think

If you're not Black or brown, you might be reading this thinking, "That's terrible, but it doesn't affect me."

Here's why you're wrong:

1. It affects everyone eventually. The same technologies being tested on marginalized communities will expand. Predictive policing doesn't stop at race. It can incorporate economic status, social media activity, location data, and more. Today, it's Black neighborhoods. Tomorrow, it could be your neighborhood based on some other factor.

2. It erodes everyone's rights. When we accept unconstitutional surveillance of some communities, we weaken constitutional protections for all communities. Rights that aren't defended universally aren't really rights.

3. It costs everyone money. ShotSpotter contracts cost millions of dollars. Chicago alone has spent huge sums on a technology that produces zero evidence 90% of the time. That's taxpayer money that could fund education, healthcare, or violence prevention programs that actually work.

4. It makes everyone less safe. When police are chasing false ShotSpotter alerts and investigating innocent people based on bad facial recognition matches, they're not solving actual crimes. The real perpetrators remain free while innocent people are traumatized.

What The Research Actually Shows Works

Here's what Brooklyn Defenders recommends instead of ShotSpotter: "education, health, poverty reduction, curb-violence and community-based programs, and other resources."​

Investing in communities reduces crime. Surveilling communities with flawed technology does not.

---

What You Can Do About This

Okay, this is depressing. But you're not powerless. Here's how to push back:

1. Know What's Deployed In Your City

Does your city use:

• Predictive policing software?

• Facial recognition technology?

• ShotSpotter or similar gunshot detection?

• Gang databases or risk scoring systems?

File public records requests. Attend city council meetings. Demand transparency.

2. Support Bans And Regulations

Several cities have banned or restricted facial recognition use by police. Support these efforts. The ACLU and other civil rights organizations are actively fighting for bans and strict regulations.

3. Fund The Alternatives

Support community-based violence intervention programs. Support funding for education, housing, healthcare, and economic development in over-policed communities. These actually reduce crime. Surveillance doesn't.

4. Demand Accountability

When wrongful arrests happen, support the victims' lawsuits. Demand policy changes. Robert Williams' settlement with Detroit required policy reforms, but it took years of fighting. The more pressure, the faster change happens.

5. Talk About This

Share this post. Have conversations. Most people have no idea these technologies exist or how biased they are. Awareness is the first step to action.

---

The Real Data: Accuracy Rates By Demographic Group

Here's a summary of what the research shows:

This isn't anecdotal. This is documented. This is measured. This is undeniable.

---

TL;DR (Because This Is Heavy)

• Predictive policing uses historically biased crime data to predict future policing, creating a feedback loop that compounds discrimination against Black and brown communities, National Academies​

• Facial recognition has false positive rates up to 100x higher for Black faces than white faces, with Black women having the worst accuracy rates.​

• At least eight people have been wrongfully arrested due to facial recognition errors. All of them were Black.​

• ShotSpotter places surveillance almost exclusively in Black and brown neighborhoods, with 90% of alerts in Chicago producing zero evidence of gunfire.​

• These technologies don't reduce bias. They automate it and hide it behind the myth of algorithmic objectivity

• Alternatives that actually work: education, healthcare, economic development, and community-based violence intervention. ​

---

The Uncomfortable Truth

Here's what I need you to understand: AI isn't making policing more objective. It's making discrimination more efficient.

Every wrongful arrest. Every unconstitutional stop. Every armed response to a false alert in a Black neighborhood. These aren't bugs in the system. They're features of a system that was built on biased data by teams that didn't prioritize equity and deployed by institutions with long histories of discrimination.

The algorithm thinks Black people are suspicious because the data it learned from was generated by a society that thinks Black people are suspicious.

Garbage in, garbage out. Racism in, racism out.

The only difference is now there's a computer to blame instead of a human being. And somehow, that makes it okay.

It's not okay.

---

That's Algorithm Wednesday

You just learned:

• How predictive policing creates feedback loops that compound discrimination

• Why facial recognition fails dramatically for Black faces, especially Black women

• How ShotSpotter surveils Black neighborhoods with a technology that's wrong 90% of the time

• The real human cost: innocent people arrested, jailed, traumatized

• What you can actually do about it

This isn't about being anti-technology. I'm a software engineer. I love technology. But I love justice more.

And technology that automates injustice isn't innovation. It's oppression with better PR.

Drop a comment: Did you know about any of this? Does your city use these technologies? What questions do you have?

See you Friday for Code Confessional, where we'll talk about something less heavy. You've earned it after this one.

Stay aware. Stay angry. Stay active.

✊🏾💻⚖️