What AI Does: Generate, Rank, Predict, Detect
What Is AI?
Four real systems, each making a different move. One generated. One ranked. One predicted. One detected. Each one hurt someone — and in every case a human was somewhere in the loop, deciding how much to trust the machine. Work the evidence before you name the move.
Coca-Cola Remade Its Beloved 1995 Christmas Ad With AI — The Internet Called It 'Soulless'
In November 2024 Coca-Cola rebuilt its classic 'Holidays Are Coming' advert using four generative-AI models across three studios (Secret Level, Silverside AI, Wild Card). No illustrators, no film crew — the red trucks, the snow, the cheering crowds were all generated. Creatives erupted: the ad was branded 'soulless', critics noted the bitter irony of its 'Real Magic' sign-off, and artists pointed out the models had been trained on human work nobody paid for. Coca-Cola defended it as 'experimentation' — and ran another AI version the next year.
"A machine generated the warmth. That was the problem."
Source ↗Netflix Showed Black Users Different Movie Posters — Built Around Black Actors Who Were Barely In The Film
In October 2018 Black viewers noticed Netflix was serving them artwork stuffed with Black actors who had tiny roles — the comedy 'Like Father' rebuilt around two minor characters, 'Love Actually' repackaged to look like a Chiwetel Ejiofor romance. The system ranked and swapped thumbnails per user to maximise clicks. Netflix denied using race, saying it only used 'viewing history' — but viewing history is a near-perfect stand-in for it. Nobody ever chose which poster they saw.
"The algorithm learned what made you click. It just never told you how."
Source ↗ProPublica: A Tool That Predicts Who Will Reoffend Was Twice As Likely To Wrongly Flag Black Defendants
In 2016 ProPublica obtained COMPAS risk scores for more than 7,000 people arrested in Broward County, Florida, then checked who actually reoffended. Black defendants were nearly twice as likely to be wrongly labelled high-risk — a 44.9% false-positive rate versus 23.5% for white defendants. Judges saw these scores at bail and sentencing. The maker, Northpointe, insisted the tool was 'equally accurate' for both groups — and by their definition of fairness, it was. Both sides were right, using different definitions of the same word.
"The machine only predicted. A judge decided. The judge trusted the machine."
Source ↗Detroit Police Arrested Robert Williams In Front Of His Daughters — A Face-Match Algorithm Picked The Wrong Man
In January 2020 Robert Williams, a Black man, was handcuffed on his own lawn and held more than 30 hours. A detective had run a grainy shop-CCTV still through facial recognition; it surfaced Williams's old driver's-licence photo as a match, and nobody checked further. He is the first person known to be wrongfully arrested in the US because of face recognition — one of three Black men arrested this way by Detroit police. A 2019 NIST study had already found these systems misidentify Black and Asian faces far more often than white ones. Detroit later paid Williams $300,000.
"The machine flagged a face. A human believed it. A father was arrested."
Source ↗In each case: who made the final call — the human or the machine? And who paid when it went wrong? Rank the four cases from 'the human is most to blame' to 'the machine is most to blame', and defend your order.
Mechanism And Stakes
The Machine Doesn't Decide. A Human Does.
In 2016, journalists at ProPublica pulled the risk scores of more than 7,000 people arrested in one Florida county. The scores came from software called COMPAS, which predicts how likely someone is to commit another crime. Judges saw the number when setting bail and sentences.
Here is the thing to hold onto: COMPAS never sent anyone to jail. It produced a number. A judge — a human — read that number and decided. But judges are busy, the number looks objective, and "the computer said high-risk" is a hard thing to argue against. So in practice, the machine's prediction became the decision.
That is the whole game, and it runs the same way in every AI system you will ever meet. Three questions.
One: name the move. Every system does exactly one of four things:
- Generate — makes new content that didn't exist before (the Coca-Cola ad).
- Rank — orders or filters things that already exist (the Netflix posters).
- Predict — estimates a future you can't see yet (COMPAS).
- Detect — decides whether a thing matches a category (the face-match).
Two: find the decision boundary. A machine either recommends — a human can still say no — or it decides, acting on its own while a human only finds out later. The most dangerous systems are the ones in between: where everyone assumes a human is checking, but nobody really is. A judge who rubber-stamps the score. A detective who trusts the match. A spam filter you never open.
Three: name who pays. When the machine is wrong, the cost almost never lands on the machine, and rarely on the company that sold it. It lands on the person with the least power to argue: a defendant who doesn't get bail, Robert Williams arrested on his lawn, the artists who weren't hired.
One last thing. Northpointe said COMPAS was "equally accurate" for Black and white defendants — and that was true. ProPublica said it was biased — also true. They were using two different definitions of "fair", and both are mathematically reasonable. You usually cannot have both at once. Remember that argument. Next week, you'll have to judge it yourself.
- 1
COMPAS only produces a number — a judge decides the sentence. So why does the reading treat the software’s errors as serious?
Reveal answer
Because judges defer to the score. When a hurried human trusts the number, the machine’s recommendation quietly becomes the real decision.
- 2
A spam filter and a courtroom risk score are wildly different. What do they share in this reading?
Reveal answer
In both, everyone assumes a human is checking the machine — but in practice nobody really is. That “nobody’s actually checking” gap is the dangerous decision boundary.
- 3
Northpointe said COMPAS was “equally accurate” for both races; ProPublica said it was biased. How can both be true?
Reveal answer
They used different definitions of “fair” — overall accuracy versus who gets wrongly flagged. Both are mathematically valid, and you can’t satisfy both at the same time.
How to read any AI system
The Case-File Protocol
You just met four systems that hurt four people. Here is the method that fits all of them — and every AI system you'll meet after today. Three questions, always in this order.
**Why the contested rows sit where they do:** - **COMPAS → Predict × Human-in-loop.** On paper a judge decides. In reality the judge defers to the score — the live debate the reading flagged. Accept "machine-decides" if the student argues the deferral well. - **Police face-match → Detect × Human-in-loop.** A detective is meant to verify the match; in the Williams case nobody did. Same deferral trap. - **TikTok "For You" → Rank × Machine-decides.** You can scroll, but you never see what it buried, and you can't overrule the ranking. - **Insurance auto-pricing / Bank fraud-freeze → Machine-decides.** The system acts first; you only appeal *after* the cost has landed. - **ChatGPT draft / Spotify Discover → Recommends.** You make the real call — you choose whether to use the text or press play. **Teaching point:** the four moves are easy and nearly everyone gets them. The row is where literacy lives — spotting where "a human reviews it" is a comforting fiction.
Design Failure Test Run Document
First evidence in your AI Literacy Portfolio — and the case you'll argue at the Tribunal.
Open a case file on a system that runs your own life.
Pick one automated system you are personally subject to — your TikTok or Instagram feed, your school's proctoring or plagiarism software, Spotify, a bank or payment app, a game's matchmaking, a hiring filter you've faced. Not one from today's lesson.
Run the protocol and write it up as a one-page case file (250–300 words):
- The system — name it, and what it does to you specifically.
- The move — Generate, Rank, Predict, or Detect? Give the evidence.
- The decision boundary — where do you stop having a say? Does a human actually check it, or does everyone just assume one does?
- The failure mode — what does it look like when this system is wrong? Be concrete.
- Who pays — when it's wrong, who carries the cost? (Hint: usually not the company.)
- Verdict — one sentence: is this system in the danger zone, and why?
You pass when your decision boundary is honest (not "a human reviews it" when nobody really does) and your cost is concrete and real — something that actually happens to a real person, not "it could theoretically be unfair." This case file is the first piece of evidence in your AI Literacy Portfolio, and the raw material you'll bring to the Tribunal.
- ✓ [
- ✓ ]