Recognizing What AI Gets Wrong

AI is powerful but limited. Knowing these flaws is essential for responsible use. This lesson covers AI’s lack of understanding, hallucinations, and bias problems.

Fundamental Limitations

AI Doesn’t Understand—It Pattern Matches: AI lacks real knowledge of the world, causality, and common sense. It only predicts text patterns.

Example: 'The trophy doesn't fit in the suitcase because it's too big' → Humans know 'it' = trophy. AI guesses based on statistical patterns, not spatial reasoning.

The Hallucination Problem

What Are Hallucinations? AI invents facts, citations, or numbers with confidence.

• Fake studies, false legal citations, or non-existent products

Why It Happens: Models optimize for plausible text, not truth, and can’t verify facts.

Real Cases: A lawyer sanctioned for AI-invented legal citations (2023). News outlets retracting AI-written articles with fake studies.

Mitigation: Always verify claims, request and check sources, and use retrieval tools that ground outputs in real documents.

Bias in AI Systems

Where Bias Comes From:

• Training data bias: Reflects past discrimination (e.g., facial recognition struggles on darker skin).
• Selection bias: Internet-heavy training favors Western/English data.
• Association bias: Stereotypes (doctor=male, nurse=female).
• Algorithmic bias: Optimizing for majority harms minorities.

Cases: Amazon’s hiring AI penalized women’s resumes (2018). COMPAS tool labeled Black defendants higher risk (2016). Image generators reinforce stereotypes (CEO=white male, nurse=female).

Language & Representation Bias

• English dominance sidelines other cultures and idioms
• Western norms overrepresented
• Limited representation of LGBTQ+, body diversity, and age

Recognizing & Mitigating Bias

Warning signs: Stereotypes, lack of diversity, single perspective.

Testing: Compare prompts with swapped demographics, ask for multiple perspectives.

Mitigation: Explicitly request diversity, use gender-neutral language, and manually review for fairness.

When AI Limitations Are Dangerous

High-Stakes Uses to Avoid Without Oversight: Medicine, law, finance, safety systems, hiring, and criminal justice. Errors here cause serious harm.

Other Limitations

• Bad at advanced math, logic puzzles, and counting
• Knowledge is outdated after training cutoff
• Context window limited; may contradict itself

Responsibility Framework

Users must: Fact-check outputs, detect bias, disclose AI use, and apply human judgment.

Organizations should: Create policies, train staff, review AI outputs, audit bias, and ensure accountability.

Quick Checklist

• ☑ Facts verified and sources real
• ☑ Bias and stereotypes checked
• ☑ Representation diverse
• ☑ Language inclusive
• ☑ Expert review for high-stakes topics

Bottom line: AI is a tool, not a truth source. The best users know where to rely on it—and where human judgment is essential.