"AI has the potential to re-ontologize healthcare—to completely redesign what we consider to be a disease, what we consider to be a disability, and how we organise care. But we need to decide what good healthcare actually means before we AI-ify everything."
— Dr Jessica Morley
Listen now on Apple, Spotify, YouTube or wherever you get your podcasts.
In this episode of Clinical Changemakers, Dr Jessica Morley, an AI ethics expert and researcher from Yale Digital Ethics Centre, challenges our understanding of what large language models actually are and what they mean for healthcare. Drawing from her work on data ethics and her involvement with OpenSafely during the pandemic, Dr Morley explains why viewing AI as "stochastic parrots" is crucial for healthcare implementation, explores the concept of re-ontologizing medicine, and argues why we need ethical frameworks before technological deployment rather than after.
Key Takeaways
Stochastic Parrots in Medicine: Large language models don't understand medical concepts—they predict the most likely next word based on probability from their training data. This means they can give you different answers to the same medical question and lack the contextual understanding crucial for patient care. Understanding this fundamental limitation is essential for safe healthcare implementation.
The Re-ontologizing Power of AI: AI doesn't just replace existing tools like upgrading from a physical to digital stethoscope. It has the power to completely redesign healthcare by redefining what constitutes disease, changing how we organize care, and separating diagnosis from physical patient interaction. This transformation can be powerful and positive, but only if we're intentional about our goals.
The Inverse Data Quality Law: Just as the inverse care law states that those who need healthcare most get it least, the inverse data quality law means those who need AI healthcare tools most will have the poorest quality data about themselves. This creates a two-tier system where marginalized populations get inferior AI-driven care.
Social License Trumps Legal Permission: Technical feasibility and legal compliance aren't enough for successful AI implementation in healthcare. Projects like the UK's Care.data failed despite being legal because they lacked social acceptance. OpenSafely succeeded by building in privacy protections, transparency, and meaningful public engagement from the start.
Where to Find Our Guest
Dr Jessica Morley (LinkedIn, Google Scholar)
In This Episode
01:27 - Stochastic parrots: Understanding what LLMs actually do
04:27 - Emergent properties: Why LLMs remain sophisticated probability machines
11:57 - Moral worth and parasocial relationships: When humans attach meaning to AI
14:33 - Re-ontologizing healthcare: How AI redesigns medicine itself
19:28 - The ethical traps: What happens when we AI-ify without thinking about outcomes
25:33 - Two-tier systems: How AI could worsen healthcare inequalities
28:44 - Ethics vs. law: Why we need both rules and values
32:31 - Learning from NHS failures: The importance of not making assumptions
47:43 - Global policy tensions: EU regulation vs. US "let it rip" approaches
54:33 - False dichotomies: Moving beyond "some care vs. no care" thinking
59:02 - Current global sentiment: From tech optimism to healthcare caution
Referenced
Dr Morley’s Paper - The ethics of AI in health care: A mapping review (link)
Defining a Stochastic Parrot (link)
OpenSafely platform and approach to health data research (link)
Lessons from Care.data project and its failure in the UK (link)
Contact
If you have any feedback, questions or if you'd like to get in touch, reach out at jono@clinicalchangemakers.com
Hey there 🙌
As a small independent podcast, every rating and share makes a real difference in helping us reach more healthcare leaders. If you found value here, please rate us and pass this along to a colleague who needs to hear it.
Clinical Changemakers is a podcast that explores inspiring stories of leadership and innovation in healthcare. To learn more and join the conversation, visit: www.clinicalchangemakers.com
