ChatGPT outperforming doctors on empathy, and the growing chorus over AI regulation

25 May 2023

Hot medical journal article of the week, courtesy of JAMA Internal Medicine. This article has been blowing up all over LinkedIn and Twitter.

• Here’s what the authors did: They downloaded 195 different questions that had been asked by patients on a Reddit forum, such as “I swallowed a toothpick; am I going to die now?” along with the answers that physicians had provided on the forum. The researchers then posed those same questions to ChatGPT. A panel of medical experts rated both the physician responses and the chatbot responses on both “quality” and “empathy.” And surprise, surprise, ChatGPT outperformed the physicians on both measures. By a lot.
• Here’s what most people are getting wrong about the article:
• First of all, the researchers didn’t design the study to measure accuracy of the responses, which is an important point given that we know ChatGPT and similar apps have a tendency to make things up at times.
• There’s no way to know from this article whether physician responses on Reddit are similar to physician responses to patients in other settings. Maybe doctors are more abbreviated on Reddit (noting that length of response was highly correlated with both quality and empathy in this study). Or maybe doctors who spend time on Reddit aren’t representative of other doctors. We don’t really know.
• Most of the social media posts of the article seem to be suggesting that chatbots are ready to replace doctors. But the more practical path forward, (and the one the authors propose in the article) is for chatbots to draft initial responses that doctors could then edit. A team approach would be much safer, given what we know about chatbots’ complicated relationship with truth, and would still have significant quality, empathy, and efficiency benefits over doctors drafting responses from scratch.

This week’s voices on AI regulation

Lina Kahn, the FTC chair, just penned this NYTimes piece.

• Argument #1: Twenty years ago, there was a huge missed opportunity to regulate the then-emerging business practices of Google, Facebook, and others now known as Surveillance Capitalism. The implication here, though Kahn doesn’t come out and say it directly, is that regulation needs to start before new tech-enabled business models take hold, because it’s much harder when the companies become big and powerful.
• Argument #2: The US has a number of relevant laws that could be applied to AI, including antitrust and anti-fraud. She argues that AI will create new ways for tech companies to achieve monopolies (a bit speculative, but given that she’s FTC chair, she’s supposed to be thinking about antitrust). The argument about AI-enabled fraud, including phishing, deep-fakes, etc. is more obvious (and to me, more immediately scary).
• Argument #3: Regulation won’t slow down innovation; it will simply channel it in more socially productive directions. Kahn cites the example of IBM, which pivoted its business model back in the 1960s in response to antitrust scrutiny, and helped launch the American software industry. Seems like a self-serving argument (again, this is coming from the FTC chair) but I actually like the concept.

“Godfather of AI” quitting Google to warn the world of AI danger

• First of all, some have pointed out that it’s a stretch to call Hinton “the” godfather of AI, a technology that has arguably had many godfathers and godmothers. But Hinton has an impressive academic comp sci/AI pedigree so this is probably splitting hairs.
• Hinton announced that he was leaving Google, not out of protest against the company (which he says has acted responsibly) but simply so he could speak out more openly about AI risks without having the baggage of a commercial conflict of interest
• Hinton’s concern #1: AI is a powerful tool that will be increasingly used by bad actors to do bad things. So we need to get serious about designing ways to limit their ability to do so. This sounds obvious, but technology fans (including CEOs of major tech firms) have a strong bias toward emphasizing socially positive use cases, while pooh-poohing socially detrimental ones.
• Hinton’s concern #2: AI misinformation, including deepfake videos, has potential to destroy societies trust in the internet, not to mention things like elections. Again, not a new argument, but such an existential one that I think it merits continued attention.
• Hinton’s concern #3: AI could put a lot of people out of work. This is a venerable line of social analysis (See also https://en.wikipedia.org/wiki/Luddite) that has some truth to it, but see a counterpoint lower down in this newsletter.
  

And finally, here’s yet another great NYTimes piece by a well-pedigreed AI entrepreneur and academic.

• Markus’s main concern: misinformation
• Markus’s optimism: That AI-fueled productivity will lead to more human flourishing, not less (take that, Geoffrey Hinton). Personally I think both Markus and Hinton will turn out to be right depending on which industry you look at. But that’s just a guess.
• Markus’s proposal: An international AI regulatory body to establish standards for AI safety.