The Role of Clinical Correlation in Radiology Reports
In the field of radiology, it is important to understand the concept of “clinical correlation” in a radiology report. This refers to the correlation between the imaging findings and the patient’s clinical symptoms or medical history. It helps the radiologist interpret the images and provide a more accurate diagnosis or assessment of the patient’s condition. However, a recent study has investigated the use of AI models, such as ChatGPT, in assisting with patient communications in radiology, and the results indicate a need for human oversight.
Assessing ChatGPT for Patient Communications in Radiology
A group of radiologists at the University of Pittsburgh Medical Center conducted a study to evaluate the potential of ChatGPT in answering patient-level questions about medical imaging. They created 22 test questions covering various aspects of radiology, such as procedure safety, imaging preparation, and the meaning of terms. These questions were then fed to ChatGPT version 3.5, both with and without a prompt for an easy-to-understand response suitable for the average person.
The Accuracy and Readability of ChatGPT’s Responses
The research team, consisting of board-certified radiologists and patient advocates, assessed a total of 264 ChatGPT answers to the 22 questions. The study found that unprompted responses had an accuracy rate of 83%, which increased slightly to 87% for prompted responses. However, the consistency of the responses improved significantly from 72% to 86% when prompts were used. Most of the responses (99%) were at least partially relevant, with prompted responses being considered fully relevant 80% of the time compared to 67% for unprompted responses.
When it comes to readability, the study utilized the Flesch Kincaid Grade Level (FKGL) measures. The average FKGL for the ChatGPT responses was too high for the average reader, regardless of whether the responses were prompted (13.0) or unprompted (13.6). None of the responses reached the recommended eighth-grade readability level for patient-facing materials.
Implications and Limitations of ChatGPT in Radiology
These findings indicate that ChatGPT has potential in automating certain tasks in healthcare, such as developing patient health education materials and providing access to medical questions. However, the study emphasizes the imperfect nature of ChatGPT’s responses, with a frequency of inaccurate statements that necessitates careful human supervision or review. It is crucial to ensure that patients can easily understand the information provided by AI models like ChatGPT, as accessibility to medical knowledge is essential for improved patient access to health information.
Editor Notes: Promising Potential and the Importance of Human Oversight in AI Tools
As AI models continue to advance and find applications in healthcare, it is vital to strike a balance between automation and human oversight. The study on ChatGPT’s role in patient communications in radiology showcases its potential to streamline processes and improve patient access to health information. However, it also serves as a reminder that AI tools are not infallible and require careful monitoring and review by healthcare professionals. By combining the strengths of AI models with human expertise, we can harness their power while ensuring the accuracy and accessibility of medical information.