Intended for healthcare professionals

Rapid response to:


Using artificial intelligence to assess clinicians’ communication skills

BMJ 2019; 364 doi: (Published 18 January 2019) Cite this as: BMJ 2019;364:l161

Rapid Response:

Re: Using artificial intelligence to assess clinicians’ communication skills

Ryan et al. [1] highlight many areas in which AI could be used to improve the way we improve doctors’ communication skills.

Much of this work has already been done by our group, using the features they point towards and a few more. By managing the process of simulated interviews, and recording them, our system can analyse not only voice features as the authors suggest, but also facial expressions [2,3,4]. The eqclinic system manages the appointments between simulated patients and students, then provides an easy to use web interface where they have the teleconference. The video interview is recorded and finally the system merges feedback from tutors and simulated patients with that generated automatically. The latter includes measures of turn-taking, pitch, volume, shaking and nodding. Feedback from the computer is contextualised with feedback from human experts.

Our system has been evaluated in a 2-group randomised crossover trial with 268 medical students [3]. The study showed that SOCA communication skills measures (score range 4–16) from the first face-to-face consultation were significantly higher for students in group A who had completed EQClinic training and reviewed the nonverbal behaviour feedback, compared with group B, who had completed only the course curriculum components (P=.04).

Rather than assessing doctors (or students) using AI tools we believe it is more effective to use them as a form of feedback. Rather than the AI providing a model of how the doctor should communicate, a more humanising approach is to provide information that helps them notice, and value communication skills.

You can view a demo of the system at

1. Ryan P, Luz S, Albert P, Vogel C, Normand C, Elwyn G. Using artificial intelligence to assess clinicians' communication skills. BMJ. 2019 Jan 18;364:l161
2. Liu C, Calvo RA and Lim R (2016). Improving medical students’ awareness of their nonverbal communication through automated nonverbal behavior feedback. Front. ICT 3:11. doi: 10.3389/fict.2016.00011
3. Liu, C., Lim, R. L., McCabe, K. L., Taylor, S., & Calvo, R. A. (2016). A web-based telehealth training platform incorporating automated nonverbal behavior feedback for teaching communication skills to medical students: a randomized crossover study. Journal of medical Internet research, 18(9).
4. Liu C, Scott KM, Lim R, Taylor SC and Calvo RA (2016). EQClinic: a platform for learning communication skills in clinical consultations. Medical Education Online, [S.l.], v. 21, jul. 2016.

Competing interests: No competing interests

06 February 2019
Rafael A Calvo
University of Sydney and Imperial College London
School of Electrical and Information Engineering, University of Sydney, NSW 2006, Australia