How well can digital assistants answer questions on sex?
Google laptop searches seem to be better at finding quality online sexual health advice than digital assistants on smartphones, find experts in the Christmas issue of The BMJ.
They say their findings “show the importance of improving digital literacy in the general population” and call for more to be done “to encourage internet users to treat information in online lifestyle magazines with caution.”
A recent UK survey found that 41% of internet users go online for health related questions. But do smartphones and their digital assistants offer quality sexual health advice?
To find out, a team of academics based in New Zealand asked Siri and Google Assistant – digital assistants that use voice activated software to answer questions and perform tasks.
They selected 50 questions to test the software and then compared their answers with a laptop based Google search.
Questions were based on information from the UK NHS site Healthy Choices and recent sex related news – or were designed to test functionality, for example, locating services or finding images and videos on how to have sex.
Each author made a maximum of three attempts per question when speaking into the smartphones.
They found that a laptop based Google search performed much better than the two digital assistants, providing 72% (36/50) of the best (or equal best) responses for the sexual health questions.
Google Assistant performed better than Siri with 50% of best (or equal best responses) versus 32%. Google searches also had the lowest outright failure rate, providing no useful response for 8% (4/50) of the questions compared with 12% (6) for Google Assistant and 36% (18) for Siri.
When they excluded some of the functionality test questions, 48% (20/42) of the search questions were answered with what they determined were expert sources, such as the NHS, Family Planning (New Zealand) and the US Centers for Disease Prevention and Control (CDC).
Six (14%) searches provided websites with “some expertise” such as Wikipedia articles and commercially oriented sites (eg, condom manufacturers), and six identified online magazine articles.
Google and Google Assistant seemed better than Siri at finding pictures of how to have sex, say the authors. Somewhat surprisingly, Siri failed to find any videos of people having sex on the internet. Siri was also more likely to be diffident, with responses to some questions about sex being: “I don’t have an opinion on that” and also had trouble with New Zealand accents at times, repeatedly confusing “sex” with “six.”
Siri’s response to, “Tell me about menopause” was to suggest the show Menopause the Musical in Wikipedia (this show is apparently running in Las Vegas) and interpreted “STI” (sexually transmitted infections) as a stock market code.
Google Assistant had fewer such problems but responded to a question on STIs by providing a website link to the popular seaside resort of “St Ives” in Cornwall.
Siri was best at locating some nearby services, such as the nearest place to buy condoms or obtain emergency contraception, but less ideally suggested a local acupuncture clinic when asked for the nearest “sexual health clinic”.
Finally, questions around magazine and newspaper articles provided answers of variable quality.
“Our experiences suggest that people can find quality sexual health advice when searching online, but this is less likely if they use a digital assistant, especially Siri, instead of Google laptop searches,” say the authors.
“Parents too embarrassed to respond to their children’s questions about sex, can reasonably say “just Google it,” but we would not suggest asking Siri until it becomes more comfortable with talking about sex (or at least has an opinion),” they add.
“Clearly, the ideal is to ensure that all sexual health advice searches, including those using slang, colloquialisms, or New Zealand accents, are always directed to high quality sites with up-to-date, evidence based recommendations,” they conclude.
Journal: The BMJ