None listed
Conditions
Brief summary
The public is turning to artificial intelligence (AI) powered websites and chatbots for seeking medical advice and information. Some doctors already use AI to take medical notes and later edit the responses. Hence, the public is likely to encounter human+AI- generated advice in their daily lives. Best practice is to be transparent that AI models have been used, but little is known about how phrasing this information can affect user trust. Previous research has shown that people tend to perceive online medical advice as more empathetic and are more satisfied with it if they think it was written by a human rather than by an AI model, even when the advice is identical (Reis et al., 2024), but trust has not yet been investigated. It’s important to investigate factors that affect trust in AI-generated medical advice. Two factors that are important for trust in other forms of technology are structural assurance (about laws and privacy) and competence of the technology. These factors have not yet been explored with regard to trust in online medical advice scenarios. This study aims to investigate how perceptions of the source of medical advice affect patient trust. The advice will be the same in all conditions, but the descriptions of how the advice is generated will differ by condition. We hypothesise that perceived trust will differ significantly across conditions, with higher trust for the human-only condition than the three AI conditions. Trust will be significantly higher for the AI models labelled as having competence or structural assurance than the simple AI model. Perceived empathy, reliability, and individual willingness to follow the advice will differ significantly across conditions. The human condition will have higher ratings, whereas the simple AI model will have the lowest ratings, with moderate ratings for the two AI models labelled with competence and structural assurance. Perceived comprehensibility will not differ significantly between conditions. The exploratory hypothesis is the condition labelled as AI model with competence will have higher ratings for all outcomes than AI model with structural assurance.
Interventions
Participants will read four patient vignettes, in which a person seeks medical advice online. The advice will be the same in all conditions, but the descriptions of how the advice is generated will differ by condition. Participants will be randomised to one of the four conditions, where they will be told that the advice is written by either (1) a human; (2) a simple artificial intelligence (AI) model and edited by a human; (3) an AI model with competence and edited by a human; or (4) an AI model
Participants will read four patient vignettes, in which a person seeks medical advice online. The advice will be the same in all conditions, but the descriptions of how the advice is generated will differ by condition. Participants will be randomised to one of the four conditions, where they will be told that the advice is written by either (1) a human; (2) a simple artificial intelligence (AI) model and edited by a human; (3) an AI model with competence and edited by a human; or (4) an AI model with structural assurance and edited by a human. Information for non-drug trials: Material used: Participants will access the Participant Information Sheet (PIS), the consent form, patient vignettes, questionnaires, and the debrief sheet through the Qualtrics platform. 4 patient vignettes will be used in this study. All participants will receive all 4 vignettes. The first one will be about acid reflux. The second one will be about colonoscopy. These two vignettes were adopted versions of those used in the Reis et al. (2024) study. The adaptation was made by a human doctor (Dr Anna Perea). The third one will be about lung cancer. The fourth one will be about urinary tract infection. These two vignettes were generated by ChatGPT and refined by a human doctor (Dr Anna Perera). Procedures: Firstly, participants will read the PIS and tick a check box on the consent form to indicate that they give informed consent to participate. Next, participants will read four patient vignettes. The order of the patient vignettes will be randomly presented. The vignette will be identical but with different labels. Participants will be randomly allocated to see one of four labels with associated descriptions (conditions): “Human-only”, “Human + AI", “Human + competent AI model”, and “Human + structurally assured AI”. After reading each vignette, participants will fill in the Short Trust in Automation Scale (McGrath et al., 2025), and the empathy, reliability, comprehensibility, and individual willingness to follow advice measure (Reis et al., 2024). Lastly, participants will fill in a demographic and attention-check questionnaire. There will be an attention-check question asking about which label they saw in the vignettes. Who will deliver the intervention and their expertise: This is an online study, and no researchers will deliver the intervention. After being allocated to one of the four conditions, participants will read the pre-written medical advice and answer questionnaires through the Qualtrics platform. Mode of delivery (individual/ group): Individual. Participants will complete the study individually by clicking on the Qualtrics link. Number of times the intervention will be delivered: The 4 vignettes and the questionnaires will be presented to each participant once. The estimated study completion time is 10 - 15 minutes. The location where the intervention occurs: Online. Participants will be recruited through the Prolific platform. The study will take place on Qualtrics. Participants will click on a Qualtrics link, which gives them access to the vignettes and questionnaires presented on the University of Auckland Qualtrics platform.
Sponsors
The University of Auckland
Study design
Allocation
Randomised controlled trial
Primary purpose
Educational / counselling / training
Eligibility
Sex/Gender
All
Age
18 Years to No maximum
Healthy volunteers
Yes
Inclusion criteria
Participants need to be at least 18 years old. They also need to be able to read and understand English, be on the Prolific platform, and reside in the UK.
Exclusion criteria
Participants who do not have a stable internet connection or a digital device (e.g., a laptop or phone) will be excluded from this online study.
Outcome results
None listed