This study examines the ability of large language models to provide evidence-based information on breast and prostate cancer screening. The focus is on whether these models can deliver accurate and comprehensible health information regarding the risks, benefits, and outcomes of screening programs for these two types of cancer.
Conditions
Interventions
Group 1: Participants will receive a minimal intervention (boosting): “Please consider the OARS rule: You need to know your options, the advantages and risks of each, and how steady they are to happen
Sponsors
Harding-Zentrum für Risikokompetenz, Universität Potsdam
Eligibility
Sex/Gender
All
Age
18 Years to No maximum
Inclusion criteria
Inclusion criteria: The inclusion criteria for selecting the LLMs in the study will be the following: - State-of-the-Art Performance: Only LLMs that represent current, state-of-the-art models in natural language processing, such as OpenAI’s ChatGPT, Google Gemini, and Mistral AI, will be considered. - Accessibility for Public Use: The LLMs must be accessible to the general public, ensuring that their capabilities reflect real-world use cases and that the findings can be generalized to typical interactions by laypeople. - Multidomain Knowledge: The LLMs must demonstrate the ability to handle a wide range of topics, including healthcare and cancer-related information, ensuring their relevance for answering complex, domain-specific queries. Phase 2, Participants: - Language Proficiency: Participants must have proficiency in English, as the study involves interacting with LLMs in English and understanding health-related information presented in this language. - Access to Digital Devices: Participants must have access to and be able to use digital devices (e.g., computers, tablets) with internet access, as the study involves generating and submitting prompts to LLMs online. - Geographic Location: Participants should reside in regions where access to healthcare is comparable to international standards, such as the U.K., to ensure that the health information provided by LLMs is relevant and applicable to their context.
Exclusion criteria
Exclusion criteria: Phase 1, LLMs: The exclusion criteria for selecting the LLMs in the study will be the following: - Limited Access or Restricted Use: LLMs that are not publicly accessible or require proprietary access for specialized use will be excluded, as they do not represent general-use cases for laypeople. - Domain-Specific Models: LLMs that are specifically trained or fine-tuned for niche domains (e.g., exclusively healthcare-specific models) will be excluded, as they do not reflect the broader, general-purpose models used by the public. - Non-English Language Proficiency: LLMs that primarily operate in languages other than English or demonstrate limited proficiency in understanding and generating English-language responses will be excluded. Phase 2, Participants: The exclusion criteria for selecting the Participants in the study will be the following: - Health Professionals: Individuals with professional expertise in healthcare, particularly in cancer screening or health communication, will be excluded to avoid bias and ensure that participants reflect the general lay population. - Previous Experience with LLM Studies: Participants who have taken part in similar studies involving LLMs will be excluded to prevent familiarity with the technology from influencing the results.
Design outcomes
Primary
| Measure | Time frame |
|---|---|
| Expert scoring of how evidence-based LLM communication is (mappInfo tool in addition to a novel score based on the guideline EB health information) | — |
Secondary
| Measure | Time frame |
|---|---|
| Self-reported use and experience with LLM; preference in shared decision-making | — |
Countries
United Kingdom
Contacts
Public ContactFelix Rebitschek
Harding-Zentrum für Risikokompetenz, Universität Potsdam
Outcome results
None listed