Diagnosis
Conditions
Keywords
Clinical reasoning, Large language models, Computer-assissted diagnosis, Rheumatology
Brief summary
This trial evaluates whether providing physicians with access to Prof. Valmed, a clinical decision support medical product, improves identification of rheumatic diseases and formulation of differential diagnoses compared with conventional decision support.
Detailed description
Advanced AI, particularly large language models, shows promise for enhancing clinical reasoning, yet most systems such as ChatGPT are not certified as medical products. Prof. Valmed is a clinical decision support medical product designed to assist physicians in diagnostic decision making. Given frequent referral problems and diagnostic delays in rheumatology, evaluating such support is highly relevant for clinical workflows. This randomized controlled trial will test whether access to Prof. Valmed improves physicians' diagnostic performance in cases of suspected rheumatic disease compared with conventional decision support. Participants will be randomized to either use Prof. Valmed or rely on conventional tools while working through standardized clinical cases. For each case, participants will submit up to three differential diagnoses and a confidence rating. Independent reviewers, blinded to group allocation, will adjudicate accuracy. Findings will clarify the benefits and limitations of integrating Prof. Valmed into routine practice.
Interventions
OTHERProf. Valmed
Prof Valmed. decision support system.
Sponsors
Philipps University Marburg
University Medical Center Hamburg-Eppendorf (UKE)
Oslo University Hospital
Diakonhjemmet Hospital
University Hospital Erlangen
Charite University, Berlin, Germany
Rheumazentrum Ruhrgebiet
University of Lausanne Hospitals
Klinikum Fulda
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
DIAGNOSTIC
Masking
SINGLE (Outcomes Assessor)
Masking description
The evaluation of responses will be performed by assessors blinded to participant identity and treatment assignment.
Eligibility
Sex/Gender
ALL
Healthy volunteers
Yes
Inclusion criteria
* Participants must be licensed physicians. * Training in rheumatology, internal medicine, emergency medicine, family medicine, dermatology or orthopedics.
Exclusion criteria
* Not currently practicing clinically.
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Diagnostic accuracy of top diagnosis | directly (within 10 minutes) after Intervention | Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 3 suggestions. Percentage of exact matches of the top suggestion with the actual diagnosis will be analyzed |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Diagnostic accuracy of top 3 suggestions | directly (within 10 minutes) after Intervention | Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 3 suggestions. Percentage of exact matches with the actual diagnosis included in the top 3 suggestions will be analyzed |
| Diagnostic confidence | directly (within 10 minutes) after Intervention | For each case participants will be asked for their diagnostic confidence (VAS 0-10). The mean score will be compared between groups. |
| Time spent for diagnosis | directly (within 10 minutes) after Intervention | We will compare how much time (in seconds) participants spend per case between the two study arms. |
| Perceived Information Timeliness | directly (within 10 minutes) after Intervention | Perceived ability to receive the information needed without delay (Likert scale from 1 to 5) |
| Perceived diagnostic support quality | directly (within 10 minutes) after Intervention | Perceived quality of the diagnostic support (Likert scale from 1 to 5) |
| Diagnostic reasoning | during evaluation | For each case, participants will receive 1 point for each plausible diagnosis and 2 points for a completely correct response. The total scores will be compared between the randomized groups. |
Countries
Germany
Contacts
PRINCIPAL_INVESTIGATORJohannes Knitza, MD PhD
University Marburg
Outcome results
None listed