Large Language Models, Thyroid Cancer
Conditions
Keywords
Thyroid Cancer, large language models, LLMs, natural language processing, NLP
Brief summary
This study aims to evaluate the clinical feasibility of adopting artificial intelligence (AI)-based models to improve clinical management of thyroid cancer.
Detailed description
With recent advancements in technology, AI has become widely applicable to visual text recognition in clinical settings. AI-powered text recognition is emerging as a highly efficient, sustainable, and cost-effective tool for decision making and personalised medicine. Numerous studies have employed natural language processing (NLP) algorithms, particularly large language models (LLMs), to convert unstructured free-text from clinical consultation notes within electronic health records (EHR) into structured data, thus enriching individual clinical profiles in the EHR databases. Over time, these AI models have continuously improved their predictive accuracy and performance through self-learning (or unsupervised learning). While AI models had made a significant impact in oncology practices overseas, their utility for text recognition in oncology remains limited in Hong Kong. This proposed study aims to evaluate the clinical feasibility of adopting AI-based models to improve time efficiency, accuracy, and end-users' confidence in diagnostic assessment and risk prediction, compared against traditional workflows without AI assistant for thyroid cancer management.
Interventions
OTHERAI-enabled clinical assistant
Participants will provide the caner staging and risk category of each thyroid cancer patient as well as the participants' confidence for the above diagnostic assessments with AI-enabled clinical assistant as the intervention. The AI assistant is powered by LLMs and comprises a clinical dashboard. The clinical dashboard displays the original clinical notes and summarizes cancer staging and risk category of each thyroid cancer patient generated from the backend processing of the clinical assistant. Supporting evidence from original clinical notes is also highlighted for participants' verification.
Sponsors
The University of Hong Kong
Innovation and Technology Commission, Hong Kong
Study design
Allocation
RANDOMIZED
Intervention model
CROSSOVER
Primary purpose
HEALTH_SERVICES_RESEARCH
Masking
SINGLE (Outcomes Assessor)
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
Yes
Inclusion criteria
* Consenting medical students * Consenting clinicians who are directly involved in the care of thyroid cancer patients, including endocrine surgeons, endocrinologists, oncologists, and pathologists.
Exclusion criteria
• Medical students and clinicians who had reviewed the clinical notes or were involved in the processing of the clinical notes prior to the commencement of trial
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Efficiency | Between intervention group and non-intervention group. Cross-over in 4-26 weeks | The time required to complete reviewing one set of clinical notes is compared between intervention and non-intervention groups |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Accuracy of Cancer Staging and Risk Stratification by Participants Compared with Ground Truth across Intervention and Non-intervention Groups | Between intervention group and non-intervention group. Cross-over in 4-26 weeks | The study will compare the accuracy of cancer staging and risk category assessed by the participants across the intervention group with AI assitance and non-intervention group without AI asssitance. The participants will review the clinical notes and assess the cancer staging and risk category for each thyroid cancer patient with or without the AI assistant. Participant provided assessments will be compared against the ground truth established by the clinical investigators of the study to guage the accuracy which is quantified as the percentage of correctly graded cancer staging and risk stratification. The accuracy will be compared between the intervention group and non-intervention groups using t-tests to evaluate the clinical impact of the AI assistant. |
| Participants' Confidence in Cancer Staging and Risk Stratification as Assessed by a 0-10 Scale Questionnaire | Between intervention group and non-intervention group. Cross-over in 4-26 weeks | The study will compare the participants' confidence in grading cancer staging and risk category between the intervention group with AI assistance and non-intervention group without AI-assistance. After evaluating each thyroid cancer case for providing cancer staging and risk category, participants will complete a short questionnaire rating their confidence in providing their assessments on a scale from 0 (lowest) to 10 (hightest). Meanw confidence score will be compared between the intervention group and non-intervention group to evaluate the clinical impact of the AI assitant. |
Countries
Hong Kong
Contacts
PRINCIPAL_INVESTIGATORKing Ho Carlos Wong
School of Public Health The University of Hong Kong
PRINCIPAL_INVESTIGATORMan Him Matrix Fung
Department of Surgery, School of Clinical Medicine, The University of Hong Kong
Outcome results
None listed