None
Conditions
Interventions
Case Group:None
OSCE group:None
Sponsors
The University of Hongkong - Shenzhen Hospital
Eligibility
Sex/Gender
All
Inclusion criteria
Inclusion criteria: 1. OSCE Question Selection Criteria (1) Relevance of Questions: Questions should be closely related to medical clinical practice and can effectively assess the medical knowledge and clinical skills of LLMs. (2) Diversity of Questions: Questions should cover multiple medical fields and clinical scenarios, including but not limited to internal medicine, surgery, obstetrics and gynecology, pediatrics, emergency medicine, etc. (3) Difficulty of Questions: The difficulty of the questions should be moderate, capable of distinguishing different levels of medical knowledge and clinical skills. (4) Quality of Questions: Questions should be reviewed by experts to ensure their scientific validity and rationality. 2. Real Case Selection Criteria (1) Authenticity of Cases: Cases should be real cases from our hospital, with complete medical records and clinical data. (2) Diversity of Cases: Cases should cover various types of diseases and clinical scenarios, including common diseases, frequently occurring diseases, and rare or difficult conditions. (3) Completeness of Cases: Cases should include detailed descriptions of the condition, diagnostic results, treatment plans, and follow-up records. (4) Representativeness of Cases: Cases should be representative, reflecting common issues and challenges encountered in clinical practice. (5) Time Range of Cases: Cases should cover data from the past 5 years to ensure their timeliness and relevance. Specific Description OSCE Questions: 600 questions will be selected from the existing OSCE question bank to ensure coverage of a wide range of medical fields and clinical scenarios. The questions will be reviewed by experts to ensure their scientific validity and rationality. Real Cases: 600 real cases will be selected from the hospital's case database, covering various types of diseases and clinical scenarios. The cases will undergo strict screening to ensure their completeness and representativeness. With the above sample size and selection criteria, this study will ensure the representativeness and diversity of the samples, providing a scientific basis for comprehensively evaluating the true medical capabilities of LLMs such as DeepSeek and their value in clinical services.
Exclusion criteria
Exclusion criteria: None
Design outcomes
Primary
| Measure | Time frame |
|---|---|
| Medical strength of specific large language models;OSCE Question Bank Evaluation Accuracy; | — |
Countries
China
Contacts
Public ContactZhang Wenzhi
The University of Hongkong - Shenzhen Hospital
Outcome results
None listed