Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination

Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination: An Empirical Analysis Based on the "Answering-Generating" Closed Loop and Transformation of Competency Assessment

Status

Recruiting

Phases

Unknown

Study type

Observational

Source

ClinicalTrials.gov

Registry ID

NCT07505862

Enrollment

Registered

2026-04-01

Start date

2025-10-01

Completion date

2026-03-30

Last updated

2026-04-01

For informational purposes only — not medical advice. Sourced from public registries and may not reflect the latest updates. Terms

Conditions

AI (Artificial Intelligence)

Brief summary

This is a cross-sectional study that primarily employs quantitative analysis, supplemented by qualitative assessment. The research is conducted in two stages: Phase I consists of a model performance comparison experiment, and Phase II involves an item quality evaluation experiment. The entire study adheres to the principles of single-blinding, randomization, and standardization to ensure scientific rigor and reproducibility. The single-blind design is implemented during the "standardized testing" phase, where the system intersperses AI-generated items with those authored by human experts. Participants remain blinded to the source of each item (AI-generated vs. human-authored) throughout the testing and scoring processes, thereby ensuring the objectivity of the evaluation results.

Interventions

None listed

Study design

Observational model

OTHER

Time perspective

CROSS_SECTIONAL

Eligibility

Sex/Gender

ALL

Age

18 Years to 60 Years

Healthy volunteers

Yes

Inclusion criteria

* 1.Professional Status: Medical students currently enrolled in a Standardized Residency Training (SRT) program. 2.Educational Background: Holders of a Bachelor of Medicine degree or higher, with foundational clinical knowledge. 3.Informed Consent: Voluntarily participate in the study and provide written informed consent. 4.Technical Competency: Proficient in using digital platforms to complete assessments and scoring.

Exclusion criteria

* 1.Conflict of Interest: Individuals involved in the AI model training, prompt engineering, or the creation of the human-authored question bank for this study. 2.Inability to Complete: Presence of visual/auditory impairments or severe illness that precludes completion of the assessment within the specified time. 3.Investigator's Discretion: Any other condition that, in the opinion of the investigator, renders the participant unsuitable for the study.

Design outcomes

Primary

Measure	Time frame	Description
Comparative Analysis of Item Accuracy	Baseline	The investigators will evaluate the average accuracy rates coefficients of AI-generated items compared to human-generated items

Countries

China

Contacts

CONTACTZhuoyi Chen MD

18737552662@163.com+86 18737552662

Outcome results