External Validation of AI-Aided Weaning Software Using Multicenter Retrospective Data

Using Multicenter Retrospective Data to Validate the Performance of AI-Aided Weaning Software

Status

Completed

Phases

Unknown

Study type

Observational

Source

ClinicalTrials.gov

Registry ID

NCT07658131

Enrollment

1500

Registered

2026-06-18

Start date

2020-01-01

Completion date

2024-12-31

Last updated

2026-06-22

For informational purposes only — not medical advice. Sourced from public registries and may not reflect the latest updates. Terms

Conditions

Critical Illness, Mechanical Ventilation, Respiratory Failure

Keywords

Artificial Intelligence, Machine Learning, Mechanical Ventilation, Extubation, Intensive Care Unit, Weaning, Critical Care, External Validation

Brief summary

This multicenter retrospective study aims to externally validate an artificial intelligence-aided weaning software developed using intensive care unit data from Taichung Veterans General Hospital between 2015 and 2019. The model predicts the optimal timing for extubation using routinely collected clinical variables including ventilator parameters, physiologic measurements, and fluid and nutrition information. De-identified data from four hospitals collected between 2020 and 2024 will be used to evaluate model performance. Performance metrics include sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC), and F1 score.

Detailed description

Critical care generates a large amount of digitized clinical data that may benefit from artificial intelligence-assisted decision support. The AI-Aided Weaning Software was previously developed using ICU data from Taichung Veterans General Hospital collected between 2015 and 2019. This retrospective multicenter validation study will evaluate the external performance of the established model using independent datasets from four hospitals in Taiwan, including Taichung Veterans General Hospital, Mackay Memorial Hospital, Kaohsiung Medical University Chung-Ho Memorial Hospital, and Tungs' Taichung MetroHarbor Hospital. The study population includes adult ICU patients with respiratory failure who received mechanical ventilation for at least 72 hours between January 2020 and December 2024. De-identified routine clinical records will be collected according to a predefined case report form and analyzed centrally. The primary objective is to assess the external validity of the AI-Aided Weaning Software across different hospitals. Model performance will be evaluated using sensitivity, specificity, accuracy, AUROC, and F1 score.

Interventions

None listed

Study design

Observational model

COHORT

Time perspective

RETROSPECTIVE

Eligibility

Sex/Gender

ALL

Age

20 Years to No maximum

Healthy volunteers

Inclusion criteria

* Adult patients aged 20 years or older. * Admitted to the intensive care unit (ICU) at one of the participating hospitals between January 1, 2020 and December 31, 2024. * Received invasive mechanical ventilation for at least 72 hours. * Availability of de-identified clinical data required for model validation.

Exclusion criteria

* Patients who did not receive invasive mechanical ventilation. * Duration of mechanical ventilation less than 72 hours. * Missing key clinical variables required for model validation.

Design outcomes

Primary

Measure	Time frame	Description
Model Performance (AUROC)	Using data collected during ICU admission	Area under the receiver operating characteristic curve (AUROC) for predicting successful extubation. AUROC ranges from 0.5 to 1.0, with higher values indicating better discriminative performance of the prediction model.

Secondary

Measure	Time frame	Description
Sensitivity	ICU admission	SensitivitySensitivity of the prediction model for successful extubation. Sensitivity ranges from 0 to 1 (or 0% to 100%), with higher values indicating better identification of patients who achieve successful extubation.
Specificity	ICU admission	Specificity of the prediction model for successful extubation. Specificity ranges from 0 to 1 (or 0% to 100%), with higher values indicating better identification of patients who do not achieve successful extubation.
Accuracy	ICU admission	Accuracy of the prediction model for successful extubation. Accuracy ranges from 0 to 1 (or 0% to 100%), with higher values indicating better overall prediction performance.
F1 Score	ICU admission	F1 score of the prediction model for successful extubation. F1 score ranges from 0 to 1, with higher values indicating better balance between precision and recall.

Countries

Taiwan

Outcome results