Artificial Intelligence, Usability
Conditions
Brief summary
Clinical rounds in the intensive care unit (ICU) involve substantial manual documentation. Retrieving the correct protocol text and structuring notes at the bedside is time-consuming and may contribute to variation in documentation quality. Modern artificial intelligence (AI) can help structure existing information and automate protocol look-ups within a restricted, manually selected document set. The tool evaluated in this study acts as an AI-based informational assistant for clinicians. It (1) pre-populates a standardized physical-exam and daily-rounds format, (2) prepares a concise ICU course/overview using predefined formatting, and (3) retrieves relevant passages from protocols to enable rapid consistency checks by the clinician. The AI-based informational assistant does not provide treatment recommendations or patient-specific advice; all outputs require clinician verification and clinical responsibility remains with the physician.
Interventions
None listed
Sponsors
Willemijn Berkhout
Study design
Observational model
COHORT
Time perspective
PROSPECTIVE
Eligibility
Sex/Gender
ALL
Inclusion criteria
* ICU physician (nurse practicioner, resident, or staff intensivist) at the Erasmus MC. * Signed informed-consent for study participation.
Exclusion criteria
\- Physicians not expected to work on the ICU during the study period will not be approached.
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Implementation outcomes acceptability, appropriateness, and feasibility | Before integration of the AI-based informational assistant and 4-, 8-, and 12-weeks after integration. | The mean scores and standard deviations of the 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree) closed-ended questions of the survey on the physicians' perspectives will be calculated. Standardised questionnaires AIM, IAM and FIM are used. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Perceived time saved when using the AI-based informational assistant during ICU rounds | 12-weeks after integration of the AI-based informational assistant . | The mean scores and standard deviations of the 5-point Likert scale closed-ended questions of the survey on the physicians' perceptions on retrieval speed. |
| Task-based efficiency, including time to (i) produce a structured rounds note and (ii) retrieve relevant protocol text | Before integration of the AI-based informational assistant and 12-weeks after integration | Timed predefined ICU round documentation tasks with and without AI-based informational assistant. Time difference will be calculated. |
| Perceived usefulness, clarity, and trustworthiness | Before integration of the AI-based informational assistant and during the 12-weeks utilization. | The mean scores and standard deviations of the 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree) closed-ended questions of the survey on the physicians' perspectives will be calculated. |
| Adoption and use, including frequency of use, retention over time, and interaction patterns (e.g., number/type of edits, use cases, feature use) | During the 12-weeks utilization of the AI-based informational assistant. | Adoption will be determined by frequency of use (interactions per participant per week) and retention (continued use over time), expressed as counts and proportions. Fidelity will be determined by the misusage per participant, reported as counts and proportions. Adoption and fidelity will be aggregated at both participant and cohort level. Interaction logs will be used to characterize use patterns, including number and type of edits, use cases and feature usage. |
| Technical output quality | Before integration of the AI-based informational assistant and during the 12-weeks utilization. | Outputs is reviewed on accuracy, recall, precision, groundedness, contextual usefulness, and hallucination presence. Reported as counts and proportions. |
| Trust in the system, perceived workload, and task satisfaction | 12-weeks after integration of the AI-based informational assistant. | The mean scores and standard deviations of the 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree) closed-ended questions of the survey on the physicians' perspectives will be calculated. Standardised questionnaires S-TIAS and NASA-TLX are used. |
Outcome results
None listed