Consistency of Assessor Scoring on the Direct Observation of Procedural Skills Form

None listed

Commencing in 2013, the Australian and New Zealand College of Anaesthetists (ANZCA) implemented the Direct Observation of Procedural Skills (DOPS) form as part of the new curriculum that uses workplace based assessments (WBA) as an integral part of trainee assessment. DOPS is a generic tool using a 10-item, 9-point Likert scale scoring system for all procedural skills performed in anaesthesia. A vital aspect of confidence in the results generated by any assessment tool is the concept of reliability, which refers to the reproducibility of a test. This implies that a broad range of assessors, when viewing the same trainee performing a regional anaesthesia block, comes to a similar result. Components which improve reliability of an assessment tool include objective criteria for assessment, training of the assessors, direct observation at the moment of performance (rather than retrospective), and ability of the tool to discriminate between different levels of performance. As there is no published literature on the ANZCA DOPS reliability, we are performing this study to collect the consistency of scoring. The results will generate baseline data to inform future studies, such as whether training can improve reliability. This research project will recruit ANZCA Fellows and ask them to view 2 videos of regional anaesthesia block performance. The videos are scripted, and the trainee, anaesthetic assistant, and patient are roleplayed. Participants will view each video and score on the DOPS form. The results from all participants are analysed to estimate the degree of consistency by using measures of correlation.

No related papers found yet.