Skip to the content

The influence of experience and training in a group of novice observers : A jackknife alternative free-response receiver operating characteristic analysis

Buissink, V, Thompson, John D., Voet, M and Hogg, P 2014, 'The influence of experience and training in a group of novice observers : A jackknife alternative free-response receiver operating characteristic analysis' , Radiography, 20 (4) , pp. 300-305.

[img] Microsoft Word - Accepted Version
Download (38kB)

Abstract

Purpose: The study evaluates the pre- and post-training lesion localisation ability of a group of novice observers. Parallels are drawn with the performance of inexperienced radiographers taking part in preliminary clinical evaluation (PCE) and ‘red-dot’ systems, operating within radiography practice. Materials and methods: Thirty-four novice observers searched 92 images for simulated lesions. Pretraining and post-training evaluations were completed following the free-response the receiver operating characteristic (FROC) method. Training consisted of observer performance methodology, the characteristics of the simulated lesions and information on lesion frequency. Jackknife alternative FROC (JAFROC) and highest rating inferred ROC analyses were performed to evaluate performance difference on lesion-based and case-based decisions. The significance level of the test was set at 0.05 to control the probability of Type I error. Results: JAFROC analysis (F(3,33) ¼ 26.34, p < 0.0001) and highest-rating inferred ROC analysis (F(3,33) ¼ 10.65, p ¼ 0.0026) revealed a statistically significant difference in lesion detection performance. The JAFROC figure-of-merit was 0.563 (95% CI 0.512,0.614) pre-training and 0.677 (95% CI 0.639,0.715) post-training. Highest rating inferred ROC figure-of-merit was 0.728 (95% CI 0.701,0.755) pre-training and 0.772 (95% CI 0.750,0.793) post-training. Conclusions: This study has demonstrated that novice observer performance can improve significantly. This study design may have relevance in the assessment of inexperienced radiographers taking part in PCE or commenting scheme for trauma.

Item Type: Article
Themes: Health and Wellbeing
Schools: Schools > School of Health Sciences > Centre for Health Sciences Research
Journal or Publication Title: Radiography
Publisher: Elsevier
Refereed: Yes
ISSN: 1078-8174
Related URLs:
Funders: Erasmus
Depositing User: P Hogg
Date Deposited: 27 Jan 2015 17:55
Last Modified: 11 Jan 2016 13:42
URI: http://usir.salford.ac.uk/id/eprint/33467

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)

Downloads

Downloads per month over past year