Inter-rater reliability of paediatric emergency assessment : physiological and clinical features

Heal, C, Cotterill, S, Rowland, AG ORCID:, Garratt, N, Long, T ORCID:, Brown, S, O'Connor, G, Rishton, C, Woby, S and Roland, D 2021, 'Inter-rater reliability of paediatric emergency assessment : physiological and clinical features' , Archives of Disease in Childhood, 106 (2) , pp. 149-153.

PDF - Published Version
Available under License Creative Commons Attribution 4.0.

Download (351kB) | Preview
[img] PDF - Accepted Version
Restricted to Repository staff only

Download (220kB)
[img] Microsoft Word - Accepted Version
Restricted to Repository staff only

Download (60kB)


Objective The Paediatric Admission Guidance in the Emergency Department (PAGE) score is an assessment tool currently in development that helps predict hospital admission using components including patient characteristics, vital signs (heart rate, temperature, respiratory rate, oxygen saturation) and clinical features (e.g. breathing, behaviour, nurse judgement). It aims to assist in safe admission and discharge decision making in environments such as emergency departments and urgent care centres. Determining the inter-rater reliability of scoring tools such as PAGE can be difficult. The aim of this study was to determine the inter-rater reliability of seven clinical components of the PAGE Score. Design Inter-rater reliability was measured by each patient having their clinical components recorded by two separate raters in succession. The first rater was the assessing nurse, the second rater was a research nurse. Setting Two emergency departments and one urgent care centre in the North West of England. Measurements were recorded over one week; data was collected for half a day at each of the three sites. Patients A convenience sample of 90 paediatric attendees (aged 0 to 16), 30 from each of the three sites. Main Outcome Measures Two independent measures for each child were compared using kappa or prevalence-adjusted bias-adjusted kappa (PABAK). Bland-Altman plots were also constructed for continuous measurements. Results Inter-rater reliability ranged from moderate (0.62 [95% CI 0.48 to 0.74] weighted kappa) to very good (0.98 [95% CI 95 to 0.99] weighted kappa) for all measurements except ‘nurse judgement’ for which agreement was fair (0.30 95% CI [0.09 to 0.50] PABAK). Complete information from both raters on all the clinical components of the PAGE score were available for 73 children (81%). These total scores showed good’ inter-rater reliability (0.64 [95% CI 0.53 to 0.74] weighted kappa). Conclusions Our findings suggest different nurses would demonstrate good inter-rater reliability when collecting acute assessments needed for the PAGE score, reinforcing the applicability of the tool. The importance of determining reliability in scoring systems is highlighted and a suitable methodology presented.

Item Type: Article
Schools: Schools > School of Health and Society > Centre for Health Sciences Research
Schools > School of Health and Society > Centre for Applied Research in Health, Welfare and Policy
Journal or Publication Title: Archives of Disease in Childhood
Publisher: BMJ Publishing Group
ISSN: 0003-9888
Related URLs:
Funders: National Institute for Health Research (NIHR)
Depositing User: Professor Andrew G Rowland
Date Deposited: 16 Jul 2020 10:44
Last Modified: 16 Feb 2022 05:06

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)


Downloads per month over past year