Inter Rater Reliability

Often thought of as qualitative data, anything produced by the interpretation of laboratory scientists (as opposed to a measured value) is still a form of quantitative data, albeit in a slightly different form. As such different statistical methods from those used for data routinely assessed in the laboratory are required.

The approach taken will vary depending on the nature of the assessment being made, and the result being produced/reported. Analysis falling into each of these categories, and their application to uncertainty quantification/reliability of results is summarised below. The report provided will clearly outline which methods have been used along with an interpretation of results provided. In depth discussion of each of the methods outlined below is out with the scope of this protocol. The summary table below indicates in what situation each may be used. 


Method Usage
Cohen’s Kappa Statistic Categorical data – two observers
Fleiss’s Kappa and Conger’s Kappa Categorical data >2 observers
Weighted Kappa Ordinal data
Weighted Kappa with factors Ordinal data using Factors
Intraclass correlation coefficient Continuous data

Statistical methods employed for the assessment of inter observer reliability

Move on to The Budget

%d bloggers like this: