How many pathologists do you need to correctly diagnose cancer?

A recently released study in the journal Statistics in Medicine describes a new method, named the Observers Needed for Evaluation of Subjective Tests (ONEST), developed to determine the optimal number of pathologists needed for a correct diagnosis.

The method was developed by Gang Han, associate professor in the Department of Epidemiology and Biostatistics at the Texas A&M University School of Public Health, with former graduate student Bohong Guo and colleagues from the Moffitt Cancer Center & Research Institute, Saint Louis University School of Medicine, and Yale University School of Medicine.

Han and colleagues developed a statistical framework to assess the performance of a diagnostic test with multiple observers. The proposed method includes an exploratory analysis, a statistical test of whether the observers’ agreement percentage will plateau to a non-zero value, and a statistical model to estimate the agreement percentage and the number of observers for reaching the plateau.This method was applied in a non-small cell lung cancer example and a triple negative breast cancer example. Reads of the immunohistochemical tests with SP142 and SP263 assays for expression of Programmed death-ligand 1 (PD-L1) were used to determine the number of observers needed for evaluation of the subjective tests.