Objective: To explore the methods of estimating measurement reliability and their limitations.
Methods: According to the given examples, reliabilities of measurement were calculated and limitations of the methods of estimating reliability were analyzed.
Results: The Kappa value of interobserver reliability was 0.793 between two pathologists. Between the two populations with different prevalence rates, the values of Kappa were 0.800 and 0.137 respectively, and with big difference. Cronbach's alpha coefficient of compositive index for A type behavior was 0.55.
Conclusion: The Kappa index and alpha coefficient were both inherently population-specific. Before generalizing to different populations, the reliability needs to be measured.