5/29/2023 0 Comments Lost judgment kappa![]() When observers classify items on a categorical scale, reliability refers to the ability of the scale to differentiate between the items, despite the presence of measurement error while agreement refers to the degree of closeness between two assessments made on the same items. ![]() They both provide information about the quality of the measurements (Kottner et al., 2011). ![]() Reliability and agreement studies are of paramount importance in behavioral, social, biological, and health sciences. Although this paper focuses on kappa coefficients, the method easily extends to other statistical measures. Moreover, the statistical properties of the proposed method are studied using simulations. The method is based on simple matrix calculations and is available in the R package “multiagree”. Dependent kappa coefficients can result from the assessment of the same individuals at various occasions or when each member of a group is compared to an expert, for example. This situation frequently arises in medical sciences, where multilevel data are common. The present paper provides two simple alternatives to more advanced modeling techniques, which are not always adequate in case of a very limited number of subjects, when comparing several dependent kappa coefficients obtained on multilevel data. When the scale is categorical, reliability and agreement can be quantified through different kappa coefficients. They provide information about the quality of the measurements. Reliability and agreement are two notions of paramount importance in medical and behavioral sciences.
0 Comments
Leave a Reply. |