Title: Variance estimates and confidence intervals for the Kappa measure of classification accuracy
Author: Kalkhan, M. A.; Reich, R. M.; Czaplewski, R. L.;
Source: Canadian Journal of Remote Sensing. 23(3): 210-216.
Publication Series: Scientific Journal (JRNL)
The Kappa statistic is frequently used to characterize the results of an accuracy assessment used to evaluate land use and land cover classifications obtained by remotely sensed data. This statistic allows comparisons of alternative sampling designs, classification algorithms, photo-interpreters, and so forth. In order to make these comparisons, it is important to know how far in error the estimate might reasonably be. This is accomplished by constructing confidence intervals around the point estimate. The decision to use either the asymptotic variance formulae or bootstrapping variance estimates in constructing confidence intervals for the Kappa statistic is not a simple task. This study was designed to help answer this question. Nine error matrices representing three levels of accuracy (poor, average, and good) of Landsat TM Data consisting of 4, 8 and 16 land cover types in North Carolina were used in this study. Each error matrix was sampled, with replacement, using sample sizes of 50, 100, 150, 300 and 800 pixels to obtain estimates of the Kappa statistic and sample variance. Each of the sample error matrices were resampled 500 times to obtain bootstrap estimates of the variance. The asymptotic variance formula for the Kappa statistic and bootstrap variance provided unbiased estimates of the sample variance. In general, the asymptotic variance estimates were larger than those obtained using bootstrapping, even though the differences were not significant. Confidence intervals based on percentiles of the bootstrap distribution provided the best 95 percent coverage rates (92 to 96 percent with a median of 95 percent). The lowest 95 percent coverage rates were obtained using the bootstrap variance estimate (median of 83 percent).
Keywords: Kappa statistic, land use, and cover classifications, remotely sensed data
- We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
- This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.
XML: View XML
Kalkhan, M. A.; Reich, R. M.; Czaplewski, R. L. 1997. Variance estimates and confidence intervals for the Kappa measure of classification accuracy. Canadian Journal of Remote Sensing. 23(3): 210-216.
Get the latest version of the Adobe Acrobat reader or Acrobat Reader for Windows with Search and Accessibility