Survey of Ophthalmology
Volume 47, Issue 4 , Pages 375-386 , July 2002

The Reliability of Clinical Methods in Ophthalmology

  • Curtis E. Margo, MD, MPH

      Affiliations

    • The Department of Ophthalmology, Watson Clinic, Lakeland, FL, USA
    • ,
  • Lynn E. Harman, MD

      Affiliations

    • Eye Specialists of Mid-Florida, Winter Haven, and the Department of Ophthalmology University of South Florida, College of Medicine, Tampa, FL, USA
    • ,
  • Zuber D. Mulla, MSPH, PhD

      Affiliations

    • Department of Epidemiology and Biostatistics, University of South Florida, Tampa, FL, USA
    • Corresponding Author InformationReprints of this article are not available

References 

  1. Photodynamic therapy with verteporfin for age-related macular degeneration. American Academy of Ophthalmology. Ophthalmology. 2000;107:2314–2317
  2. Optic nerve head and retinal nerve fiber layer analysis. American Academy of Ophthalmology. Ophthalmology. 1999;106:1414–1424
  3. Photodynamic therapy of subfoveal choroidal neovascularization in age-related macular degeneration with verteporfin (one-year results of 2 randomized clinical trials—TAP report. Treatment of age-related macular degeneration with photodynamic therapy (TAP)). Arch Ophthalmol. 1999;117:1329–1345
  4. Occult choroidal neovascularization. Influence on visual outcome in patients with age-related macular degeneration. Macular Photocoagulation Study Group. Arch Ophthalmol. 1996;114:400–412
  5. Visual outcome after laser photocoagulation for subfoveal choroidal neovascularization secondary to age-related macular degeneration. The influence of initial lesion size and initial visual acuity. Macular Photocoagulation Study Group. Arch Ophthalmol. 1994;112:480–488
  6. Laser photocoagulation of subfoveal neovascular lesions of age-related macular degeneration. Updated findings from two clinical trials. Macular Photocoagulation Study Group. Arch Ophthalmol. 1993;111:1200–1209
  7. Subfoveal neovascular lesions in age-related macular degeneration. Guidelines for evaluation and treatment in the macular photocoagulation study. Macular Photocoagulation Study Group. Arch Ophthalmol. 1991;109:1242–1257
  8. Abrams LS, Scott IU, Spaeth GL, et al.  Agreement among optometrists, ophthalmologists, and residents in evaluating the optic disc for glaucoma. Ophthalmology. 1994;101:1662–1667
  9. Armaly MF, Sayegh RE. The cup-disc ratio. The findings of tonometry and tonography in the normal eye. Arch Ophthalmol. 1969;82:191–196
  10. Baak JP, Langley FA, Talerman A, Delemarre JF. The prognostic variability of ovarian tumor grading by different pathologists. Gynecol Oncol. 1987;27:166–172
  11. Berry V, Drance SM, Wiggins RL, Schulzer M. A study of the errors of applanation tonometry and tonography on two groups of normal people. Can J Ophthalmol. 1966;1:213–220
  12. Blackman NJ, Koval JJ. Interval estimation for Cohens kappa as a measure of agreement. Stat Med. 2000;19:723–741
  13. Bland JM, Altman DG. Comparing methods of measurement (why plotting difference against standard method is misleading). Lancet. 1995;346:1085–1087
  14. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310
  15. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46
  16. Coindre JM, Trojani M, Contesso G, et al.  Reproducibility of a histopathologic grading system for adult soft tissue sarcoma. Cancer. 1986;58:306–309
  17. Coleman AL, Sommer A, Enger C, et al.  Interobserver and intraobserver variability in the detection of glaucomatous progression of the optic disc. J Glaucoma. 1996;5:384–389
  18. Cook RJ. Kappa. In:  Armitage P,  Colton T editor. Encyclopedia of Biostatistics, Vol 2. Chichester: John Wiley & Sons; 1998;p. 2160–2166
  19. Cook RJ. Kappa and its dependence on marginal rates. In:  Armitage P,  Colton T editor. Encyclopedia of Biostatistics, Vol 3. Chichester: John Wiley & Sons; 1998;p. 2166–2168
  20. Cooper RL, Alder VA, Constable IJ. Measurement vs. judgement of cup-disc ratios (statistical evaluation of intraobserver and interobserver error). Glaucoma. 1982;4:169–176
  21. Daniel WW. Biostatistics (A Foundation for Analysis in the Health Sciences). New York: John Wiley & Sons; 1991;ed 5, pp 399–407
  22. Dielemans I, Vingerling JR, Hofman A, et al.  Reliability of intraocular pressure measurement with the Goldmann applanation tonometer in epidemiological studies. Graefes Arch Clin Exp Ophthalmol. 1994;232:141–144
  23. Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic (confidence interval construction, significance-testing and sample size estimation). Stat Med. 1992;11:1511–1519
  24. Elmore JG, Feinstein AR. A bibliography of publications on observer variability (final installment). J Clin Epidemiol. 1992;45:567–580
  25. Farmer ER, Gonin R, Hanna MP. Discordance in the histopathologic diagnosis of melanoma and melanocytic nevi between expert pathologists. Hum Pathol. 1996;27:528–531
  26. Fechtner RD, Lama P. Advances in optic nerve head analysis in glaucoma. Semin Ophthalmol. 1999;14:180–188
  27. Feinstein AR. A bibliography of publications on observer variability. J Chronic Dis. 1985;38:619–632
  28. Fleiss J. Statistical Methods for Rates and Proportions. New York, NY: John Wiley & Sons Inc; 1981;ed 2, pp 212–36
  29. Fletcher CM. The problem of observer variation in medical diagnosis with special reference to chest diseases. Methods Inf Med. 1965;3:98–103
  30. Foster PJ, Wong JS, Wong E, et al.  Accuracy of clinical estimates of intraocular pressure in Chinese eyes. Ophthalmology. 2000;107:1816–1821
  31. Friedman SM, Margo CE. Choroidal neovascular membranes (reproducibility of angiographic interpretation). Am J Ophthalmol. 2000;130:839–841
  32. Galen RS, Gambino SR. Beyond Normality (The Predictive Value and Efficiency of Medical Diagnoses). New York: John Wiley & Sons; 1975;
  33. Garland LH. The problem of observer error. Bull NY Acad Med. 1960;36:570–584
  34. Garland LH. Studies on the accuracy of diagnostic procedures. Am J Roentgenol Radium Ther Nucl Med. 1959;82:25–38
  35. Gilchrist KW, Kalish L, Gould VE, et al.  Interobserver reproducibility of histopathological features in stage II breast cancer. An ECOG study. Breast Cancer Res Treat. 1985;5:3–10
  36. Grossniklaus HE, Margo CE, Solomon AR. Indeterminate melanocytic proliferations of the conjunctiva. Arch Ophthalmol. 1999;117:1131–1136
  37. Haslett RS, Batterbury M, Cuypers M, Cooper RL. Inter- observer agreement in clinical optic disc measurement using a modified 60 D lens. Eye. 1997;11:692–697
  38. Hatch WV, Trope GE, Buys YM, et al.  Agreement in assessing glaucomatous discs in a clinical teaching setting with stereoscopic disc photographs, planimetry, and laser scanning tomography. J Glaucoma. 1999;8:99–104
  39. Hirsch FR, Matthews MJ, Yesner R. Histopathologic classification of small cell carcinoma of the lung (comments based on an interobserver examination). Cancer. 1982;50:1360–1366
  40. Hulley SB, Cummings SR. Planning the measurements (precision and accuracy). In:  Hulley SB,  Cummings SR editor. Designing Clinical Research. Baltimore: Williams & Wilkins; 1998;Ch 3
  41. Kassirer JP. The principles of clinical decision making (an introduction to decision analysis). Yale J Biol Med. 1976;49:149–164
  42. Kassirer JP, Kopelman RI. Learning Clinical Reasoning. Baltimore: Williams & Wilkins; 1991;
  43. Kelsey JL, Whittemore AS, Evans AS, Thompsons WD. Methods in Observational Epidemiology. New York: Oxford University Press; 1996;ed 2, pp 346–7, 412
  44. Klein BE, Magli YL, Richie KA, et al.  Quantitation of optic disc cupping. Ophthalmology. 1985;92:1654–1656
  45. Klein BE, Moss SE, Magli YL, et al.  Optic disc cupping as clinically estimated from photographs. Ophthalmology. 1987;94:1481–1483
  46. Koran LM. The reliability of clinical methods, data and judgments (first of two parts). N Engl J Med. 1975;293:642–646
  47. Krebs DE: Computer communication. Phys Ther 64:1102,1106,1108–9, 1984
  48. Kylstra JA, Brown JC, Jaffe GJ, et al.  The importance of fluorescein angiography in planning laser treatment of diabetic macular edema. Ophthalmology. 1999;106:2068–2073
  49. Lahey MA, Downey RG, Saal FE. Intraclass correlations (there is more there than meets the eye). Psychol Bull. 1983;93:586–595
  50. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174
  51. In:  Last JM editors. A Dictionary of Epidemiology, third edition. New York: Oxford University Press; 1995;
  52. Lichter PR. Variability of expert observers in evaluating the optic disc. Trans Am Ophthalmol Soc. 1976;74:532–572
  53. Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol. 1987;126:161–169
  54. Margo CE, Roper DL, Hidayat AA. Borderline melanocytic tumor of the conjunctiva (diagnostic and therapeutic considerations). J Pediatr Ophthalmol Strabismus. 1991;28:268–270
  55. Motolko MA, Feldman F, Hyde M, Hudy D. Sources of variability in the results of applanation tonometry. Can J Ophthalmol. 1982;17:93–95
  56. Nam JM. Interval estimation of the kappa coefficient with binary classification and an equal marginal probability model. Biometrics. 2000;56:583–585
  57. Nelson LM, Longstreth WT, Koepsell TD, van Belle G. Proxy respondents in epidemiologic research. Epidemiol Rev. 1990;12:71–86
  58. Phelps CD, Phelps GK. Measurement of intraocular pressure (a study of its reproducibility). Albrecht Von Graefes Arch Klin Exp Ophthalmol. 1976;198:39–43
  59. Read RM, Spaeth GL. The practical clinical appraisal of the optic disc in glaucoma (the natural history of cup progression and some specific disc-field correlations). Trans Am Acad Ophthalmol Otolaryngol. 1974;78:OP255–OP274
  60. Rothstein JM, Echternach JL. Primer on Measurement (An Introductory Guide to Measurement Issues). Alexandria, VA: American Physical Therapy Association; 1999;
  61. Schmidt TAF. The clinical application of the Goldmann applanation tonometer. Am J Ophthalmol. 1960;49:967–968
  62. Schwartz B. Cupping and pallor of the optic disc. Arch Ophthalmol. 1973;89:269–274
  63. Shah S. Accurate intraocular pressure measurement-the myth of modern ophthalmology?. Ophthalmology. 2000;107:1805–1807
  64. Shrout PE, Fleiss JL. Intraclass correlations (uses in assessing rater reliability). Psychol Bull. 1979;86:420–428
  65. Shuttleworth GN, Khong CH, Diamond JP. A new digital optic disc stereo camera (intraobserver and interobserver repeatability of optic disc measurements). Br J Ophthalmol. 2000;84:403–407
  66. Speicher CE. All laboratory tests are not created equal. Arch Pathol Lab Med. 1985;109:709–710
  67. Speicher CE. The Right Test. A Physician's Guide to Laboratory Medicine. Philadelphia: W.B. Saunders; 1993;ed 2
  68. Spencer AF, Vernon SA. Optic disc height measurement with the Zeiss 4-mirror contact lens and 78 dioptre lens compared. Eye. 1996;10:371–376
  69. Takamoto T, Schwartz B. Reproducibility of photogrammetric optic disc cup measurements. Invest Ophthalmol Vis Sci. 1985;26:814–817
  70. Tielsch JM, Katz J, Quigley HA, et al.  Intraobserver and interobserver agreement in measurement of optic disc characteristics. Ophthalmology. 1988;95:350–356
  71. Trobe JD, Glaser JS, Cassady JC. Optic atrophy. Differential diagnosis by fundus observation alone. Arch Ophthalmol. 1980;98:1040–1045
  72. Varma R, Spaeth GL, Steinmann WC, Katz LJ. Agreement between clinicians and an image analyzer in estimating cup-to-disc ratios. Arch Ophthalmol. 1989;107:526–529
  73. Varma R, Steinmann WC, Scott IU. Expert agreement in evaluating the optic disc for glaucoma. Ophthalmology. 1992;99:215–221
  74. Whitacre MM, Stein R. Sources of error with use of Goldmann-type tonometers. Surv Ophthalmol. 1993;38:1–30
  75. Wilke K. Effects of repeated tonometry (genuine and sham measurements). Acta Ophthalmol (Copenh). 1972;50:574–582
  76. Zhou XH, Higgs RE. Assessing the relative accuracies of two screening tests in the presence of verification bias. Stat Med. 2000;19:1697–1705

☆ The authors have no proprietary or commercial interest in any product mentioned or concept discussed in this article.

PII: S0039-6257(02)00312-0

Survey of Ophthalmology
Volume 47, Issue 4 , Pages 375-386 , July 2002

Article Tools

* Image Usage & Resolution