Assessment of ophthalmology resident on-call performance

Purpose: To design and implement a valid tool for assessment of ophthalmology resident on-call performance. Design: Retrospective chart audit. Setting: Tertiary care academic ophthalmology programs. Participants: Ophthalmology faculty and residents at the University of Cincinnati and the University of Iowa. Methods: A 1-page on-call assessment tool (OCAT) and scoring rubric were developed to evaluate ophthalmology resident on-call performance. A retrospective chart audit of consecutive resident on-call charts was performed at the University of Cincinnati and the University of Iowa, and resident performance was scored using the OCAT. Results: A consensus of faculty comments established the face and content validity of the OCAT. One hundred ninety-one on-call consultations were assessed. Timeliness of consultation was the most common category receiving a borderline or unsatisfactory rating. Borderline ratings in knowledge-based categories (history, examination, assessment and plan, urgency rating) occurred more often for postgraduate year 2 (PGY2) residents than for PGY3 residents (P = 0.05, chi-square test). Incomplete differential diagnosis (n = 6) and lack of follow-up instruction (n = 5) were the most common deficiencies observed. Conclusions: The OCAT has face, content, and discriminative validity. It can be used to assess resident competence in patient care, professionalism, and medical knowledge. Interrater and intrarater reliability still need to be determined. The OCAT may prove to be an additional assessment tool for meeting the Accreditation Council for Graduate Medical Education competencies mandate.