Factors Associated with Performance Among Optometrists in Alberta, Canada:
A Predictive Analysis
DOI:
https://doi.org/10.15353/cjo.v87i4.6550Keywords:
optometrists, Continuing Competence, predictive modelling, risk factors, selection tool, AlbertaAbstract
Background: Risk and protective factors influencing the performance of health professionals are of significant interest to regulators and the public. We aimed to develop a predictive model to identify factors influencing optometrist performance, providing insights for improving regulatory oversight and supporting targeted interventions.
Methods: In our retrospective cohort study, we analyzed data from optometrists registered between 1987 and 2019 in the Alberta College of Optometrists Continuing Competence (CC) program to develop a predictive model for CC practice review outcome. We evaluated reviews using self-assessments, onsite visits, and clinical evaluations, with pass or fail status as the primary outcome. Key covariates included sex, age, training location, and previous review scores. We used a generalized additive model with a logit link and assessed its performance using five-fold cross-validation. Sensitivity and specificity were assessed with a holdout testing set.
Results: We analyzed 2,075 CC reviews of 916 optometrists. Of these reviews, 75.6% received a passing grade. Practitioners were primarily male (51.7%, 48.3% female) and trained in the United States (49.8%) or Canada (46.2%). Significant predictors of review outcome were sex, training location, previous review score, follow-up score, age (included as a nonlinear effect varying by sex), and years since last review. In developing a selection tool for future assessments, we replaced age with years since graduation and removed training location. Among the 388 practitioners selected for assessment since 2021, practitioners flagged as high risk had significantly higher failure rates (16.1%) compared with practitioners selected randomly (3.0%).
Discussion: Male sex, years since graduation, and poor outcomes on previous reviews emerged as significant predictors of failing an assessment. The developed selection tool effectively identified high-risk practitioners for reassessment, supporting fair and efficient resource allocation in the CC program.
Conclusions: Key factors influencing CC review outcomes were identified and a selection tool was developed to ensure fairness across subgroups defined by age and sex.
References
1. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012(6): Cd000259. doi:10.1002/14651858.CD000259.pub3
2. Government of Alberta. Regulated health professions and colleges. Accessed January 17, 2025. https://www.alberta.ca/regulated-health-professions
3. Austin EE, Do V, Nullwala R, et al. Systematic review of the factors and the key indicators that identify doctors at risk of complaints, malpractice claims or impaired performance. BMJ Open. 2021;11(8): e050377. doi:10.1136/bmjopen-2021-050377
4. Cooper WO, Martinez W, Domenico HJ, et al. Unsolicited patient complaints identify physicians with evidence of neurocognitive disorders. Am J Geriatr Psychiatry. 2018;26(9):927-36. doi:10.1016/j.jagp.2018.04.005
5. Fathy CA, Pichert JW, Domenico H, Kohanim S, Sternberg P, Cooper WO. Association between ophthalmologist age and unsolicited patient complaints. JAMA Ophthalmol. 2018;136(1):61-7. doi:10.1001/jamaophthalmol.2017.5154
6. Liu J, Hyman DA. Targeting Bad Doctors: Lessons from Indiana, 1975–2015. SSRN. Published January 29, 2018. Accessed January 7, 2025. doi:10.2139/ssrn.2994529
7. Thurman M, Cameron A, Pickles K. Risk in the optical professions: Final report General Optical Council. Published July 2019. Accessed January 17, 2025. https://optical.org/static/46fdcd8c-0777-4491-82df6cab23b1ddbf/risk-in-the-optical-professions-2019.pdf
8. Prajapati B, Dunne M, Bartlett H, Cubbidge R. The influence of learning styles, enrolment status and gender on academic performance of optometry undergraduates. Ophthalmic Physiol Opt. 2011;31(1):69–78. doi:10.1111/j.1475-1313.2010.00798.x
9. Wood, SN. Generalized Additive Models: An Introduction with R, Second Edition. Chapman and Hall/CRC; 2017. doi:10.1201/9781315370279
10. Zwick R. Fairness in measurement and selection: Statistical, philosophical, and public perspectives. Educational Measurement. 2019; 38(4): 34-41. doi:10.1111/emip.12299
11. Bismark MM, Spittal MJ, Gurrin LC, Ward M, Studdert DM. Identification of doctors at risk of recurrent complaints: A national study of healthcare complaints in Australia. BMJ Qual Saf. 2013;22(7):532-40. doi:10.1136/bmjqs-2012-001691
12. Studdert DM, Bismark MM, Mello MM, Singh H, Spittal MJ. Prevalence and characteristics of physicians prone to malpractice claims. N Engl J Med. 2016;374(4):354-362. doi:10.1056/NEJMsa1506137
13. Spittal MJ, Bismark MM, Studdert DM. Identification of practitioners at high risk of complaints to health profession regulators. BMC Health Serv Res. 2019;19(1): 380. doi.org/10.1186/s12913-019-4214-y
14. Fielding DW, Rogers WT, Tench E, O’Bryne CC, Page GG, Schulzer M. Predictors of pharmacists’ continuing competence. Am J Pharm Educ. 2001;65:106-118.
15. Alam A, Matelski JJ, Goldberg HR, Liu JJ, Klemensberg J, Bell CM. The characteristics of international medical graduates who have been disciplined by professional regulatory colleges in Canada: A retrospective cohort study. Acad Med. 2017;92(2):244-249. doi:10.1097/ACM.0000000000001356
16. Khaliq, AA, Dimassi H, Huang CY, Narine L, Smego RA Jr. Disciplinary action against physicians: Who is likely to get disciplined? Am J Med. 2005;118(7): 773-777. doi:10.1016/j.amjmed.2005.01.051
17. Croft E, Clark MT, Efstathiou N, Bradbury-Jones C. A focused mapping review and synthesis of a priori risk factors associated with medical misconduct. BMJ Open Qual. 2019;8(2): e000538. doi:10.1136/bmjoq-2018-000538
18. Dore KL, Reiter HI, Kreuger S, Norman GR. CASPer, an online pre-interview screen for personal/professional characteristics: Prediction of national licensure scores. Adv Health Sci Educ Theory Pract. 2017;22(2):327-336. doi:10.1007/s10459-016-9739-9
19. De Champlain AF, Ashworth N, Kain N, Qin S, Wiebe D, Tian F. Does pass/fail on medical licensing exams predict future physician performance in practice? A longitudinal cohort study of Alberta physicians. J. Med. Regul. 2020;106(4):17–26. doi:10.30770/2572-1852-106.4.17
20. Ivers N, Yogasingam S, Lacroix M, et al. Audit and feedback: Effects on professional practice. Cochrane Database Syst Rev. 2025;3(3):CD000259. doi:10.1002/14651858.CD000259.pub4
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Nigel Ashworth, Nicole Kain, Matthew Pietrosanu, Thomas Wilk, Kim Bugera, Homeira Hamayeli-Mehrabani, Nancy Hernandez-Ceron, Iryna Hurava, Kushagr Kumar
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
