Application of naïve Bayesian artificial intelligence to referral refinement of chronic open angle glaucoma

Abstract

The purpose of this study was to determine whether naïve Bayesian artificial intelligence could accurately predict clinical decisions made during the referral refinement of Chronic open angle glaucoma (COAG) by three specialist independent prescribing optometrists using the highly structured standard operating procedure (SOP) adopted by the Community Ophthalmology Team (COT) of the West Kent Clinical Commissioning Group (CCG). The effectiveness of the COT, in terms of reducing false positive referrals and costs to the National Health Service (NHS), was also explored. This was the first study of its kind. Treating the study as a clinical audit allowed collection of unconsented fully anonymised data from the worst affected eyes or right eyes of 1006 cases referred into the COT. Each case was classified according to race, sex, age, family history of COAG, reason for referral, intraocular pressure and its inter-ocular asymmetry (Goldmann Applanation Tonometry), several optic nerve head dimensions (vertical size, cup disc ratio and its inter-ocular asymmetry; dilated stereoscopic slit lamp biomicroscopy with Volk lens), central corneal thickness (ultrasound pachymetry) and the severity of any visual field defects (Humphrey Visual Field Analyser, SITA FAST 24-2 testing strategy, Hodapp-Parrish-Anderson classification). Grouping of data into multiple cut-off points was informed by previous research and National Institute for Health and Care Excellence (NICE) guidelines. Preliminary analyses showed that most cases (79%) were discharged, 7% were followed up and 14% were referred to the NHS hospital eye service. The high discharge rate led to NHS cost savings of over £50 per case. Previous reports of increased intraocular pressure with central corneal thickness and increased cup disc ratios with cup disc size were also confirmed. Despite a high degree of inter-dependency between clinical tests, which violated the key assumption of naïve Bayesian analyses, the scheme learned rapidly and its weighted accuracy, based on randomised stratified tenfold cross-validation, was high (95%, 2.0% SD). However, false discharge (3.4%, 1.6% SD) and referral rates (3.1%, 1.5% SD) were considered unsafe. Making the analysis cost sensitive led to an 80 fold increase in COT follow-ups that would have reduced cost effectivity. The transferability of likelihood ratios was explored along with their use, compared to Chi-square, to rank clinical tests and explore redundancy in the SOP adopted by the COT. In summary, high discharge rates were consistent with the level of false positive referrals for suspected COAG reported in the literature and reduced NHS costs. Although use of a structured SOP led to high accuracy, naïve Bayesian artificial intelligence could not safely predict the decisions of COT optometrists as it caused too many false discharges and referrals. More sophisticated forms of machine learning need to be explored.