Processing fundus images using machine learning models

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing fundus images using fundus image processing machine learning models. One of the methods includes obtaining a model input comprising one or more fundus images, each fundus image being an image of a fundus of an eye of a patient; processing the model input using a fundus image processing machine learning model, wherein the fundus image processing machine learning model is configured to process the model input comprising the one or more fundus image to generate a model output; and processing the model output to generate health analysis data.

What is claimed is: 
     
         1 . (canceled) 
     
     
         2 . A method comprising:
 obtaining a model input comprising one or more fundus images, each fundus image being an image of a fundus of an eye of a patient;   processing the model input using a fundus image processing machine learning model, wherein the fundus image processing machine learning model is configured to process the model input comprising the one or more fundus images to generate a model output that characterizes the health of the patient with respect to one or more risk factors for a type of neurological disorder; and   processing the model output to generate health analysis data that characterizes an aspect of the health of the patient with respect to the one or more risk factors for the type of neurological disorder.   
     
     
         3 . The method of  claim 2 , wherein the model input further comprises other patient data comprising ocular measurement data, patient demographic data, or both. 
     
     
         4 . The method of  claim 2 , wherein the fundus image processing machine learning model is a feedforward machine learning model, and wherein the one or more fundus images comprise a single fundus image that captures a current state of the fundus of the eye of the patient. 
     
     
         5 . The method of  claim 2 , wherein the fundus image processing machine learning model is a feedforward machine learning model, and wherein the one or more fundus images comprise a plurality of fundus images that each capture a different aspect of a current state of the fundus of the eye of the patient. 
     
     
         6 . The method of  claim 2 , wherein the fundus image processing machine learning model is a recurrent machine learning model, and wherein the one or more fundus images comprise a temporal sequence of a plurality of fundus images that capture how the fundus of the patient has evolved over time. 
     
     
         7 . The method of  claim 2 , wherein the model output comprises a respective predicted value for each of the one or more risk factors for the type of neurological disorder. 
     
     
         8 . The method of  claim 7 , wherein the type of neurological disorder includes a neurodegenerative condition, the neurodegenerative condition including Parkinson's disease and Alzheimer's disease. 
     
     
         9 . The method of  claim 2 , wherein the fundus image processing machine learning model comprises an ensemble of machine learning models. 
     
     
         10 . The method of  claim 9 , wherein each of the ensemble of machine learning models is configured to generate a predicted value for a different subset of the one or more risk factors for the type of neurological disorder. 
     
     
         11 . The method of  claim 2 , wherein the fundus image processing machine learning model comprises an attention mechanism that is configured to:
 receive a respective feature vector for each of a plurality of regions in a fundus image of the one or more fundus images, the respective feature vector being generated by one or more initial layers of the fundus image processing machine learning model,   compute a respective attention weight for each of the regions, and   generate an attention output by attending to the respective feature vectors in accordance with the respective attention weights for the regions in fundus image; and   wherein the health analysis data comprises data identifying the respective attention weights generated by the attention mechanism.   
     
     
         12 . The method of  claim 11 , wherein the data identifying the respective attention weights is an attention map that specifies the respective attention weights for the regions in the fundus image. 
     
     
         13 . A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
 obtaining a model input comprising one or more fundus images, each fundus image being an image of a fundus of an eye of a patient;   processing the model input using a fundus image processing machine learning model, wherein the fundus image processing machine learning model is configured to process the model input comprising the one or more fundus images to generate a model output that characterizes the health of the patient with respect to one or more risk factors for a type of neurological disorder; and   processing the model output to generate health analysis data that characterizes an aspect of the health of the patient with respect to the one or more risk factors for the type of neurological disorder.   
     
     
         14 . The system of  claim 13 , wherein the model output comprises generating a respective predicted value for each of the one or more risk factors for the type of neurological disorder. 
     
     
         15 . The system of  claim 13 , wherein the fundus image processing machine learning model is a feedforward machine learning model, and wherein the one or more fundus images comprise a single fundus image that captures a current state of the fundus of the eye of the patient. 
     
     
         16 . The system of  claim 13 , wherein the fundus image processing machine learning model is a recurrent machine learning model, and wherein the one or more fundus images comprise a temporal sequence of a plurality of fundus images that capture how the fundus of the patient has evolved over time. 
     
     
         17 . One or more non-transitory computer-readable storage media encoded with instructions that when executed by one or more computers cause the one or more computers to perform to perform operations comprising:
 obtaining a model input comprising one or more fundus images, each fundus image being an image of a fundus of an eye of a patient;   processing the model input using a fundus image processing machine learning model, wherein the fundus image processing machine learning model is configured to process the model input comprising the one or more fundus images to generate a model output that characterizes the health of the patient with respect to one or more risk factors for a type of neurological disorder; and   processing the model output to generate health analysis data that characterizes an aspect of the health of the patient with respect to the one or more risk factors for the type of neurological disorder.   
     
     
         18 . The one or more non-transitory computer-readable storage media of  claim 17 , wherein the model output comprises a respective predicted value for each of the one or more risk factors for the type of neurological disorder. 
     
     
         19 . The one or more non-transitory computer-readable storage media of  claim 17 , wherein the type of neurological disorder includes a neurodegenerative condition, the neurodegenerative condition including Parkinson's disease and Alzheimer's disease. 
     
     
         20 . The one or more non-transitory computer-readable storage media of  claim 17 , wherein the fundus image processing machine learning model is a feedforward machine learning model, and wherein the one or more fundus images comprise a single fundus image that captures a current state of the fundus of the eye of the patient. 
     
     
         21 . The one or more non-transitory computer-readable storage media of  claim 17 , wherein the fundus image processing machine learning model is a recurrent machine learning model, and wherein the one or more fundus images comprise a temporal sequence of a plurality of fundus images that capture how the fundus of the patient has evolved over time.