Bright Spot Removal Using A Neural Network

Abstract

A method for image capture includes identifying a bright spot in an image. A neural network is used to recover details in bright spot area through a trained de-noising process. Post-processing of the image is conducted to match image parameters of recovered details in the bright spot area to another area of the image.

1 . A method to perform supervised training on a neural network, the method comprising:
 receiving source data and target data associated with the supervised training;   independently packaging the source data and the target data;   preprocessing the packaged source data with a source lambda and the packaged target data with a target lambda;   generating target output data based on the source lambda-based preprocessing;   receiving the preprocessed packaged target data by the neural network;   the neural network processing the preprocessed packaged data;   the neural network generating source output data based on the processing; and   generating an objective function based on the target output data and the source output data.   
     
     
         2 . The method of  claim 1 , wherein the supervised training is associated with recovering detail in a first area of a digital photograph that contains at least one bright spot by performing a de-noising process by the neural network. 
     
     
         3 . The method of  claim 1 , wherein the source data and target data are sourced from any combination of a profiling system, one or more mathematical models and one or more publicly available datasets. 
     
     
         4 . The method of  claim 1 , wherein the packaging further comprises:
 normalizing the each of the source data and the target data according to a respective determined scheme; and   arranging the respective normalized data for input to the neural network in a corresponding tensor.   
     
     
         5 . The method of  claim 4 , wherein each of the source lambda and the target lambda adds synthetic noise or bright-spots to the tensor associated with the source data and the target data, respectively. 
     
     
         6 . The method of  claim 1 , further comprising classifying one or more objects in a tensor using a classifier head. 
     
     
         7 . The method of  claim 1 , wherein the source output data, the target output data, and the objective function collectively define a loss function to be reduced in value by the neural network. 
     
     
         8 . The method of  claim 1 , further comprising training the neural network to both identify and reduce one or more bright spots simultaneously in an image. 
     
     
         9 . The method of  claim 8 , further comprising receiving the image from a camera associated with a vehicle imaging system. 
     
     
         10 . The method of  claim 8 , further comprising receiving the image from a surgical imaging and teleoperation system with active illumination. 
     
     
         11 . The method of  claim 8 , wherein the image is a part of a digital video stream. 
     
     
         12 . The method of  claim 1 , wherein each of the source data and the target data is sequential data or temporal data. 
     
     
         13 . The method of  claim 1 , wherein the neural network is a fully convolutional neural network. 
     
     
         14 . The method of  claim 13 , wherein the fully convolutional neural network comprises:
 a plurality of first pairs of 3×3 convolutions each followed by a first rectified linear unit and a 2×2 max pooling operation with stride 2; and   a plurality of second pairs of 3×3 convolutions followed by 2×2 up-convolution and a second rectified linear unit.   
     
     
         15 . The method of  claim 14 , further comprising training the fully convolutional neural network as part of a generative adversarial network including a network trained to add synthetic bright spots. 
     
     
         16 . The method of  claim 1 , wherein each of the source data and the target data includes imaging system profiling data. 
     
     
         17 . The method of  claim 16 , wherein the imaging system profiling data includes raw profiling data that further includes any combination of natural and manmade scenes captured from a reference imaging system, and procedurally-generated scenes. 
     
     
         18 . The method of  claim 1 , further comprising generating the source data and the target data by a profiling process that includes knowledge of a photon noise floor associated with an imaging system. 
     
     
         19 . The method of  claim 18 , wherein the profiling process includes imaging system sensor pixel pitch, display system pixel dimensions, imaging system focal length, imaging system working f-number, number of sensor pixels, and number of display system pixels. 
     
     
         20 . The method of  claim 1 , further comprising performing warps and deformation to the any combination of the source data and the target data for alignment purposes.