Systems and Methods for Reducing Noise in Video Streams

Abstract

Disclosed are apparatus and methods for denoising a video stream of a camera. A current frame of the video stream and a temporally adjacent frame of the video stream that has been previously spatially and temporally denoised are obtained. The current frame is first spatially denoised, while preserving edges in such current frame to generate a plurality of spatially denoised pixels for the current frame. A particular pixel of the current frame is then both spatially and temporally denoised based on a weighted averaging of the particular pixel's associated spatially denoised pixel from the current frame and a plurality of pixels from the temporally adjacent frame that have already been spatially and temporally denoised.

1 . A method of denoising a video stream of a camera, the method comprising:
 spatially denoising a current frame of the video stream to generate a plurality of spatially denoised pixels for the current frame; and   after the current frame is spatially denoised, temporally denoising a particular pixel of the current frame.   
     
     
         2 . The method of  claim 1 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel. 
     
     
         3 . The method of  claim 2 , wherein each pixel of the current frame being denoised and its proximate pixels include a 3 by 3 grid of pixels surrounding such pixel being denoised. 
     
     
         4 . The method of  claim 2 , wherein the bilateral filtering technique includes providing a weight for each proximate pixel that is inversely proportional to both a Euclidean distance and a color difference. 
     
     
         5 . The method of  claim 4 , wherein the plurality of pixels from the temporally adjacent frame include pixels at locations corresponding to the particular pixel and the proximate pixels. 
     
     
         6 . The method of  claim 5 , wherein the plurality of pixels from the temporally adjacent frame further include interpolated pixels at locations not corresponding to the particular pixel and the proximate pixels. 
     
     
         7 . The method of  claim 5 , wherein a weight of the particular pixel's associated spatially denoised pixel has a value of one. 
     
     
         8 . A camera comprising:
 one or more processors and memory that are operable to perform the following:   spatially denoising a current frame of a video stream of the camera to generate a plurality of spatially denoised pixels for the current frame; and   after the current frame is spatially denoised, temporally denoising a particular pixel of the current frame.   
     
     
         9 . The camera of  claim 8 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel. 
     
     
         10 . The camera of  claim 9 , wherein each pixel of the current frame being denoised and its proximate pixels include a 3 by 3 grid of pixels surrounding such pixel being denoised. 
     
     
         11 . The camera of  claim 9 , wherein the bilateral filtering technique includes providing a weight for each proximate pixel that is inversely proportional to both a Euclidean distance and a color difference. 
     
     
         12 . The camera of  claim 11 , wherein the plurality of pixels from the temporally adjacent frame include pixels at locations corresponding to the particular pixel and the proximate pixels. 
     
     
         13 . The camera of  claim 12 , wherein the plurality of pixels from the temporally adjacent frame further include interpolated pixels at locations not corresponding to the particular pixel and the proximate pixels. 
     
     
         14 . The camera of  claim 12 , wherein a weight of the particular pixel's associated spatially denoised pixel has a value of one. 
     
     
         15 . A non-transitory computer readable storage medium having computer program instructions stored thereon that are arranged to perform the following operations:
 spatially denoising a current frame of a video stream of a camera to generate a plurality of spatially denoised pixels for the current frame; and   after the current frame is spatially denoised, temporally denoising a particular pixel of the current frame.   
     
     
         16 . The non-transitory computer readable storage medium of  claim 15 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel. 
     
     
         17 . The method of  claim 1 , wherein the temporal denoising is based on a weighted function of a Euclidian distance of the pixel being weighted to the pixel being denoised and the color difference between the pixel being weighted and the pixel being denoised. 
     
     
         18 . The method of  claim 17 , wherein a large color difference will result in a lower weight while a low color difference will result in a higher weight. 
     
     
         19 . The camera of  claim 8 , wherein the temporal denoising is based on a weighted function of a Euclidian distance of the pixel being weighted to the pixel being denoised and the color difference between the pixel being weighted and the pixel being denoised. 
     
     
         20 . The camera of  claim 19 , wherein a large color difference will result in a lower weight while a low color difference will result in a higher weight.