Display apparatus, virtual reality display system having the same and method of estimating user motion based on input image

Abstract

A display apparatus includes a display panel, a driving controller and a data driver. The display panel displays an image based on input image data. The driving controller generates a data signal based on the input image data, determines an optical flow based on previous frame data of the input image data and present frame data of the input image data and determines a user's self-motion using the optical flow. The data driver converts the data signal to a data voltage and outputs the data voltage to the display panel.

What is claimed is: 
     
       1. A display apparatus comprising:
 a display panel which displays an image based on input image data; 
 a driving controller which generates a data signal based on the input image data, determines an optical flow based on previous frame data of the input image data and present frame data of the input image data and determines a user's self-motion based on the optical flow; and 
 a data driver which converts the data signal to a data voltage and outputs the data voltage to the display panel, 
 wherein the driving controller comprises an optical flow estimator which applies a high boost filter to the input image data and determines a first optical flow of first high-boosted image data generated by applying the high boost filter to the input image data, applies the high boost filter to first resized data having a reduced size from the input image data and determines a second optical flow of second high-boosted image data generated by applying the high boost filter to the first resized data, and estimates the optical flow of the input image data based on the first optical flow and the second optical flow. 
 
     
     
       2. The display apparatus of  claim 1 , wherein the driving controller further comprises:
 a feature extractor which extracts features to determine a motion type and a motion direction in the image from the optical flow. 
 
     
     
       3. The display apparatus of  claim 2 , wherein the feature extractor generates a saliency map representing the motion direction and a motion magnitude in the image from the optical flow. 
     
     
       4. The display apparatus of  claim 3 , wherein when the saliency map of the present frame data is denoted by Saliency_map(t), an average of the saliency map from a present frame to k previous frames is denoted by Average(t-k;t), and a self-motion saliency map of the present frame data is denoted by Saliency_Difference(t), the following equation is satisfied: |Saliency_map(t)−Average(t-k;t)|=Saliency_Difference(t). 
     
     
       5. The display apparatus of  claim 2 , wherein the feature extractor generates a histogram of direction representing a frequency of each direction of the optical flow. 
     
     
       6. The display apparatus of  claim 5 , wherein when all of components of the histogram of direction are less than a threshold value, the feature extractor determines the present frame data as a scene motion in which the user's self-motion is less. 
     
     
       7. The display apparatus of  claim 6 , wherein when at least one of the components of the histogram of direction is greater than the threshold value, the feature extractor determines that the present frame data include the user's self-motion. 
     
     
       8. The display apparatus of  claim 7 , wherein when at least one of the components of the histogram of direction is greater than the threshold value, the feature extractor determine a direction having a greatest frequency among the components of the histogram of direction as a direction of the user's self-motion. 
     
     
       9. The display apparatus of  claim 5 , wherein the optical flow estimator estimates the optical flow of the input image data by multiplying the second optical flow by a weight and adding the first optical flow thereto. 
     
     
       10. The display apparatus of  claim 9 , wherein the weight is greater than one. 
     
     
       11. The display apparatus of  claim 3 ,
 wherein the optical flow estimator applies the high boost filter to second resized data having a reduced size from the first resized data and determines a third optical flow of third high-boosted image data generated by applying the high boost filter to the second resized data, and 
 wherein the optical flow estimator estimates the optical flow of the input image data based on the first optical flow, the second optical flow and the third optical flow. 
 
     
     
       12. The display apparatus of  claim 11 ,
 wherein the optical flow estimator applies the high boost filter to third resized data having a reduced size from the second resized data and determines a fourth optical flow of fourth high-boosted image data generated by applying the high boost filter to the third resized data, and 
 wherein the optical flow estimator estimates the optical flow of the input image data based on the first optical flow, the second optical flow, the third optical flow and the fourth optical flow. 
 
     
     
       13. The display apparatus of  claim 12 , wherein the optical flow estimator varies a number of resizing operations based on the input image data. 
     
     
       14. The display apparatus of  claim 1 , wherein when the input image data is denoted by ORIGINAL, a result of applying a low pass filter to the input image data is denoted by LOW PASS, a high boost parameter is denoted by β, a result of applying the high boost filter to the input image data is denoted by HIGH BOOST, the following equation is satisfied: HIGH BOOST=β×ORIGINAL−LOW PASS. 
     
     
       15. The display apparatus of  claim 14 , wherein the high boost parameter is greater than one. 
     
     
       16. A virtual reality display system comprising:
 a lens unit; 
 a display apparatus comprising:
 a display panel which displays an image based on input image data; 
 a driving controller which generates a data signal based on the input image data, determines an optical flow based on previous frame data of the input image data and present frame data of the input image data and determines a user's self-motion using the optical flow; and 
 a data driver which converts the data signal to a data voltage and outputs the data voltage to the display panel; and 
 
 a housing which receives the lens unit and the display apparatus, 
 wherein the driving controller comprises an optical flow estimator which applies a high boost filter to the input image data and determines a first optical flow of first high-boosted image data generated by applying the high boost filter to the input image data, applies the high boost filter to first resized data having a reduced size from the input image data and determines a second optical flow of second high-boosted image data generated by applying the high boost filter to the first resized data, and estimates the optical flow of the input image data based on the first optical flow and the second optical flow. 
 
     
     
       17. A method of estimating a user motion based on an input image, the method comprising:
 determining an optical flow based on previous frame data of input image data and present frame data of the input image data in a unit of a pixel; 
 extracting features from the optical flow to determine a motion type and a motion direction in the input image; and 
 determining a user's self-motion using the features, 
 wherein the determining the optical flow in the unit of the pixel comprises: 
 applying a high boost filter to the input image data; 
 determining a first optical flow of first high-boosted image data generated by applying the high boost filter to the input image data; 
 generating first resized data having a reduced size from the input image data; 
 applying the high boost filter to the first resized data; 
 determining a second optical flow of second high-boosted image data generated by applying the high boost filter to the first resized data; and 
 determining the optical flow based on the first optical flow and the second optical flow. 
 
     
     
       18. The method of  claim 17 ,
 wherein the extracting features from the optical flow comprises generating a histogram of direction representing a frequency of each direction of the optical flow, 
 wherein the determining the user's self-motion using the features comprises determining that the present frame data include the user's self-motion when at least one of components of the histogram of direction is greater than a threshold value.