Head-mounted display having an image sensor array

Abstract

Head-mounted Displays (HMDs) are commonly used for virtual reality, mixed reality, and augmented reality. HMDs are, by definition, worn on the head of a user to provide a display in the line of sight of the user. By viewing the display, the user is able to experience one of the aforementioned types of reality. Oftentimes, HMDs are configured to integrate live video captured from the user's perspective, especially in the case of the HMD providing augmented reality where a virtual environment is combined with video of the real world. The present disclosure provides a configuration for a HMD having an array of image sensors to accurately capture image data to form the live video from the user's perspective.

What is claimed is: 
     
       1. A method for a head-mounted display (HMD), comprising:
 calibrating the HMD for a user wearing the HMD, by:
 determining iris positions of the user, and 
 mapping every pixel of a display of the HMD to a separate image sensor in an image sensor array, based on the iris positions determined for the user; and 
 
 forming live video by the HMD via the image sensor array, by:
 capturing image data using the image sensors in the image sensor array, and 
 coloring the pixels of the display based on the image data and the mapping. 
 
 
     
     
       2. The method of  claim 1 , wherein coloring the pixels of the display based on the image data and the mapping includes:
 using each portion of image data captured by each image sensor to color the pixel to which the image sensor is mapped. 
 
     
     
       3. The method of  claim 1 , further comprising:
 integrating, by the HMD, the live video with a virtual environment to provide a modified virtual reality, a mixed reality, or an augmented reality to the user. 
 
     
     
       4. A method for a head-mounted display (HMD), comprising:
 calibrating the HMD for a user wearing the HMD, by:
 determining iris positions of the user, and 
 mapping only a subset of pixels of a display of the HMD to a separate image sensor in an image sensor array, based on the iris positions determined for the user; and 
 
 forming live video by the HMD via the image sensor array, by:
 capturing image data using the image sensors in the image sensor array, and 
 coloring the pixels of the display based on the image data and the mapping, including:
 using each portion of the image data captured by each image sensor to color the pixel to which the image sensor is mapped, and 
 coloring each remaining pixel not mapped to a separate image sensor in the image sensor array based on a color of one or more adjacent pixels to the remaining pixel. 
 
 
 
     
     
       5. A non-transitory computer-readable media storing computer instructions for a head-mounted display (HMD) that, when executed by one or more processors, cause the one or more processors to perform the steps of:
 calibrating the HMD for a user wearing the HMD, by:
 determining iris positions of the user, and 
 mapping every pixel of a display of the HMD to a separate image sensor in an image sensor array, based on the iris positions determined for the user; and 
 
 forming live video by the HMD via the image sensor array, by:
 capturing image data using the image sensors in the image sensor array, and 
 coloring the pixels of the display based on the image data and the mapping.