Augmented reality lighting effects

Abstract

The present invention embraces a system, device, and method for adding lighting effects to augmented reality (AR) content (i.e., virtual objects). Light sensors in an augmented reality (AR) system monitor an environment's lighting conditions to acquire lighting data that can be used to create (or update) virtual light sources. Depth sensors in the AR system sense the environment to acquire mapping data that can be used to create a 3D model of the environment while tracking the system's location within the environment. Algorithms running on a processor may then add the virtual light sources to the 3D model of the environment so that, when AR content is created, lighting effects corresponding to the virtual light sources can be added. The resulting AR content with virtual lighting effects appear more realistic to a user.

The invention claimed is: 
     
       1. An augmented reality (AR) system, comprising:
 a display for displaying AR content to a user, wherein the AR content is configured to be overlaid on a perspective view of an environment; 
 a sensor for gathering mapping data for constructing a three-dimensional (3D) model of the environment; and 
 a computing device communicatively coupled to the display and the sensor, the computing device comprising a processor that is configured to:
 construct the 3D model of the environment using the mapping data; and 
 create the AR content corresponding to the environment, and transmit the AR content to the display, wherein the AR content comprises a graphical 3D object that indicates a location of a physical object in the environment and a textual object, wherein the textual object indicates item identification information of an item included in the physical object, wherein the item identification comprises a name of the item included in the physical object, and a count of the items included in the physical object. 
 
 
     
     
       2. The AR system according to  claim 1 , wherein the AR content includes a lighting effect corresponds to an environment's ambient light. 
     
     
       3. The AR system according to  claim 2 , wherein the lighting effect corresponds to (i) a characteristic of a virtual light source added to the 3D model of the environment, wherein the virtual light source corresponds to a light source in the environment and (ii) a position and an orientation of the virtual light source with respect to the AR content. 
     
     
       4. The AR system according to  claim 3 , wherein the characteristic of the virtual light source comprises a direction of radiation. 
     
     
       5. The AR system according to  claim 3 , wherein the characteristic of the virtual light source comprises a light color. 
     
     
       6. The AR system according to  claim 3 , wherein the characteristic of the virtual light source comprises a light intensity. 
     
     
       7. The AR system according to  claim 3  further comprising a light sensor that comprises a charge-coupled device (CCD), wherein the CCD comprises at least one pixel, wherein the light sensor facilitates detection of the light source. 
     
     
       8. The AR system according to  claim 7 , wherein the processor is further configured to detect the light source, and wherein the detection of the light source comprises comparing a pixel value for the at least one pixel to a threshold level. 
     
     
       9. The AR system according to  claim 3 , wherein the virtual light source added to the 3D model of the environment is facilitated by simultaneous location and mapping (SLAM). 
     
     
       10. The AR system according to  claim 1 , wherein the sensor comprises a depth sensor, and wherein the depth sensor is an optical 3D scanner. 
     
     
       11. The AR system according to  claim 1 , wherein the display comprises a transparent plate that is (i) positioned in front of an eye or eyes of the user, allowing the user to view the environment through the transparent plate, and is arranged to display the AR content to the user such that the AR content appears superimposed on to the perspective view of the environment. 
     
     
       12. The AR system according to  claim 1 , wherein the display comprises a liquid crystal display (LCD). 
     
     
       13. A method comprising:
 receiving mapping data from a sensor for constructing a three-dimensional (3D) model of an environment; and 
 creating augmented reality (AR) content that comprises a graphical 3D object that appears to be proximate to a physical object in the environment, and a textual object, wherein the textual object comprise an item identification information of an item included in the physical object, wherein the item identification information comprises a name of the item included in the physical object, and a count of the items included in the physical object. 
 
     
     
       14. The method according to  claim 13 , further comprising providing an augment reality (AR) system that comprises a head mounted display (HMD). 
     
     
       15. The method according to  claim 13 , wherein the sensor comprises at least one light sensor, and wherein the at least one light sensor is a charge-coupled device (CCD) camera. 
     
     
       16. The method according to  claim 13 , wherein a lighting effect included in the AR content comprises coloring, shading, and/or lightening at least a portion of the AR content. 
     
     
       17. An augmented reality (AR) device, comprising:
 a sensor to gather mapping data for constructing a three-dimensional (3D) model of an environment; and 
 a processor and a memory, the memory having computer-coded instructions therein, wherein the computer-coded instructions are configured to, in execution with the processor, cause the processor to:
 construct the 3D model of the environment using the mapping data; and 
 create AR content corresponding to the environment, wherein the AR content comprises a graphical 3D object that appears to be proximate to a physical object in the environment and a textual object, wherein the textual object comprises an item identification information of an item included in the physical object, wherein the item identification information comprises a name of the item included in the physical object, and a count of the items included in the physical object. 
 
 
     
     
       18. The AR device according to  claim 17 , wherein the processor is further configured to construct the 3D model of the environment including a virtual light source, wherein the virtual light source comprises a diffuse light source, a point light source, a directional light source, a distributed light source, and/or an ambient light source. 
     
     
       19. The AR device according to  claim 18 , wherein the processor is further configured to add a lighting effect to the AR content that corresponds to a physical lighting effect on the physical object, and wherein the addition of the lighting effect to the AR content comprises changing at least one of a color, an intensity, or a shading of the AR content to appear illuminated by the virtual light source. 
     
     
       20. The AR device according to  claim 18 , wherein the virtual light source added to the 3D model of the environment is facilitated by simultaneous location and mapping (SLAM).