Augmented Reality Assisted Communication

Abstract

The disclosed embodiments illustrate methods and systems for training users in sports using mixed reality. The method includes retrieving data from athletes wearing helmets, wearable glasses, and/or motion-capture suits in real time. The helmets and the wearable glasses are integrated with mixed-reality technology. Further, physical performance data of the athletes is captured using a variety of time-synchronized measurement techniques. Thereafter, the athletes are trained using the captured data and audio, visual and haptic feedback.

We claim as our invention the following: 
     
         1 . A headwear configured to be worn by a first user, comprising:
 a frame;   one or more lenses;   one or more microphones;   an augmented reality (“AR”) display;   one or more sensors to track objects in the first user's environment, wherein the sensor is selected from the group consisting of cameras, LiDAR sensors, depth sensors, infrared sensors, and any combination thereof;   an augmented reality (“AR”) interface configured to visually present identifiers indicating an identity, position or other information about one or more objects;   a gaze tracking imaging element configured to monitor a gaze direction of the first user;   one or more processors; and   a memory storing program instructions that when executed by the one or more processors cause the one or more processors to at least
 determine an eye profile of the first user for gaze tracking, wherein, the eye profile includes at least one of the information regarding a position, size, range of movement of the first user eye with respect to the gaze tracking imaging element, a separation between each eye of the first user, a pupil shape of each eye of the first user, 
   determine the focal depth of the eye view of the user by tracking at least one of the direction, vergence or dilation, in order to track where and how far the user is looking,   dynamically rendering the images displayed in real-time so that near field, mid field and far field images are properly focused or unfocused to simulate their correct depth with respect to the user,   performing at least one action, based on the gaze direction, wherein the action is at least one of aim, focus, dynamically increasing or decreasing the resolution, rendering quality, compression rate, data size, and clipping region for portions of the image based on their viewability and focal relevance to the user,   receiving, at an augmented reality (“AR”) interface of the headwear, worn by the first user, wherein, the selection can be one or more of a voice control, touch control, or based on determining a gaze direction of the first user,   establishing an audio connection between the first headwear and a second headwear worn by the second user; sending, via the audio connection, a first audio data from the first headwear to the second headwear for output by the second headwear,   receiving at the first headwear and via the audio connection, a second audio data sent from the second headwear; indicating on the AR interface of the first headwear that second audio data is received from the second headwear and is of the second user speaking,   receiving, at the first headwear, visual information, and   outputting at the first headwear the second audio data.   
     
     
         2 . The headwear according to  claim 1  wherein the lenses are detachably mounted on the frame. 
     
     
         3 . A headwear configured to be worn by a first user, comprising:
 a frame;   one or more lenses;   one or more microphones;   an augmented reality (“AR”) display;   one or more sensors to track objects in the first user's environment, wherein the sensor is selected from the group consisting of cameras, LiDAR sensors, depth sensors, infrared sensors, and any combination thereof;   an augmented reality (“AR”) interface configured to visually present identifiers indicating an identity, position or other information about one or more objects; the one or more sensors, to capture and analyze unique biometric features of the user's eye;   a gaze tracking imaging element configured to monitor a gaze direction of the first user;   one or more processors; and   a memory storing program instructions that when executed by the one or more processors cause the one or more processors to at least   determine an eye profile of the first user for gaze tracking, wherein, the eye profile includes at least one of the information regarding a position, size, range of movement of the first user eye with respect to the gaze tracking imaging element, a separation between each eye of the first user, a pupil shape of each eye of the first user;   determine the focal depth of the eye view of the user by tracking at least one of the direction, vergence or dilation, in order to track where and how far the user is looking,   dynamically rendering the images displayed in real-time so that near field, mid field and far field images are properly focused or unfocused to simulate their correct depth with respect to the user,   performing at least one action, based on the gaze direction, wherein the action is at least one of aim, focus, dynamically increasing or decreasing the resolution, rendering quality, compression rate, data size, and clipping region for portions of the image based on their viewability and focal relevance to the user;   receiving, at an augmented reality (“AR”) interface of the headwear, worn by the first user, wherein, the selection can be one or more of a voice control, touch control, or based on determining a gaze direction of the first user,   establishing an audio connection between the first headwear and a second headwear worn by the second user; sending, via the audio connection, a first audio data from the first headwear to the second headwear for output by the second headwear,   receiving at the first headwear and via the audio connection, a second audio data sent from the second headwear; indicating on the AR interface of the first headwear that second audio data is received from the second headwear and is of the second user speaking,   receiving, at the first headwear, visual information and   outputting at the first headwear the second audio data.   
     
     
         4 . The headwear according to  claim 3  wherein the lenses are detachably mounted on the frame. 
     
     
         5 . A headwear configured to be worn by a first user, comprising:
 a frame;   one or more lenses;   an augmented reality (“AR”) display;   one or more sensors to track objects in the first user's environment, wherein the sensor is selected from the group consisting of cameras, LiDAR sensors, depth sensors, infrared sensors, and any combination thereof;   an augmented reality (“AR”) interface configured to visually present identifiers indicating an identity, position or other information about one or more objects;   one or more processors; and   a memory storing program instructions that when executed by the one or more processors cause the one or more processors to at least
 establishing an audio connection between the first headwear and a second headwear worn by the second user; sending, via the audio connection, a first audio data from the first headwear to the second headwear for output by the second headwear, 
 receiving at the first headwear and via the audio connection, a second audio data sent from the second headwear; indicating on the AR interface of the first headwear that second audio data is received from the second headwear and is of the second user speaking, 
 receiving, at the first headwear, visual information, 
 outputting at the first headwear the second audio data, wherein outputting the second audio data such has a virtual sound in a direction of the second user with respect to the first user. 
   
     
     
         6 . The headwear according to  claim 5  wherein the lenses are detachably mounted on the frame.