Mixed-reality golf tracking and simulation

Abstract

A mixed-reality golf simulation system includes a ball-tracking sub-system to generate ball-tracking data when a golf ball is hit by a user, a near-eye display, a storage device to store panoramic images of a hole of a golf course associated with location coordinates of a plurality of locations along the hole, and a controller. The controller may direct the near-eye display to display a mixed-reality environment including a virtual reality scene from the perspective of a first location on the hole over based on images associated with location coordinates of the first location, receive ball-tracking data of a ball hit by the user in real-time, and direct the near-eye display to display the mixed-reality environment including a virtual reality scene from the perspective of a landing location of the ball based on images device associated with location coordinates of the landing location of the ball.

What is claimed: 
     
       1. A mixed-reality golf simulation system comprising:
 a ball-tracking sub-system configured to generate ball-tracking data when a golf ball is hit by a user; 
 a near-eye display with a partially-transparent display element configured to display mixed-reality virtual objects displayed over physical objects within a user field of view, wherein the near-eye display includes a user input device, wherein the near-eye display includes one or more head orientation sensors to determine a head orientation of the user; 
 a storage device to store panoramic images of a hole of a golf course from a plurality of locations along the hole, wherein the panoramic images are associated with location coordinates of the plurality of locations along the hole; and 
 a controller communicatively coupled to the ball-tracking sub-system and the near-eye display, the controller including one or more processors configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display, based on the head orientation of the user received from the near-eye display, a mixed-reality environment including a virtual reality scene from the perspective of a first location on the hole over at least a portion of the user field of view when the user is looking in one or more selected directions and at least a partially unobstructed real-world view through the partially-transparent display element when the user is looking at a location from which the ball is to be hit, wherein the virtual reality scene from the perspective of the first location includes images from the storage device associated with location coordinates of the first location; 
 receive ball-tracking data of a ball hit by the user in real-time from the ball-tracking sub-system, wherein the ball-tracking data includes a first landing location of the ball associated with a first shot; 
 direct the near-eye display to display, based on the head orientation of the user received from the near-eye display, the mixed-reality environment including a virtual reality scene from the perspective of the first landing location of the ball over at least a portion of the user field of view, wherein the virtual reality scene from the perspective of the first landing location of the ball includes images from the storage device associated with location coordinates of the first landing location of the ball; and 
 receive ball-tracking data of a ball hit by the user in real-time from the ball-tracking sub-system, wherein the ball-tracking data includes a second landing location of the ball associated with a second shot. 
 
 
     
     
       2. The mixed-reality golf simulation system of  claim 1 , wherein the one or more processors are further configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display a virtual object representing a trajectory of the ball associated with the first shot; and 
 direct the near-eye display to display a virtual object representing a trajectory of the ball associated with the second shot. 
 
     
     
       3. The mixed-reality golf simulation system of  claim 1 , wherein the ball-tracking sub-system comprises:
 one or more sensors integrated within the ball, wherein the one or more sensors transmit the ball-tracking data to the controller via wireless communication. 
 
     
     
       4. The mixed-reality golf simulation system of  claim 3 , wherein the ball-tracking sub-system comprises:
 at least one of a motion sensor, an accelerometer, an orientation sensor, or a position sensor. 
 
     
     
       5. The mixed-reality golf simulation system of  claim 1 , wherein the ball-tracking sub-system generates ball-tracking data of the ball over a launch window, wherein the launch window ends when motion of the ball is hindered by a ball containment device, wherein the ball containment device comprises:
 at least one of a net or a screen. 
 
     
     
       6. The mixed-reality golf simulation system of  claim 1 , wherein the near-eye display is configured to display an unobstructed real-world view when the near-eye display determines that a gaze direction of the user is directed at the ball prior to a shot, wherein the near-eye display is further configured to transition to a display of the mixed reality environment when the near-eye determines that a gaze direction of the user is directed above a selected angle. 
     
     
       7. The mixed-reality golf simulation system of  claim 6 , wherein the near-eye display is configured to fade transparency values of the one or more virtual objects of the mixed reality environment based on the gaze direction of the user. 
     
     
       8. The mixed-reality golf simulation system of  claim 1 , wherein the one or more processors are further configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display a virtual object representing at least one of a launch velocity, a launch angle, a hook angle, a rotation axis, a rotation rate, a hang time, a carry distance, a roll distance, or a total distance. 
 
     
     
       9. The mixed-reality golf simulation system of  claim 1 , wherein the virtual reality scene comprises:
 an avatar associated with a remote user. 
 
     
     
       10. The mixed-reality golf simulation system of  claim 1 , wherein the one or more processors are further configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to provide coaching feedback to the user based on the predicted trajectory. 
 
     
     
       11. The mixed-reality golf simulation system of  claim 1 , wherein the coaching feedback comprises:
 at least one of audio information or virtual display objects. 
 
     
     
       12. The mixed-reality golf simulation system of  claim 1 , further comprising:
 a user-tracking sub-system including one or more sensors to generate user-motion data indicative of motion of the user during a shot. 
 
     
     
       13. The mixed-reality golf simulation system of  claim 12 , wherein the one or more sensors comprise:
 at least one of a camera or one or more wearable sensors. 
 
     
     
       14. The mixed-reality golf simulation system of  claim 12 , wherein the one or more processors are further configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display a virtual object including an avatar representing the user after a shot, wherein the avatar simulates motion of the user during the shot based on the user-motion data. 
 
     
     
       15. The mixed-reality golf simulation system of  claim 1 , wherein the ball-tracking sub-system is located external to the ball. 
     
     
       16. The mixed-reality golf simulation system of  claim 15 , wherein the ball-tracking sub-system comprises:
 at least one of a camera, a doppler-tracking device, or a LIDAR-tracking device. 
 
     
     
       17. A mixed-reality golf simulation system comprising:
 a ball-tracking sub-system configured to generate ball-tracking data for golf balls hit by a user; 
 a near-eye display with a partially-transparent display element configured to display mixed-reality virtual objects displayed over physical objects within a user field of view; and 
 a controller communicatively coupled to the ball-tracking sub-system and the near-eye display, the controller including one or more processors configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display, based on a head orientation of the user received from the near-eye display, a mixed-reality environment including virtual objects within at least a portion of the user field of view when the user is looking in one or more selected directions and at least a partially unobstructed real-world view through the partially-transparent display element when the user is looking at a location from which the ball is to be hit; 
 receive ball-tracking data for a plurality of balls hit by the user with a first club type of two or more club types; 
 receive ball-tracking data for a plurality of balls hit by the user with at least a second club type of the two or more club types; 
 train a club selector to select a club for the user based on the ball-tracking data associated with the first club type and the ball-tracking data associated with the at least a second club type; 
 select a club for the user from any of the two or more club types based on a distance to a pin with the club selector. 
 
 
     
     
       18. The mixed-reality golf simulation system of  claim 17 , wherein select, with the club selector, clubs for the user for one or more subsequent shots based on the distance to the pin comprises:
 train the club selector using a machine learning technique. 
 
     
     
       19. The mixed-reality golf simulation system of  claim 17 , wherein the one or more processors are further configured to execute program instructions causing the one or more processors to:
 receive environmental conditions associated with at least a portion of the ball-tracking data; and 
 train the club selector based on the environmental conditions. 
 
     
     
       20. The mixed-reality golf simulation system of  claim 19 , wherein receive environmental conditions associated with at least a portion of the ball-tracking data comprises:
 receive environmental conditions associated with at least a portion of the ball-tracking data from the user via a user input device. 
 
     
     
       21. The mixed-reality golf simulation system of  claim 19 , wherein receive environmental conditions associated with at least a portion of the ball-tracking data comprises:
 receive environmental conditions associated with at least a portion of the ball-tracking data from one or more environmental sensors. 
 
     
     
       22. The mixed-reality golf simulation system of  claim 17 , wherein the mixed-reality environment includes at least one of at least one of a fairway, a green, or a pin. 
     
     
       23. The mixed-reality golf simulation system of  claim 17 , wherein the mixed-reality environment includes a virtual object displayed over a portion of the field of view, wherein at least a portion of the field of view is unobstructed to provide visualization of physical objects through the near-eye display, wherein a displayed perspective of the virtual object is continually updated based on a location of the user within the mixed reality environment. 
     
     
       24. The mixed-reality golf simulation system of  claim 17 , wherein the mixed-reality environment includes a virtual reality scene obstructing at least a portion of the field of view from a vantage point of the user at a selected location within the virtual reality scene. 
     
     
       25. A mixed-reality golf simulation system comprising:
 a ball-tracking sub-system configured to generate ball-tracking data when a golf ball is hit by a user; 
 a user-tracking sub-system configured to generate user-tracking data when the golf ball is hit by the user; 
 a near-eye display with a partially-transparent display element configured to display mixed-reality virtual objects displayed over physical objects within a user field of view, wherein the near-eye display includes a user input device, wherein the near-eye display includes one or more head orientation sensors to determine a head orientation of the user; and 
 a controller communicatively coupled to the ball-tracking sub-system and the near-eye display, the controller including one or more processors configured to execute program instructions causing the one or more processors to:
 direct the near-eye display to display, based on the head orientation of the user received from the near-eye display, a mixed-reality environment including virtual objects within at least a portion of the user field of view when the user is looking in one or more selected directions and at least a partially unobstructed real-world view through the partially-transparent display element when the user is looking at a location from which the ball is to be hit; 
 receive ball-tracking data of a ball hit by the user in real-time from the ball-tracking sub-system; 
 receive user-tracking data of the user as the ball is hit by the user in real-time from the user-tracking sub-system, wherein the user-tracking data is indicative of a motion of the user as the ball is hit by the user; and 
 direct the near-eye display to display virtual objects in the mixed-reality display environment including a virtual avatar simulating the motion of the user as the ball is hit by the user, wherein the virtual avatar is a three-dimensional virtual object viewable from multiple angles by the user in the mixed-reality environment.