Audio-haptic cursor for assisting with virtual or real-world object selection in extended-reality (xr) environments, and systems and methods of use thereof

Abstract

A method for providing audio and haptic feedback to guide object selection while using an extended-reality device is disclosed. The method includes while a user is wearing an extended-reality headset that is associated with an output device, in accordance with a determination that a focus selector for the artificial-reality headset is directed to a first object with first visual characteristics, providing first haptic feedback and first audio feedback corresponding to the first visual characteristics via the output device. The method further includes in accordance with a determination that the focus selector for the artificial-reality headset is directed to a second object with second visual characteristics, distinct from the first visual characteristics, providing second haptic feedback and second audio feedback, distinct from the first haptic feedback and the first audio feedback, corresponding to the second visual characteristics via the output device.

What is claimed is: 
     
         1 . A non-transitory computer-readable storage medium including instructions that, when executed by a computing device, cause the computing device to:
 while a user is wearing an extended-reality headset that is associated with an output device:
 at a first point in time, in accordance with a determination that a focus selector for the extended-reality headset is directed to a first object with first visual characteristics, provide first haptic feedback and first audio feedback corresponding to the first visual characteristics via the output device; and 
 at a second point in time that is after the first point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a second object with second visual characteristics, distinct from the first visual characteristics, provide second haptic feedback and second audio feedback, distinct from the first haptic feedback and the first audio feedback, respectively, corresponding to the second visual characteristics via the output device. 
   
     
     
         2 . The non-transitory computer-readable storage medium of  claim 1 , further comprising instructions for causing the computer device to:
 at a third point in time that is after the first point in time and after the second point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a third object with third visual characteristics, distinct from the first and second visual characteristics, provide third haptic feedback and third audio feedback, the third haptic feedback being distinct from the first and second haptic feedback and the third audio feedback being distinct from the first and second audio feedback, corresponding to the third visual characteristics via the output device.   
     
     
         3 . The non-transitory computer-readable storage medium of  claim 1 , wherein:
 the first and second visual characteristics include object material, object position, object color lightness, and object size,   each of the first and second audio feedback is provided using respective values for includes audio timbre, audio direction, and audio pitch, and   each of the first and second haptic feedback is provided using respective values for haptic intensity,   wherein respective values for audio timbre are selected based on object material, respective values for audio direction are selected based on object position, respective values for audio pitch are selected based on object color lightness, and respective values for haptic intensity are selected based on object size.   
     
     
         4 . The non-transitory computer-readable storage medium of  claim 1 , wherein the focus selector is a gaze-based cursor. 
     
     
         5 . The non-transitory computer-readable storage medium of  claim 1 , wherein the output device is one of: a wrist-wearable device, a head-wearable device, or a wearable glove. 
     
     
         6 . The non-transitory computer-readable storage medium of  claim 1 , wherein the extended-reality headset does not include a display. 
     
     
         7 . The non-transitory computer-readable storage medium of  claim 1 , wherein the first haptic feedback and the first audio feedback are provided by different devices. 
     
     
         8 . A method, comprising:
 while a user is wearing an extended-reality headset that is associated with an output device:
 at a first point in time, in accordance with a determination that a focus selector for the extended-reality headset is directed to a first object with first visual characteristics, providing first haptic feedback and first audio feedback corresponding to the first visual characteristics via the output device; and 
 at a second point in time that is after the first point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a second object with second visual characteristics, distinct from the first visual characteristics, providing second haptic feedback and second audio feedback, distinct from the first haptic feedback and the first audio feedback, respectively, corresponding to the second visual characteristics via the output device. 
   
     
     
         9 . The method of  claim 8 , further comprising:
 at a third point in time that is after the first point in time and after the second point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a third object with third visual characteristics, distinct from the first and second visual characteristics, providing third haptic feedback and third audio feedback, the third haptic feedback being distinct from the first and second haptic feedback and the third audio feedback being distinct from the first and second audio feedback, corresponding to the third visual characteristics via the output device.   
     
     
         10 . The method of  claim 8 , wherein:
 the first and second visual characteristics include object material, object position, object color lightness, and object size,   each of the first and second audio feedback is provided using respective values for includes audio timbre, audio direction, and audio pitch, and   each of the first and second haptic feedback is provided using respective values for haptic intensity,
 wherein respective values for audio timbre are selected based on object material, respective values for audio direction are selected based on object position, respective values for audio pitch are selected based on object color lightness, and respective values for haptic intensity are selected based on object size. 
   
     
     
         11 . The method of  claim 8 , wherein the focus selector is a gaze-based cursor. 
     
     
         12 . The method of  claim 8 , wherein the output device is one of: a wrist-wearable device, a head-wearable device, or a wearable glove. 
     
     
         13 . The method of  claim 8 , wherein the extended-reality headset does not include a display. 
     
     
         14 . The method of  claim 8 , wherein the first haptic feedback and the first audio feedback are provided by different devices. 
     
     
         15 . A system that includes an extended-reality headset that is associated with an output device, and the extended-reality headset is configured to perform operations including:
 while a user is wearing an extended-reality headset that is associated with an output device:
 at a first point in time, in accordance with a determination that a focus selector for the extended-reality headset is directed to a first object with first visual characteristics, providing first haptic feedback and first audio feedback corresponding to the first visual characteristics via the output device; and 
 at a second point in time that is after the first point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a second object with second visual characteristics, distinct from the first visual characteristics, providing second haptic feedback and second audio feedback, distinct from the first haptic feedback and the first audio feedback, respectively, corresponding to the second visual characteristics via the output device. 
   
     
     
         16 . The system of  claim 15 , further comprising:
 at a third point in time that is after the first point in time and after the second point in time, in accordance with a determination that the focus selector for the extended-reality headset is directed to a third object with third visual characteristics, distinct from the first and second visual characteristics, providing third haptic feedback and third audio feedback, the third haptic feedback being distinct from the first and second haptic feedback and the third audio feedback being distinct from the first and second audio feedback, corresponding to the third visual characteristics via the output device.   
     
     
         17 . The system of  claim 15 , wherein:
 the first and second visual characteristics include object material, object position, object color lightness, and object size,   each of the first and second audio feedback is provided using respective values for includes audio timbre, audio direction, and audio pitch, and   each of the first and second haptic feedback is provided using respective values for haptic intensity,
 wherein respective values for audio timbre are selected based on object material, respective values for audio direction are selected based on object position, respective values for audio pitch are selected based on object color lightness, and respective values for haptic intensity are selected based on object size. 
   
     
     
         18 . The system of  claim 15 , wherein the focus selector is a gaze-based cursor. 
     
     
         19 . The system of  claim 15 , wherein the output device is one of: a wrist-wearable device, a head-wearable device, or a wearable glove. 
     
     
         20 . The system of  claim 15 , wherein the extended-reality headset does not include a display.