Locating spatialized sounds nodes for echolocation using unsupervised machine learning

Abstract

Described herein is a system for generating echolocation sounds to assist a user having no sight or limited sight to navigate a three-dimensional space (e.g., physical environment, computer gaming experience, and/or virtual reality experience). Input is received from a user to generate echolocation sounds to navigate a three-dimensional space. Based at least on the received input, a digital representation of the three-dimensional space is segmented into one or more depth planes using an unsupervised machine learning algorithm. For each depth plane, object segments are determined for each object within the particular depth plane. Locations of a plurality of echo sound nodes are determined in accordance with the depth level and surface area of each object defined by the determined segments. The echolocation sounds comprising a spatialized sound from each echo sound node originating from the determined location are generated.

What is claimed is: 
     
       1. A system, comprising:
 one or more processors; and 
 a memory having computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the system to: 
 segment a digital representation of a three-dimensional space into two or more depth planes using an unsupervised machine learning algorithm; 
 determine object segments for respective objects within respective depth planes; 
 determine respective surface areas of the respective objects; 
 based at least on the respective surface areas determined for the respective objects and the respective depth planes of the respective objects, select a first plurality of discrete locations in the three-dimensional space for placement of a first plurality of echo sound nodes on a first object in a first depth plane and a second plurality of discrete locations in the three-dimensional space for placement of a second plurality of echo sound nodes on a second object in a second depth plane, wherein at least some other locations in the three-dimensional space are not selected to receive echo sound nodes; 
 place the first plurality of echo sound nodes at the first plurality of discrete locations on the first object and the second plurality of echo sound nodes at the second plurality of discrete locations on the second object; and 
 generate a plurality of spatialized echolocation sounds originating from the first plurality of echo sound nodes and the second plurality of echo sound nodes. 
 
     
     
       2. The system of  claim 1 , wherein the unsupervised machine learning algorithm comprises a clustering algorithm, with each cluster identified by the clustering algorithm comprising a different depth level. 
     
     
       3. The system of  claim 1 , the memory having further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the system to:
 capture the digital representation of the three-dimensional space. 
 
     
     
       4. The system of  claim 3 , wherein the digital representation of the three-dimensional space is captured using at least one of a digital camera, a three-dimensional camera, or a depth camera. 
     
     
       5. The system of  claim 1 , wherein the plurality of spatialized echolocation sounds are generated by at least one of a virtual reality headset, a mixed-reality headset, or an augmented reality headset. 
     
     
       6. The system of  claim 1 , wherein the plurality of spatialized echolocation sounds are generated using at least one of a channel-based audio output, a spherical sound representation, or an object-based audio output. 
     
     
       7. The system of  claim 1 , the memory having further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the system to:
 based upon a movement of a user, infer an input requesting to generate the plurality of spatialized echolocation sounds; and 
 generate the plurality of spatialized echolocation sounds responsive to inferring the input. 
 
     
     
       8. The system of  claim 1 , the memory having further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the system to:
 detect a gesture of a user; and 
 generate the plurality of spatialized echolocation sounds responsive to detecting the gesture. 
 
     
     
       9. The system of  claim 1 , wherein the three-dimensional space comprises a computer-generated gaming experience. 
     
     
       10. The system of  claim 1 , wherein the three-dimensional space comprises a physical environment. 
     
     
       11. A method, comprising:
 receiving input from a user to generate echolocation sounds to navigate a three-dimensional space; and 
 responsive to the received input:
 segmenting a digital representation of the three-dimensional space into two or more depth planes using an unsupervised machine learning algorithm; 
 determining object segments for respective objects within respective depth planes; 
 determining respective surface areas of the respective objects; 
 based at least on the respective surface areas determined for the respective objects and the respective depth planes of the respective objects, selecting a first plurality of discrete locations in the three-dimensional space for placement of a first plurality of echo sound nodes on a first object in a first depth plane and a second plurality of discrete locations in the three-dimensional space for placement of a second plurality of echo sound nodes on a second object in a second depth plane; 
 placing the first plurality of echo sound nodes at the first plurality of discrete locations on the first object and the second plurality of echo sound nodes at the second plurality of discrete locations on the second object, wherein at least some other discrete locations on the first object and the second object do not receive echo sound nodes; and 
 generating a plurality of echolocation sounds originating from the first plurality of echo sound nodes and the second plurality of echo sound nodes. 
 
 
     
     
       12. The method of  claim 11 , wherein the unsupervised machine learning algorithm comprises a k-means clustering algorithm employing an elbow method which reviews a percentage of variance as a function of a number of clusters to determine a quantity of clusters. 
     
     
       13. The method of  claim 11 , further comprising:
 capturing the digital representation of the three-dimensional space using at least one of a digital camera, a three-dimensional camera, or a depth camera. 
 
     
     
       14. The method of  claim 11 , wherein the plurality of echolocation sounds are generated by at least one of a virtual reality headset, a mixed-reality headset, or an augmented reality headset. 
     
     
       15. The method of  claim 11 , wherein the plurality of echolocation sounds are generated using at least one of a channel-based audio output, a spherical sound representation, or an object-based audio output. 
     
     
       16. The method of  claim 11 ,
 wherein the first plurality comprises a greater number of echo sound nodes than the second plurality, and 
 wherein the first depth plane of the first object is relatively closer to the user than the second depth plane of the second object. 
 
     
     
       17. The method of  claim 16 , wherein generating the plurality of echolocation sounds comprises employing different volume and delay for first sounds originating from the first plurality of echo sound nodes than for second sounds originating from the second plurality of echo sound nodes. 
     
     
       18. A computer storage media storing computer-readable instructions that, when executed, cause a computing device to:
 segment a digital representation of a three-dimensional space into two or more depth planes using an unsupervised machine learning algorithm; 
 determine object segments for respective objects within respective depth planes; 
 determine respective surface areas of the respective objects; 
 based at least on the respective surface areas determined for the respective objects and the respective depth planes of the respective objects, select a first plurality of discrete locations in the three-dimensional space for placement of a first plurality of echo sound nodes on a first object in a first depth plane and a second plurality of discrete locations in the three-dimensional space for placement of a second plurality of echo sound nodes on a second object in a second depth plane; 
 place the first plurality of echo sound nodes at the first plurality of discrete locations on the first object and the second plurality of echo sound nodes at the second plurality of discrete locations on the second object, at least some other discrete locations on the first object and the second object lacking echo sound nodes; and 
 generate a plurality of spatialized echolocation sounds originating from the first plurality of echo sound nodes and the second plurality of echo sound nodes. 
 
     
     
       19. The computer storage media of  claim 18 , wherein the unsupervised machine learning algorithm comprises a clustering algorithm, with each cluster identified by the clustering algorithm comprising a depth level. 
     
     
       20. The computer storage media of  claim 19 , wherein the plurality of spatialized echolocation sounds are generated by at least one of a virtual reality headset, a mixed-reality headset, or an augmented reality headset, and, wherein the plurality of spatialized echolocation sounds are generated using at least one of a channel-based audio output, a spherical sound representation, or an object-based audio output.