Systems and methods for transposing spoken or textual input to music

Abstract

Described herein are real-time musical translation devices (RETM) and methods of use thereof. Exemplary uses of RETMs include optimizing the understanding and/or recall of an input message for a user and improving a cognitive process in a user.

1 . A method of transforming textual input to a musical score comprising:
 receiving text input;   transliterating the text input into a standardized phonemic representation of the text input;   determining for the phonemic text input, a plurality of spoken pause lengths and a plurality of spoken phoneme lengths;   mapping the plurality of spoken pause lengths to a respective plurality of sung pause lengths;   mapping the plurality of spoken phoneme lengths to a respective plurality of sung phoneme lengths;   generating, from the plurality of sung pause lengths and the plurality of sung phoneme lengths, a timed text input;   generating a plurality of matching metrics for each of a respective plurality of portions of the timed text input against a plurality of melody segments; and   generating a patterned musical message from the timed text input and the plurality of melody segments based at least in part on the plurality of matching metrics.   
     
     
         2 . The method of  claim 1 , wherein the method is performed in real-time or in near-real-time, and further comprises causing the patterned musical message to be played audibly on a transducer. 
     
     
         3 . The method of  claim 1 , wherein the patterned musical message is expected to optimize, for a user, at least one of an understanding of the input message and a recall of the input message. 
     
     
         4 . The method of  claim 1 , further comprising providing to a user a visual image relating to the patterned musical message aimed at enhancing comprehension and learning. 
     
     
         5 . The method of  claim 1 , wherein the patterned musical message is presented to a user having a cognitive impairment, a behavioral impairment, or a learning impairment. 
     
     
         6 . The method of  claim 5 , wherein the user has a comprehension disorder, including at least one of autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, aphasia, dementia, dyspraxia, dyslexia, dysphasia, apraxia, stroke, traumatic brain injury, brain surgery, surgery, schizophrenia, schizoaffective disorder, depression, bipolar disorder, post-traumatic stress disorder, Alzheimer's disease, Parkinson's disease, age-related cognitive impairment, a language comprehension impairment, an intellectual disorder, a developmental disorder, stress, anxiety, Williams syndrome, Prader Willi syndrome, Smith Magenis syndrome, Bardet Biedl syndrome, or Down's syndrome or other neurological disorder. 
     
     
         7 . The method of  claim 1 , wherein the input is a spoken message. 
     
     
         8 . The method of  claim 1 , wherein the input is a written message. 
     
     
         9 . The method of  claim 1 , wherein the user selects the input text from a plurality of texts. 
     
     
         10 . The method of  claim 1 , further comprising:
 generating a textual message relating to the input text and representing an output message to be displayed to a user;   modifying at least one character of the textual message in a manner expected to optimize at least one of the user's understanding and recall of the textual message as seen on a visual display; and   displaying the modified textual message on a display device.   
     
     
         11 . The method of  claim 10 , wherein modifying the at least one character of the textual message in the manner expected to optimize the user's understanding and/or recollection of the textual message includes at least one of removing or modifying at least one segment of the at least one character, modifying a size of the at least one character relative to other characters in the textual message, and modifying a display time of the at least one character relative to the other characters in the textual message. 
     
     
         12 . The method of  claim 1 , wherein generating the patterned musical message from the timed text input and the plurality of melody segments based at least in part on the plurality of matching metrics comprises:
 accessing pitch information and timing information about a note in a melody segment; and   setting a pitch and a timing for a phoneme in the timed text input based on the pitch information and the timing information.   
     
     
         13 . The method of  claim 1 , wherein the output device is at least one of a virtual reality device, an augmented reality headset device, and a smart speaker executing a digital personal assistant. 
     
     
         14 . The method of  claim 1 , wherein the plurality of melody segments is selected according to at least one of a heart rate of the user, an eye movement of the user, a respiration rate of the user, a muscle tone measurement of the user, a movement of the user, and a measurement of an efficacy of a treatment of the patient. 
     
     
         15 . A real time musical translation device (RETM) comprising:
 an input interface;   a processor;   an audio output component; and   a memory communicatively coupled to the processor and comprising instructions that when executed by the processor cause the processor to:
 receive text input from the input interface; 
 determine, for the text input, a plurality of spoken pause lengths and a plurality of spoken phoneme lengths; 
 map the plurality of spoken pause lengths to a respective plurality of sung pause lengths; 
 map the plurality of spoken phoneme lengths to a respective plurality of sung phoneme lengths; 
 generate, from the plurality of sung pause lengths and the plurality of sung phoneme lengths, a timed text input; 
 generate a plurality of matching metrics for each of a respective plurality of portions of the timed text input against a plurality of melody segments; 
 generate a patterned musical message from the timed text input and the plurality of melody segments based at least in part on the plurality of matching metrics; and 
 output the patterned musical message using the audio output component. 
   
     
     
         16 . The RETM of  claim 15 , further comprising a display device, wherein the processor is further configured to provide to a user a visual image on the display device, the visual image relating to the patterned musical message. 
     
     
         17 . The RETM of  claim 16 , wherein the display device is incorporated into the output device. 
     
     
         18 . The RETM of  claim 15 , wherein the processor is further configured to:
 generate a textual message relating to the input text and representing an output message to be displayed to a user;   modify at least one character of the textual message in a manner expected to optimize the user's understanding of the textual message; and   display the modified textual message on the display device.   
     
     
         19 . The RETM of  claim 18 , wherein the processor is configured to modify the at least one character of the textual message in the manner expected to optimize the user's understanding and/or recollection of the textual message by at least one of removing or modifying at least one segment of the at least one character, modifying a size of the at least one character relative to other characters in the textual message, and modifying a display time of the at least one character relative to the other characters in the textual message. 
     
     
         20 . The RETM of  claim 15 , wherein the patterned musical message is presented to a user having a cognitive impairment, a behavioral impairment, or a learning impairment. 
     
     
         21 . The RETM of  claim 20 , wherein the user has a comprehension disorder, including at least one of autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, aphasia, dementia, dyspraxia, dyslexia, dysphasia, apraxia, stroke, traumatic brain injury, brain surgery, surgery, schizophrenia, schizoaffective disorder, depression, bipolar disorder, post-traumatic stress disorder, Alzheimer's disease, Parkinson's disease, age-related cognitive impairment, a language comprehension impairment, an intellectual disorder, a developmental disorder, stress, anxiety, Williams syndrome, Prader Willi syndrome, Smith Magenis syndrome, Bardet Biedl syndrome, or Down's syndrome or other neurological disorder. 
     
     
         22 . The RETM of  claim 15 , wherein the input text is at least one of a spoken message and a written message. 
     
     
         23 . The RETM of  claim 15 , wherein the user selects the input text from a plurality of texts. 
     
     
         24 . A method of transforming textual input to a musical score for improving a cognitive process in a user comprising:
 providing the user with access to a real-time musical translation device (RETM), wherein the RETM comprises:
 an input interface; 
 a processor; 
 an audio output component; and 
 a memory communicatively coupled to the processor and comprising instructions that when executed by the processor cause the processor to:
 receive text input from the input interface; 
 determine, for the text input, a plurality of spoken pause lengths and a plurality of spoken phoneme lengths; 
 map the plurality of spoken pause lengths to a respective plurality of sung pause lengths; 
 map the plurality of spoken phoneme lengths to a respective plurality of sung phoneme lengths; 
 generate, from the plurality of sung pause lengths and the plurality of sung phoneme lengths, a timed text input; 
 generate a plurality of matching metrics for each of a respective plurality of portions of the timed text input against a plurality of melody segments; 
 generate a patterned musical message from the timed text input and the plurality of melody segments based at least in part on the plurality of matching metrics; and 
 output the patterned musical message to the user using the audio output component. 
 
   
     
     
         25 . The method of  claim 24 , wherein the patterned musical message is expected to optimize a user's understanding of the input message. 
     
     
         26 . The method of  claim 24 , further comprising providing to the user a visual image relating to the patterned musical message. 
     
     
         27 . The method of  claim 24 , wherein the patterned musical message is presented to a user having a cognitive impairment, a behavioral impairment, or a learning impairment. 
     
     
         28 . The method of  claim 27 , wherein the user has a comprehension disorder, including at least one of autism spectrum disorder, attention deficit disorder, attention deficit hyperactivity disorder, aphasia, dementia, dyspraxia, dyslexia, dysphasia, apraxia, stroke, traumatic brain injury, brain surgery, surgery, schizophrenia, schizoaffective disorder, depression, bipolar disorder, post-traumatic stress disorder, Alzheimer's disease, Parkinson's disease, age-related cognitive impairment, a language comprehension impairment, an intellectual disorder, a developmental disorder, stress, anxiety, Williams syndrome, Prader Willi syndrome, Smith Magenis syndrome, Bardet Biedl syndrome, or Down's syndrome or other neurological disorder. 
     
     
         29 . The method of  claim 28 , wherein the user has dyslexia, and the RETM is configured to present a series of predefined tests and/or tasks to the user in order to evaluate and improve comprehension. 
     
     
         30 . The method of  claim 28 , wherein the user has had a stroke, and the RETM is configured to evaluate the user's ability to respond and show improved comprehension to the patterned musical message and/or shows an improved ability to speak or otherwise communicate. 
     
     
         31 . The method of  claim 28 , wherein the user has been diagnosed with autism spectrum disorder, and the RETM is configured to evaluate the user's ability to respond to the patterned musical message. 
     
     
         32 . The method of  claim 24 , wherein the patterned message is presented to a user for at least one of enhancing comprehension, improving communication, and increasing social interaction. 
     
     
         33 . The method of  claim 24 , further comprising:
 tracking a performance of the user over successive uses of the RETM; and   determining, from the performance of the user, a measure of improvement of the user in at least one area.   
     
     
         34 . The method of  claim 24 , wherein the input text is a spoken message. 
     
     
         35 . The method of  claim 24 , wherein the input text is a written message. 
     
     
         36 . The method of  claim 24 , wherein the user selects the input text from a plurality of texts. 
     
     
         37 . The method of  claim 24 , further comprising:
 generating a textual message relating to the input text and representing an output message to be displayed to a user;   modifying at least one character of the textual message in a manner expected to optimize at least one of the user's understanding and recall of the textual message as seen on a visual display; and   displaying the modified textual message on a display device.   
     
     
         38 . The method of  claim 37 , wherein modifying the at least one character of the textual message in the manner expected to optimize the user's understanding and/or recollection of the textual message includes at least one of removing or modifying at least one segment of the at least one character, modifying a size of the at least one character relative to other characters in the textual message, and modifying a display time of the at least one character relative to the other characters in the textual message. 
     
     
         39 . The method of  claim 24 , wherein generating the patterned musical message from the timed text input and the plurality of melody segments based at least in part on the plurality of matching metrics comprises:
 accessing pitch information and timing information about a note in a melody segment; and   setting a pitch and a timing for a phoneme in the timed text input based on the pitch information and the timing information.   
     
     
         40 . The method of  claim 24 , wherein the output device is a virtual reality or augmented reality headset device. 
     
     
         41 . A method of determining a melody track in a music file comprising:
 accessing a plurality of tracks in the music file;   scoring each of the plurality of tracks according a plurality of melody heuristics; and   identifying a melody track from among the plurality of tracks based at least in part on the plurality of melody heuristics for the melody track.   
     
     
         42 . The method of  claim 41 , wherein the plurality of melody heuristics comprises at least one of a motion of the melody track, a number of notes in the melody track, a rhythmic density of the melody track, an entropy the melody track, and a pitch/height ambitus of melody track.