Optimal human-machine conversations using emotion-enhanced natural speech using hierarchical neural networks and reinforcement learning

Abstract

A system and method for emotion-enhanced natural speech using dilated convolutional neural networks, wherein an audio processing server receives a raw audio waveform from a dilated convolutional artificial neural network, associates text-based emotion content markers with portions of the raw audio waveform to produce an emotion-enhanced audio waveform, and provides the emotion-enhanced audio waveform to the dilated convolutional artificial neural network for use as a new input data set.

What is claimed is: 
     
         1 . A system for emotion-enhanced natural speech audio generation using dilated convolutional neural networks, comprising:
 an automated emotion engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a network-connected computing device and configured to provide a plurality of input data to, and receive a plurality of output data from, a dilated convolutional artificial neural network;   wherein the automated emotion engine is configured to receive at least a raw audio waveform from the dilated convolutional artificial neural network; and   wherein the automated emotion engine is configured to recognize a plurality of emotional states within the raw audio waveform.   
     
     
         2 . The system of  claim 1 , wherein the automated emotion engine is configured to produce an emotion-enhanced audio waveform by associating a plurality of emotion content markers, each comprising at least a text label describing an emotional state, with at least a portion of the audio waveform. 
     
     
         3 . The system of  claim 2 , wherein the automated emotion engine is configured to provide the emotion-enhanced audio waveform to the dilated convolutional artificial neural network as an input data set. 
     
     
         4 . The system of  claim 1 , wherein at least a portion of the emotion content markers are based on a text-to-speech script that was used in the generation of the raw audio waveform. 
     
     
         5 . A method for emotion-enhanced natural speech audio generation using dilated convolutional neural networks, comprising the steps of:
 receiving, at an automated emotion engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a network-connected computing device and configured to provide a plurality of input data to, and receive a plurality of output data from, a dilated convolutional artificial neural network, at least a raw audio waveform from the dilated convolutional artificial neural network;   associating a plurality of text-based emotion content markers with at least a portion of the audio waveform, producing an emotion-enhanced audio waveform; and   optionally providing the emotion-enhanced audio waveform to the dilated convolutional artificial neural network as an input data set for training.