Speech recognition device and method of identifying speech

Abstract

A speech recognition device includes a speech acquiring unit, a speech outputting unit, a camera unit, and a processor. The processor obtains speech input acquired by the speech acquiring unit, obtains images acquired by the camera unit, chronologically links the obtained speech and the obtained images together, compares the obtained speech to a speech database to confirm matching speech and a confidence level of the matching speech, determines whether the confidence level of the matching speech exceeds a predetermined confidence level, and outputs the matching speech when the confidence level of the matching speech exceeds the predetermined confidence level.

What is claimed is: 
     
       1. A speech recognition device comprising:
 a speech acquiring unit configured to acquire speech input; 
 a speech outputting unit; 
 a camera unit configured to acquire images; and 
 a processor configured to:
 obtain the speech input acquired by the speech acquiring unit; 
 obtain the images acquired by the camera unit and chronologically link the obtained speech and the obtained images together; 
 compare the obtained speech to a speech database to confirm matching speech and a confidence level of the matching speech, wherein the confidence level of the speech represents an accuracy of the matching speech to the obtained speech; 
 determine whether the confidence level of the matching speech exceeds a predetermined confidence level; and 
 output the matching speech when the confidence level of the matching speech exceeds the predetermined confidence level. 
 
 
     
     
       2. The speech recognition device of  claim 1 , wherein the processor is further configured to:
 identify the images linked to the obtained speech, when the confidence level does not exceed the predetermined level, and identify hand gestures in the images; 
 match the identified hand gestures to the matching speech of the images to match the matching speech to a meaning of the hand gestures; and 
 outputting the matching speech matched to the identified hand gestures. 
 
     
     
       3. The speech recognition device of  claim 2 , wherein the processor is further configured to search a gesture database according to the identified hand gestures to confirm the meaning of the hand gestures. 
     
     
       4. The speech recognition device of  claim 1 , wherein the processor is further configured to:
 divide the obtained speech into a plurality of sentences; 
 divide the obtained images into a plurality of images; and 
 link the plurality of images to the plurality of sentences chronologically; 
 wherein the processor obtains phonological features from the obtained speech and identifies an end point of each sentence according to an end-point algorithm; and 
 wherein the processor divides the obtained speech into the plurality of sentences according to the identified end points. 
 
     
     
       5. The speech recognition device of  claim 1 , wherein the processor is further configured to:
 divide the speech into words and phrases; 
 compare the words and phrases to words and phrases of the speech database to confirm matching words and phrases of the speech; 
 confirm a confidence level of the matching words and matching phrases according to the comparison; 
 confirm the matching speech according to the matching words and matching phrases; and 
 confirm the confidence level of the matching speech according to the confidence level of the matching words and matching phrases. 
 
     
     
       6. A method of identifying speech comprising:
 obtaining speech input acquired by a speech acquiring unit of a speech acquisition device; 
 obtaining images acquired by a camera unit of the speech acquisition device and chronologically linking the obtained speech and the obtained images together; 
 comparing the obtained speech to a speech database to confirm matching speech and a confidence level of the matching speech, wherein the confidence level of the speech represents an accuracy of the matching speech to the obtained speech; 
 determining whether the confidence level of the matching speech exceeds a predetermined confidence level; and 
 outputting the matching speech when the confidence level of the matching speech exceeds the predetermined confidence level. 
 
     
     
       7. The method of  claim 6 , further comprising:
 identifying the images linked to the obtained speech, when the confidence level does not exceed the predetermined level, and identifying hand gestures in the images; 
 matching the identified hand gestures to the matching speech of the images to match the matching speech to a meaning of the hand gestures; and 
 outputting the matching speech matched to the identified hand gestures. 
 
     
     
       8. The method of  claim 7 , further comprising searching a gesture database according to the identified hand gestures to confirm the meaning of the hand gestures. 
     
     
       9. The method of  claim 6 , further comprising:
 dividing the obtained speech into a plurality of sentences; 
 dividing the obtained images into a plurality of images; and 
 linking the plurality of images to the plurality of sentences chronologically; 
 wherein the processor obtains phonological features from the obtained speech and identifies an end point of each sentence according to an end-point algorithm; and 
 wherein the processor divides the obtained speech into the plurality of sentences according to the identified end points. 
 
     
     
       10. The method of  claim 6 , further comprising:
 dividing the speech into words and phrases; 
 comparing the words and phrases to words and phrases of the speech database to confirm matching words and phrases of the speech; 
 confirming a confidence level of the matching words and matching phrases according to the comparison; 
 confirming the matching speech according to the matching words and matching phrases; and 
 confirming the confidence level of the matching speech according to the confidence level of the matching words and matching phrases. 
 
     
     
       11. A non-transitory storage medium having stored thereon instructions that, when executed by at least one processor of a speech recognition device, causes the least one processor to execute instructions of a method for identifying speech, the method comprising:
 obtaining speech input acquired by a speech acquiring unit of a speech acquisition device; 
 obtaining images acquired by a camera unit of the speech acquisition device and linking the obtained speech and the obtained images together chronologically; 
 comparing the obtained speech to a speech database to confirm matching speech and a confidence level of the matching speech, wherein the confidence level of the speech represents an accuracy of the matching speech to the obtained speech; 
 determining whether the confidence level of the matching speech exceeds a predetermined confidence level; and 
 outputting the matching speech when the confidence level of the matching speech exceeds the predetermined confidence level. 
 
     
     
       12. The non-transitory storage medium of  claim 11 , further comprising:
 identifying the images linked to the obtained speech, when the confidence level does not exceed the predetermined level, and identifying hand gestures in the images; 
 matching the identified hand gestures to the matching speech of the images to match the matching speech to a meaning of the hand gestures; and 
 outputting the matching speech matched to the identified hand gestures. 
 
     
     
       13. The non-transitory storage medium of  claim 12 , further comprising searching a gesture database according to the identified hand gestures to confirm the meaning of the hand gestures. 
     
     
       14. The non-transitory storage medium of  claim 11 , further comprising:
 dividing the obtained speech into a plurality of sentences; 
 dividing the obtained images into a plurality of images; and 
 linking the plurality of images to the plurality of sentences chronologically; 
 wherein the processor obtains phonological features from the obtained speech and identifies an end point of each sentence according to an end-point algorithm; and 
 wherein the processor divides the obtained speech into the plurality of sentences according to the identified end points. 
 
     
     
       15. The non-transitory storage medium of  claim 11 , further comprising:
 dividing the speech into words and phrases; 
 comparing the words and phrases to words and phrases of the speech database to confirm matching words and phrases of the speech; 
 confirming a confidence level of the matching words and matching phrases according to the comparison; 
 confirming the matching speech according to the matching words and matching phrases; and 
 confirming the confidence level of the matching speech according to the confidence level of the matching words and matching phrases.