Continuous speech recognition method and device

Abstract

In a system for recognizing a continuous speech pattern, based on a speech pattern entered through a microphone, a feature vector α i is extracted by a feature extraction unit, and a feature vector β j n is read out from a reference pattern memory. A first recursive operation unit (DPM 1 ) computes a set of similarity measures S n (i, j) between the feature vectors. A maximum similarity measure at a time point i is determined and produced by a first decision unit (DSC 1 ) and is stored in a maximum similarity memory (MAX). A second recursive operation unit (DMP 2 ) computes a reversed similarity measure. Based on a computed result g(v, l) and the output from said maximum similarity measure memory, a second decision unit determines a boundary v max . A word W u based on data u=v max -1 obtained by the boundary v is stored as n x (x=1, . . . , Y) in an order reversing unit (REV). The order reversing unit finally reverses the order of data and produces an output n x (x=1, . . . , Y).

What we claim is: 
     
       1. A machine method for recognizing continuous speech comprising the steps of: converting input speech to an input pattern A=(α1, α2, . . . , αi, . . . , αI) as a time sequence of a feature vector αi representing a feature of said input speech at a time point i (1≦i≦I); and   recognizing a series of words most similar to said input speech by the use of dynamic programming between said input pattern A and a plurality of reference pattern B n  preset as B n  =(β 1   n , β 2   n , . . . , β j   n , . . . , β jn   n ) with respect to a word number n(1≦n≦N), said recognizing step obtaining maximum similarity measures Di and words Wi sequentially for i=1 through i=I by repeatedly maximizing a sum Dp+S(A(p+1, q), B n ) from q=1 through q=I with respect to a boundary P(1≦P≦I) by the use of a dynamic programming for each word number n, said sum including a maximum similarity measure Dp between a partial pattern A(1, P)=(α1, α2, . . . , αp having as an end point a time point i=p of said input pattern A and an optimum combination Bp of said reference patterns B n  between the time points i=1 and i=p, said maximum similarity measure Dp being defined as Dp=0 for P=0 and Dp=S(A(1, P), Bp) for P=1 through P=I, and a maximum value S(A(p+1, q), B n ) of a sum of similarity measures s(αi, β j   n ) between said vectors αi and β j   n , said similarity measures s(αi, β j   n ) being defined between the time axis i and a function j(i) which most optimally correlates the time axis i of a partial pattern A(p+1, q)=(αp+1, ap+2, . . . , αi, . . . , α) of said input pattern A having a starting point i=P+1 and the end point i=q to the time axis j of said reference patterns B n , and to obtain a maximum similarity measure Dq defined as ##EQU47## which gives the maximum value of the maximized sum ##EQU48## with respect to the word number n and a word Wq which corresponds to the word number n giving the maximum similarity measure Dq.   
     
     
       2. A machine method according to claim 1 wherein said recognizing step also includes subsequent steps of: repeatedly obtaining a similarity measure S(A(u, v), B wu ) by giving u=Vmax-1 between a reversed partial pattern A(u, v) and a reversed reference pattern B wu  by the use of a dynamic programming, said reversed partial pattern A(u, v) being a time sequentially reversed pattern of a partial pattern A(u, v) having as a starting point u the time point i=I of said input pattern A to use said word Wi at the time point i=I as a recognized word Wu and an end point v(I≧u>v≧1) and said reversed reference pattern B wu  being a time sequentially reversed pattern of said reference pattern B wu  ; and   determining a boundary v as Vmax which maximizes a sum of said similarity measure S(A(u, V), B wu ) and said similarity measure Dv-1 at a time point i=v-1, and reverses the order of recognized words Wu which are sequentially obtained at the time point u.   
     
     
       3. A machine method according to claim 1 or 2 wherein said sum Dp+S(A(1+p, q), B n ), with respect to the boundary P for each word number n, is obtained by calculating a similarity measure s(αq, β j   n ) between vectors αq and β j   n  at the time point i=q and determining a similarity measure g n  (q, j) which gives the optimum path from the starting point i=1 to the end point i=q by the use of a predetermined gradual equation defined by said similarity measure S(αq, β j   n ) (and a plurality of path information to a point (q, j), ) said determination of the similarity measure g n  (q, j) being repeated from j=1 to j=Jn so that the similarity measure g n  (q, Jn) is used as said sum Dp+S(A(1+p, q), B n ). 
     
     
       4. A machine method according to claim 3 wherein said gradual equation g n  (q, j) is defined as ##EQU49## in which h n  (q, j) is used as the path information at the time point i=q and defined as h n  (q, j)=g n  (q-1, j-1)+2s(αq, β j   n ). 
     
     
       5. A machine method according to claim 3 wherein said gradual equation g n  (q, j) is defined as ##EQU50## in which h n  (q, j) and f n  (q, j) are used as the path information at the time point i=q and defined as h n  (q, j)=g n  (q-1, j-1)+2s(αi, β j   n ) and f n  (g, j)=h n  (q-1, j-1)+2s(αi, β j   n ). 
     
     
       6. An apparatus for recognizing a continuous speech comprising: input means for converting input speech to an input pattern A=(α1, α2, . . . , αi, . . . , αI) as a time sequence of a feature vector αi representing a feature of said input speech at a time point i (1≦i≦I);   reference means for memorizing a plurality of reference patterns preset as B n  =(β 1   n , β 2   n , . . . , β j   n , . . . , β Jn .sup. n) with respect to a word number n(1≦n≦N); and   recognition means for recognizing a series of words most similar to said input speech by the use of dynamic programming between said input pattern A and said reference patterns B n , said recognition means including:   maximum similarity measure memory means for memorizing for each q(1≦q≦I) a maximum similarity measure Dq between a partial pattern A(1, q)=(α1, α2, . . . , αq) having as starting and end points time points i=1 and i=q, respectively, of said input pattern A and an optimum combination of said reference patterns B n  between the time points i=i and i=q,   word memory means for memorizing for each q a word Wq corresponding to the last one of said optimum combination of said reference patterns B n  which gives said maximum similarity measure Dq,   pattern matching means for maximizing with respect to a boundary P, by the use of dynamic programming, a sum of a maximum similarity measure Dp and a maximum value S(A(P+1, q), B n ) of a sum of similarity measure s(αi, β j   n ) between said vectors αi and β j   n , said similarity measure s(αi, β j   n ) being defined between the time axis i and a function j(i) which most optimumly correlates the time axis i of a partial pattern A(P+1, q)=(αp+1, αp+2 , . . . , αi, . . . , αq) of said input pattern A having a starting point i=P+1(1≦P+1<q≦I) and the end point i=q to the time axis j of said reference patterns B n , and   decision means for deciding, with respect to a word number n, said maximum similarity measure Dq as ##EQU51## which gives the maximum value of said maximized sum ##EQU52## maximized by said pattern matching means and said word Wq which corresponds to the word number n giving said maximum similarity measure Dq so that said maximum similarity measure Dq and said word Wq are memorized by said maximum similarity measure memory means and said word memory means, respectively.   
     
     
       7. An apparatus according to claim 6 wherein said recognition means further includes: another decision means for repeatedly deciding, by giving u=Vmax-1, a similarity measure S(A(u, v), B wu ) between a reversed partial pattern A(u, v) and a reversed reference pattern B wu  by the use of a dynamic programming and a boundary v as Vmax which maximizes a sum of said similarity measure S(A(u, v), B wu ) and the similarity measure Dv-1 at a time point i=v-1, said reversed partial pattern A(u, v) being a time sequentially reversed pattern of a partial pattern A(u, v) having as a starting point u the time point i=I of said input pattern A to use said word Wi at the time point i=I as a recognized word Wu and an end point v(I≧u≧1) and said reversed reference pattern B wu  being a time sequentially reversed pattern of said reference pattern B wu  ; and   order reversing means for reversing the order of said words Wu decided by said second decision means.   
     
     
       8. An apparatus according to claim 6 or 7 wherein said pattern matching means includes: similarity measure calculation means for calculating a similarity measure s(αq, β j   n ) between vectors α and β j   n  at the time point i=q; and   gradual equation calculation means for calculating a similarity measure g n  (q, j) which gives the optimum path from the starting point i=1 to the end point i=q by the use of a predetermined gradual equation defined by said similarity measure s(αq, β j   n ) and a plurality of path information to a point (q, j), said calculation of the similarity measure g n  (q, j) being repeated from j=1 to j=jn so that the similarity measure g n  (q, Jn) is used as said sum Dp+S(A(1+p, q), B n ).   
     
     
       9. An apparatus according to claim 8 wherein said gradual equation calculation means includes: path information means for providing the path information h n  (q, j-1), h n  (q, j) and h n  (q-1, j) defined as h n  (i, j)=g n  (i-1, j-1)+2s(αq, β j   n ) for the time point i; and   a gradual equation execution means for executing said gradual equation g n  (q, j) defined as ##EQU53##   
     
     
       10. An apparatus according to claim 8 wherein said gradual equation calculation means includes: path information means for providing the path information h n  (q, j-1), h n  (q, j) and f n  (q-1, j) defined as h n  (i, j)=g n  (i-1, j-1)+2s(αi, β j   n ) and f n  (i, j)=h n  (i-1, j-1)+2s(αi, β j   n ) for the time point i, respectively; and   a gradual equation execution means for executing said gradual equation g n  (q, j) defined as ##EQU54##