Dialogue Detector and Correction

Abstract

Compounds of the following formula are provided for use with kinases: wherein the variables are as defined herein. Also provided are pharmaceutical compositions, kits and articles of manufacture comprising such compounds; methods and intermediates useful for making the compounds; and methods of using said compounds.

1 . In an electronic audio portion of an electronic radio or audio video system with an audio program carried by multiple audio channel signals, an apparatus for monitoring the presence of correct audio sounds in one or more channels of the audio program, the apparatus including:
 a) an input circuit for receiving a dialogue channel of an audio signal, the dialogue channel intended to carry spoken words heard by a listener;   b) a phoneme detection circuit to provide dialogue data in response to the repeated presence in the dialogue channel of known ones of a plurality of spoken phonemes corresponding to spoken vowels and/or consonants created by the corresponding particular shape and/or movement of a speaker's lips;   c) a channel phoneme logic circuit responsive to the dialogue data of b) and to parameters including the number of spoken phonemes found to be present in the dialogue channel in a known time period and operating to provide a report and/or alarm in response thereto.   
     
     
         2 . The apparatus as claimed in  claim 1  wherein the known spoken phonemes of b) are a plurality of particular spoken phonemes chosen from the set of A, E, O, M, P, B and S and the dialogue data indicates which of the spoken phoneme is present. 
     
     
         3 . The apparatus as claimed in  claim 1  wherein the known spoken phonemes of b) are a plurality of particular spoken phonemes chosen from the set of A, E and O and the dialogue data indicates which of the spoken phoneme is present. 
     
     
         4 . The apparatus as claimed in  claim 1  wherein the parameters of c) are used to determine if the known spoken phonemes which are found in b) are as expected for the particular dialogue channel of a) which is received. 
     
     
         5 . The apparatus as claimed in  claim 1  wherein the dialogue channel is from an audio only system. 
     
     
         6 . The apparatus as claimed in  claim 1  wherein the parameters of c) are automatically adjusted in response to the audio signal of a). 
     
     
         7 . The apparatus as claimed in  claim 1  wherein the audio signal of a) is part of a system which includes displayable moving images and corresponding sounds and wherein the spoken words of the dialogue channel correspond to displayed moving images of a speaking person. 
     
     
         8 . The apparatus as claimed in  claim 1  wherein the audio signal of a) is part of a system which includes displayable moving images and corresponding sounds and further includes an input circuit receiving displayable moving images and wherein the known spoken phonemes of the dialogue channel correspond to displayed moving images of a speaking person and in b) the dialogue data is further responsive to the shape and/or movement of the speaker's lips shown in the moving images corresponding to one or more of the known spoken phonemes. 
     
     
         9 . The apparatus as claimed in  claim 7  further including an input circuit for receiving displayable moving images and a viseme detection and location circuit and a phoneme viseme logic circuit which operate to verify dialogue is carried in the proper audio dialogue channel thereby leading the viewer of the speaking person carried in the displayable moving images to perceive sounds as properly corresponding to the location of the displayable speaking person. 
     
     
         10 . The apparatus as claimed in  claim 7  further including an input circuit for receiving displayable moving images, the apparatus further responsive to the presence in the system of particular images of the displayable moving images and corresponding sounds which have a high probability of occurring together in time, and which presence of particular images and corresponding sounds are used to verify the corresponding sound is being carried in the proper channel of the audio signal of a) thereby leading the viewer of the displayable moving images to perceive sounds as properly corresponding to the location of the displayed moving images. 
     
     
         11 . In an electronic audio video system such as a television system with a television program which is intended for display, the television program including visually displayable motion images which include talking human speakers and audibly displayable sounds including the words spoken by the speaker and other sounds corresponding to the motion images and which sounds are carried by multiple audio channel signals, an apparatus for monitoring the placement of correct audio sounds in one or more audio channel signals, the apparatus including:
 a) an input circuit for receiving a plurality of audio channels carrying sound corresponding to motion images including left, center and right audio channels intended to carry words spoken by a speaker carried in displayable motion images with the spoken words perceived by the viewer as coming from a direction of left, center or right, from the perspective of the viewer as the viewer watches the displayed speaker;   b) a phoneme detection circuit responsive to a plurality of the left, center and right audio channels and operating to determine the presence of particular phonemes pertaining to spoken vowels and/or consonants therein created by a corresponding particular shape and/or movement of lips of a speaker carried in the displayable motion images and in response providing dialogue data indicative of the presence of the particular spoken vowels and consonants; and   c) a channel phoneme logic circuit responsive to the dialogue data of b) and to parameters including the number of the phonemes present in at least one of the left, center and right audio channels in a time period to determine if dialogue sound is being carried in one or more of the left, center and right channels and provide a report and/or alarm in response thereto.   
     
     
         12 . The apparatus of  claim 11  further including:
 d) an input circuit for receiving displayable motion images and a viseme detection circuit responsive to the displayable motion images to provide viseme activity data related to the presence therein of visemes formed by the lips of speakers in the displayable motion images, with the viseme activity data further indicating the location of the visemes as being in the left, center or right direction, relative to a viewer, of the displayable image; and 
 e) wherein in c) a tracking report is generated, and a tracking alarm may be generated, both in response to:
 i) known parameters; 
 ii) the number of phonemes present in each or all the left, center and/or right audio channels; and 
 iii) the number of visemes present in each or all of the left, center and/or right directions respectively. 
 
 
     
     
         13 . The apparatus of  claim 12  wherein in response to e) the sound signals in a plurality of audio channels of a) are swapped to correct a tracking error. 
     
     
         14 . The apparatus of  claim 12  wherein the parameters of i) include a known period of time wherein in ii) the number of phonemes present in the left, center or right audio channels which occur during the time period and in iii) the number of visemes present in the left, center or right directions which occur within the time period, are compared as part of the generating the alarm. 
     
     
         15 . The apparatus of  claim 12  wherein the parameters of i) include a known period of time wherein the degree of coincidence of ii) the number of phonemes present in the left, center or right audio channels which occur during the time period and iii) the number of visemes present in the left, center or right directions which occur within the time period, are used in generating the alarm. 
     
     
         16 . The apparatus of  claim 12  further including in e) the report is generated in response to parameters of i) including the degree of coincidence of ii) the number of phonemes present in each of the left, center and right audio channels and iii) the number of visemes present in each of the left, center and right directions respectively. 
     
     
         17 . The apparatus of  claim 11  further including:
 d) an input circuit for receiving displayable motion images and a viseme detection circuit responsive to the displayable motion images to provide viseme activity data related to the presence therein of particular visemes formed by the lips of speakers in the displayable motion images which correspond to particular phonemes present in the audio signal of b), with the viseme activity data further indicating the location of the visemes present in the displayable motion images; and 
 e) wherein in c) an alarm is generated when, within a known period of time, the number of phonemes in the left, center or right audio channels does not match, within known parameters, the number of visemes in corresponding locations in the displayable motion images 
 
     
     
         18 . The apparatus of  claim 17  wherein in e) the known period of time is automatically adjusted in response to one or more of the plurality of audio channels of a). 
     
     
         19 . The apparatus of  claim 17  wherein in d) the location of the visemes in the displayable motion images is responsive to the range of horizontal addresses in the displayable image occupied by the lips forming the visemes. 
     
     
         20 . The apparatus of  claim 17  wherein in d) the location of the visemes in the displayable motion images is responsive to the range of horizontal addresses in the displayable image occupied by the lips forming the visemes and the location is related to the lips being on the left or right of the displayable motion images. 
     
     
         21 . The apparatus of  claim 19  wherein the left or right location of the visemes is compared to the left or right channel phonemes of b) and in response thereto the alarm of e) is generated. 
     
     
         22 . In an electronic audio video system such as a television system with an audio video program intended for display which system includes video carrying motion images and corresponding multiple audio channel audio signals including one or more dialogue channels and a plurality of surround effects channels, an apparatus for monitoring the placement of correct audio sounds in one or more audio channels, the apparatus including:
 a) an input for receiving a plurality of audio channels corresponding to displayable motion images including a plurality of dialogue channels the primarily intent of which is to convey spoken words and a plurality of surround effects channels the primarily intent of which is to convey non-speech sounds, the plurality of audio channels allowing, when a viewer is viewing the motion images, the viewer to hear and perceive placement of onscreen sounds relative to the motion images;   b) a phoneme detection circuit responsive to the plurality dialogue channels of a) including left and right dialog channels to provide dialogue phoneme activity data related to the presence therein of phonemes pertaining to one or more spoken vowels and/or consonants selected from the group A, E, O, M, P, B and S created by the particular shape and/or movement of a speaker's lips;   c) a phoneme detection circuit, which may be that in b) or different, responsive to the plurality surround effects channels of a) to provide surround phoneme activity data related to the presence therein of the same phoneme(s) selected in b);   d) an input circuit for receiving the displayable motion images and a viseme detection circuit responsive to the particular shape and/or movement of the speaker's lips in the displayable motion images to provide viseme activity data related to the presence of viseme(s) corresponding to at least the selected phoneme(s) of b);   e) a channel logic circuit operative to generate a tracking alarm in response to known parameters and at least one of:
 i) the dialogue phoneme activity data of b); 
 ii) the surround phoneme activity data of c); 
 iii) the viseme activity data of d). 
   
     
     
         23 . The apparatus of  claim 22  wherein in response to element e) a plurality of audio channels of a) are swapped to correct the tracking error. 
     
     
         24 . The apparatus of  claim 22  wherein in element e) the tracking alarm is provided when an insufficient number of phonemes is found in a dialogue channel during a first time period or an excessive number of phonemes is found in a sound effects channel during a second time period. 
     
     
         25 . The apparatus of  claim 22  wherein in element e) an alarm is provided when an insufficient number of phonemes is found in a dialogue channel during a time period or an excessive number of phonemes is found in a sound effects channel during the same time period. 
     
     
         26 . The apparatus of  claim 24  wherein the first or second time period is automatically adjusted in response to one or more audio channels of a). 
     
     
         27 . The apparatus of  claim 25  wherein the time period is automatically adjusted in response to one or more audio channels of a). 
     
     
         28 . The apparatus of  claim 22  wherein in b) and c) the presence of phonemes pertains to both a plurality of spoken vowels and a plurality of spoken consonants and in d) the presence of visemes pertains to the same plurality of spoken vowels and a plurality of spoken consonants as in b) and c). 
     
     
         29 . The apparatus of  claim 22  wherein in element e) the tracking alarm is provided when during a time period an insufficient number of phonemes are present in the dialogue channels of b) match the visemes present in the displayable motion images of d). 
     
     
         30 . The apparatus of  claim 22  wherein in element e) the tracking alarm is provided when during a time period an insufficient number of phonemes are present in the dialogue channels of b) match the visemes present in the displayable motion images of d) and an insufficient number of phonemes are present in the surround channels of c) match the visemes present in the displayable motion images of d). 
     
     
         31 . The apparatus of  claim 22  wherein in element d) the viseme activity data further relates to the location of the visemes as being in the left or right of the displayable image and in element e) the tracking alarm is provided when during a time period an insufficient number of phonemes is present in each of the left and right dialog channels of b) which correspond respectively to visemes present in the left and right of the displayable image. 
     
     
         32 . The apparatus of  claim 22  wherein in element d) the viseme activity data further relates to the location of the visemes as being in the left or right of the displayable image and in element e) the tracking alarm is provided when during a time period an excessive number of phonemes present in the left dialog channel of b) correspond to visemes present in the right of the displayable image. 
     
     
         33 . In a television system with a television program, the television program including motion images carried by a video signal and further including multiple audio signal channels corresponding to the motion images, an apparatus for monitoring the placement of correct audio sounds in one or more audio channels of the audio signal, the apparatus including .  a) a receiving circuit for receiving a plurality of audio channels corresponding to displayable motion images including at least left and right dialogue channels the primarily intent of which is to convey spoken words heard by a viewer of the motion images, the spoken words including words spoken by a displayable human speaker as well as words spoken by a human speaker which is outside of the displayable image, and further including at least left and right surround channels the primarily intent of which is to convey non-speech sounds which are perceived by the viewer as coming from a direction relative to both inside or outside of the displayed motion image frame;
 b) in response to the audio carried by the dialogue channels, providing dialogue activity data in response to spoken vowels and/or consonants present therein which are created by the particular shape and/or movement of a speaker's lips, the dialogue activity data also including information indicating the location of the spoken vowels in the left, center or right direction relative to at least the left and/or right dialogue channels;   c) in response to the audio carried by the surround channels, providing surround activity data in response to surround sounds present therein other than spoken words, the surround activity data also including information indicating the location of the surround sounds in the left, center or right direction relative to at least the left and/or right surround channels and further including providing surround dialogue data in response to the presence of spoken words in one more surround channels;   d) a channel logic circuit responsive to the dialogue activity data of b) and the surround dialogue activity data of c) to provide a report and/or alarm in response thereto.   
     
     
         34 . The apparatus of  claim 33  wherein in response to element d) a plurality of audio channels of a) are swapped to correct the tracking error. 
     
     
         35 . An apparatus as claimed in  claim 33  where in d) alarms are provided for an event chosen from the list of:
 i) insufficient dialogue activity data during a first time period; 
 ii) excessive surround dialogue activity during a second time period; 
 iii) insufficient matching of left or right dialogue phonemes and left or right dialogue visemes respectively during a third time period; 
 iv) excessive matching of left or right dialogue phonemes and left or right dialogue visemes respectively during a fourth time period; 
 and wherein the first, second, third and/or fourth time periods may be the same. 
 
     
     
         36 . An apparatus as claimed in  claim 33  where in d) alarms are provided for a plurality of events chosen from the list of;
 i) insufficient dialogue activity data during a first time period, 
 ii) excessive surround dialogue activity during a second time period, 
 iii) insufficient matching of left or right dialogue phonemes and left or right dialogue visemes respectively during a third time period, 
 iv) excessive matching of left or right dialogue phonemes and left or right dialogue visemes respectively during a fourth time period, 
 
       wherein the first, second, third and/or fourth time periods may be the same.