Audio-visual monitoring using a virtual assistant

Abstract

A function of a user-controlled virtual assistant (UCVA) device, such as a smart speaker, can be augmented using video or image information about an environment. In an example, a system for augmenting an UCVA device includes an image sensor configured to monitor an environment, a processor circuit configured to receive image information from the image sensor and use artificial intelligence to discern a presence of one or more known individuals in the environment from one or more other features in the environment. The system can include an interface coupled to the processor circuit and configured to provide identification information to the UCVA device about the one or more known human beings in the environment. The UCVA device can be configured by the identification information to update an operating mode of the UCVA device.

What is claimed is: 
     
         1 . An environment analysis device comprising:
 a processor circuit configured to receive image information from an image sensor and to receive audio information from an audio sensor; and   a non-transitory memory circuit coupled to the processor circuit, the non-transitory memory circuit comprising instructions that, when performed by the processor circuit, configure the processor circuit to:
 analyze one of the image information or the audio information to identify whether a specified individual is present in an environment; 
 use the other one of the image information and the audio information to confirm that the specified individual is present in the environment; and 
 provide an analysis result indicating whether the specified individual is confirmed to be present in the environment. 
   
     
     
         2 . The environment analysis device of  claim 1 , wherein the instructions further configure the processor circuit to perform a task associated with the specified individual when the specified individual is confirmed to be present in the environment. 
     
     
         3 . The environment analysis device of  claim 1 , further comprising:
 an image sensor configured to provide the image information about the environment; and   an audio sensor configured to provide the audio information about the same environment.   
     
     
         4 . The environment analysis device of  claim 3 , wherein the processor circuit comprises a portion of an access-controlling security system. 
     
     
         5 . The environment analysis device of  claim 1 , wherein the processor circuit is configured to receive environment condition information about the environment, from an environment sensor, and wherein the instructions further configure the processor circuit to provide the analysis result based on an analysis of the image information, the audio information, and the environment information together. 
     
     
         6 . The environment analysis device of  claim 5 , wherein the instructions further configure the processor circuit to apply machine learning to analyze the image information, the audio information, and the environment information together. 
     
     
         7 . The environment analysis device of  claim 5 , further comprising the environment sensor, wherein the environment sensor comprises one or more of a temperature sensor, door sensor, window sensor, or lock sensor. 
     
     
         8 . The environment analysis device of  claim 1 , wherein the instructions further configure the processor circuit to apply deep learning to classify the received audio information about the environment as including one or more of a dog bark, a glass break or other material break, a gun shot, human speech, or an environment alarm. 
     
     
         9 . The environment analysis device of  claim 1 , wherein the instructions further configure the processor circuit to apply deep learning to classify the received image information as corresponding to or excluding the specified individual. 
     
     
         10 . The environment analysis device of  claim 1 , wherein the instructions further configure the processor circuit to provide the analysis result based on applied deep learning that uses the image information and the audio information as inputs. 
     
     
         11 . An artificial intelligence-enabled security system comprising:
 a processor circuit configured to receive information about an environment from multiple different environment sensors; and   a non-transitory memory circuit coupled to the processor circuit, the non-transitory memory circuit comprising instructions that, when performed by the processor circuit, configure the processor circuit to apply artificial intelligence-based processing to analyze the information received from the multiple different environment sensors together, and in response provide an analysis result that indicates a security status of the environment.   
     
     
         12 . The system of  claim 11 , wherein the processor circuit is configured to receive the information about the environment from an image sensor and from an audio sensor; and
 wherein the instructions further configure the processor circuit to use the information about the environment, including image information from the image sensor and audio information from the audio sensor, with the applied artificial intelligence-based processing to authenticate an individual in the environment.   
     
     
         13 . The system of  claim 11 , wherein the processor circuit is configured to receive the information about the environment from an image sensor and from an audio sensor; and
 wherein the instructions further configure the processor circuit to:
 use the information about the environment, including image information from the image sensor and audio information from the audio sensor, with the applied artificial intelligence-based processing to provide an alert about a presence or absence of an object in the environment. 
   
     
     
         14 . The system of  claim 11 , wherein the instructions to apply the artificial intelligence-based processing include instructions to use a machine learning algorithm to receive the information from the multiple different environment sensors and, in response, provide the analysis result. 
     
     
         15 . The system of  claim 11 , wherein the instructions to apply the artificial intelligence-based processing include instructions to use a deep learning algorithm to receive the information from the multiple different environment sensors and, in response, provide the analysis result. 
     
     
         16 . The system of  claim 11 , wherein the processor circuit is configured to receive image information about the environment and to receive audio information about the environment; and
 wherein the instructions to apply the artificial intelligence-based processing include instructions to:
 analyze one of the image information or the audio information to identify whether a known individual is present in the environment; 
 use the other one of the image information and the audio information to confirm that the known individual is present in the environment; and 
 provide the analysis result based on the confirmed presence or absence of the known individual in the environment. 
   
     
     
         17 . The system of  claim 16 , wherein the instructions to apply the artificial intelligence-based processing include instructions to perform a personalized task associated with the known individual when the known individual is confirmed to be present in the environment. 
     
     
         18 . A method comprising:
 receiving, at a processor circuit, respective signals with information about an environment, the signals provided by respective different environment sensors;   using a processor circuit, applying artificial intelligence-based processing to analyze together the information about the environment as-received from the different environment sensors and, based on the analysis, providing an analysis result that indicates a security status of the environment.   
     
     
         19 . The method of  claim 18 , wherein applying the artificial intelligence-based processing includes receiving the information about the environment from multiple different environment sensors as inputs to a machine learning algorithm or a deep learning algorithm that provides the analysis result. 
     
     
         20 . The method of  claim 19 , wherein receiving the signals with information about the environment includes receiving audio information from an audio sensor and receiving image information from an image sensor; and
 wherein applying the artificial intelligence-based processing includes processing the image information and the audio information together and, in response, providing an alert about a presence or absence of an object or an individual in the environment.