System and method for automated detection of situational awareness

Abstract

Embodiments of the present systems and methods may provide automated techniques that may provide enhanced security and safety and reduced costs. For example, in an embodiment, a method implemented in a computer may comprise receiving, at the computer system, data capturing an event, generating, at the computer system, a narrativization of the data characterizing the event captured in the data, detecting, at the computer system, at least one entity involved in the event captured in the data, obtaining, at the computer system, ontology information based on the generated narrativization and the detected at least one entity, determining, at the computer system, an intent of the at least one detected entity involved in the event captured in the data, and performing, at the computer system, an action responsive to the determined intent.

What is claimed is: 
     
         1 . A method implemented in a computer comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising:
 receiving, at the computer system, data capturing an event;   generating, at the computer system, a narrativization of the data characterizing the event captured in the data using at least one model for each type of the data, the models stored in a models database comprising at least some of text models, video models, audio models, combination models, enrichment models, encoder models, and decoder models;   detecting, at the computer system, at least one entity involved in the event captured in the data;   obtaining, at the computer system, ontology information based on the generated narrativization and the detected at least one entity;   determining, at the computer system, an intent of the at least one detected entity involved in the event captured in the data; and   performing, at the computer system, an action responsive to the determined intent.   
     
     
         2 . The method of  claim 1 , wherein the data capturing an event comprises at least one of image data, video data, text data, audio data, and sensor data. 
     
     
         3 . The method of  claim 2 , wherein the data capturing an event comprises at least one of real-time data relating to events occurring contemporaneously and stored data relating events that occurred in the past. 
     
     
         4 . The method of  claim 2 , wherein generating the narrativization comprises at least one of:
 captioning, at the computer system, image data,   captioning, at the computer system, image data video data,   recognizing, at the computer system, speech included in audio data,   generating summary data, at the computer system, characterizing text data, and   generating summary data, at the computer system, characterizing sensor data.   
     
     
         5 . The method of  claim 1 , wherein detecting at least one entity comprises at least one of:
 detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from image data using at least one of image object recognition models, image movement recognition models, image facial recognition models, and image situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from video data using at least one of video object recognition models, video movement recognition models, video facial recognition models, and video situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from audio data using at least one of audio object recognition models, audio movement recognition models, audio speaker recognition models, and audio situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of text object recognition models, text activity recognition models, text situation recognition models, and text person recognition models; and   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of sensor object recognition models, sensor activity recognition models, sensor situation recognition models, and sensor person recognition models.   
     
     
         6 . A system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:
 receiving data capturing an event;   generating a narrativization of the data characterizing the event captured in the data using at least one model for each type of the data, the models stored in a models database comprising at least some of text models, video models, audio models, combination models, enrichment models, encoder models, and decoder models;   detecting at least one entity involved in the event captured in the data;   obtaining ontology information based on the generated narrativization and the detected at least one entity;   determining an intent of the at least one detected entity involved in the event captured in the data; and   performing an action responsive to the determined intent.   
     
     
         7 . The system of  claim 6 , wherein the data capturing an event comprises at least one of image data, video data, text data, audio data, and sensor data. 
     
     
         8 . The system of  claim 7 , wherein the data capturing an event comprises at least one of real-time data relating to events occurring contemporaneously and stored data relating events that occurred in the past. 
     
     
         9 . The system of  claim 7 , wherein generating the narrativization comprises at least one of:
 captioning image data,   captioning image data video data,   recognizing speech included in audio data,   generating summary data characterizing text data, and   generating summary data characterizing sensor data.   
     
     
         10 . The system of  claim 6 , wherein detecting at least one entity comprises at least one of:
 detecting the at least one entity comprising at least one of an object, activity, situation, and person from image data using at least one of image object recognition models, image movement recognition models, image facial recognition models, and image situation recognition models;   detecting the at least one entity comprising at least one of an object, activity, situation, and person from video data using at least one of video object recognition models, video movement recognition models, video facial recognition models, and video situation recognition models;   detecting the at least one entity comprising at least one of an object, activity, situation, and person from audio data using at least one of audio object recognition models, audio movement recognition models, audio speaker recognition models, and audio situation recognition models;   detecting the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of text object recognition models, text activity recognition models, text situation recognition models, and text person recognition models; and   detecting the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of sensor object recognition models, sensor activity recognition models, sensor situation recognition models, and sensor person recognition models.   
     
     
         11 . A computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:
 receiving, at the computer system, data capturing an event;   generating, at the computer system, a narrativization of the data characterizing the event captured in the data using at least one model for each type of the data, the models stored in a models database comprising at least some of text models, video models, audio models, combination models, enrichment models, encoder models, and decoder models;   detecting, at the computer system, at least one entity involved in the event captured in the data;   obtaining, at the computer system, ontology information based on the generated narrativization and the detected at least one entity;   determining, at the computer system, an intent of the at least one detected entity involved in the event captured in the data; and   performing, at the computer system, an action responsive to the determined intent.   
     
     
         12 . The computer program product of  claim 11 , wherein the data capturing an event comprises at least one of image data, video data, text data, audio data, and sensor data. 
     
     
         13 . The computer program product of  claim 12 , wherein the data capturing an event comprises at least one of real-time data relating to events occurring contemporaneously and stored data relating events that occurred in the past. 
     
     
         14 . The computer program product of  claim 12 , wherein generating the narrativization comprises at least one of:
 captioning, at the computer system, image data,   captioning, at the computer system, image data video data,   recognizing, at the computer system, speech included in audio data,   generating summary data, at the computer system, characterizing text data, and   generating summary data, at the computer system, characterizing sensor data.   
     
     
         15 . The computer program product of  claim 11 , wherein detecting at least one entity comprises at least one of:
 detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from image data using at least one of image object recognition models, image movement recognition models, image facial recognition models, and image situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from video data using at least one of video object recognition models, video movement recognition models, video facial recognition models, and video situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from audio data using at least one of audio object recognition models, audio movement recognition models, audio speaker recognition models, and audio situation recognition models;   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of text object recognition models, text activity recognition models, text situation recognition models, and text person recognition models; and   detecting, at the computer system, the at least one entity comprising at least one of an object, activity, situation, and person from text data using at least one of sensor object recognition models, sensor activity recognition models, sensor situation recognition models, and sensor person recognition models.