Moving state analysis device, moving state analysis method, and program

Abstract

A moving state analysis device improves accuracy of moving state recognition by including a detection unit configured to detect, from image data associated with a frame, an object and a region of the object, for each of frames that constitute first video data captured in a course of movement of a first moving body, and a learning unit configured to learn a DNN model that takes video data and sensor data as input and that outputs a probability of each moving state, based on the first video data, a feature of first sensor data measured in relation to the first moving body and corresponding to a capture of the first video data, a detection result of the object and the region of the object, and information that indicates a moving state associated with the first video data.

What is claimed is: 
     
       1. A moving state analysis device comprising:
 a memory; and 
 a processor coupled to the memory and configured to: 
 detect, from image data associated with a frame, an object and a region of the object, for each of frames that constitute first video data captured in a course of movement of a first moving body; and 
 learn a deep neural network model that takes video data and sensor data as input and that outputs a probability of each moving state associated with a moving body, based on a combination of at least:
 the first video data, 
 a feature of first sensor data including positioning information measured in relation to the first moving body at a time of capturing the first video data, 
 a detection result of the object and the region of the object appearing in the first video data, and 
 information indicating a moving state associated with the first moving body relative to the object detected in the first video data. 
 
 
     
     
       2. The moving state analysis device according to  claim 1 , wherein the processor is configured to:
 detect, from image data associated with a frame, an object and a region of the object, for each of frames that constitute second video data captured in a course of movement of a second moving body; and 
 calculate a probability of each moving state, with respect to the second video data, by inputting into the deep neural network model a combination of at least:
 the second video data, 
 second sensor data measured in relation to the second moving body at a time of capturing the second video data, and 
 a detection result of the object and the region of the object detected from the image data associated with the frame from which the second video data is constituted, 
 
 the deep neural network model being read and executed by the hardware processor. 
 
     
     
       3. The moving state analysis device according to  claim 2 , wherein the processor is configured to:
 generate, based on the detection result of the object and the region of the object, data indicating for each object a feature of a region in which the object appears; 
 learn the deep neural network model based on the generated data in relation to the first video data; and 
 calculate a probability of each moving state associated with the moving body, based on the generated data in relation to the second video data. 
 
     
     
       4. The moving state analysis device according to  claim 1 , wherein the positioning information includes either latitude or longitude of a location corresponding to the first moving body. 
     
     
       5. The moving state analysis device according to  claim 1 , the processor further configured to capture the first sensor data using a sensor. 
     
     
       6. The moving state analysis device according to  claim 1 , wherein the first video data and the first sensor data are captured on the first moving body, and wherein the first moving body is distinct from the object. 
     
     
       7. A computer-implemented method for analyzing a moving state, the method comprising:
 detecting an object and a region of the object from image data associated with a frame, for each of frames that constitute first video data captured in a course of movement of a first moving body; and 
 learning a deep neural network model that takes video data and sensor data as input and that outputs a probability of each moving state associated with a moving body, based on a combination of at least:
 the first video data, 
 a feature of first sensor data including positioning information measured in relation to the first moving body at a time of capturing the first video data, 
 a detection result of the object and the region of the object appearing in the first video data, and 
 information indicating a moving state associated with the first moving body relative to the object detected in the first video data. 
 
 
     
     
       8. The moving state analysis method executed by a computer according to  claim 7 , further comprising:
 detecting an object and a region of the object from image data associated with a frame, for each of frames that constitute second video data captured in a course of movement of a second moving body; and 
 calculating a probability of each moving state, with respect to the second video data, by inputting into the deep neural network model a combination of at least:
 the second video data, 
 second sensor data measured in relation to the second moving body at a time of capturing the second video data, and 
 a detection result of the object and the region of the object detected from the image data associated with the frame from which the second video data is constituted. 
 
 
     
     
       9. The moving state analysis method executed by a computer according to  claim 8 , further comprising:
 generating data indicating for each object a feature of a region in which the object appears, based on the detection result of the object and the region of the object, wherein 
 in the learning, the deep neural network model is learned based on data generated in the generating in relation to the first video data; and 
 in the calculating, a probability of each moving state associated with the moving body is calculated based on data generated in the generating in relation to the second video data. 
 
     
     
       10. A non-transitory computer-readable recording medium having a program that causes a computer to execute the moving state analysis method of  claim 7 . 
     
     
       11. The non-transitory computer-readable recording medium according to  claim 10 , wherein the positioning information includes either latitude or longitude of a location corresponding to the first moving body. 
     
     
       12. The non-transitory computer-readable recording medium according to  claim 10 , the method further comprising capturing the first sensor data using a sensor. 
     
     
       13. The non-transitory computer-readable recording medium according to  claim 10 , wherein the positioning information includes either latitude or longitude of a location corresponding to the first moving body. 
     
     
       14. The computer-implemented method according to  claim 7 , wherein the positioning information includes either latitude or longitude of a location corresponding to the first moving body. 
     
     
       15. The computer-implemented method according to  claim 7 , the method further comprising:
 capturing the first sensor data using a sensor. 
 
     
     
       16. The computer-implemented method according to  claim 7 , wherein the positioning information includes either latitude or longitude of a location corresponding to the first moving body.