Method and system for providing augmented reality object tracking service based on deep learning

Abstract

A method for providing an AR object tracking service based on deep learning according to an embodiment of the present disclosure, by which a tracking application executed by at least one processor of a terminal provides an AR object tracking service based on deep learning, comprises obtaining first image data; inputting the first image data to a first deep learning neural network; obtaining 3D depth data including each descriptor of a target object within the first image data and a distance value corresponding to the descriptor from the first deep learning neural network; and performing AR object tracking based on the 3D depth data, wherein the first deep learning neural network is a deep learning algorithm performing monocular depth estimation.

What is claimed is: 
     
         1 . A method for providing an AR object tracking service based on deep learning, by which a tracking application executed by at least one processor of a terminal provides an AR object tracking service based on deep learning, the method comprising:
 obtaining first image data;   inputting the first image data to a first deep learning neural network;   obtaining 3D depth data including each descriptor of a target object within the first image data and a distance value corresponding to the descriptor from the first deep learning neural network; and   performing AR object tracking based on the 3D depth data,   wherein the first deep learning neural network is a deep learning algorithm performing monocular depth estimation.   
     
     
         2 . The method of  claim 1 , further comprising:
 inputting the first image data to a second deep learning neural network and obtaining object area information representing the area occupied by each object within the first image data from the second deep learning neural network,   wherein the second deep learning neural network is a deep learning algorithm performing semantic segmentation.   
     
     
         3 . The method of  claim 2 , further comprising:
 determining the area of the target object based on the object area information.   
     
     
         4 . The method of  claim 3 , wherein the determining of the target object area includes:
 detecting at least one target object candidate area based on the object area information and   determining one of the target object candidate areas as a target object area.   
     
     
         5 . The method of  claim 3 , further comprising:
 obtaining the 3D depth data based on the area of the target object.   
     
     
         6 . The method of  claim 2 , further comprising:
 operating the first deep learning neural network and the second deep learning neural network in parallel.   
     
     
         7 . The method of  claim 1 , further comprising:
 obtaining the 3D depth data based on a primitive model with a preconfigured 2D or 3D shape.   
     
     
         8 . The method of  claim 7 , further comprising:
 generating 3D integrated depth data combining the 3D depth data obtained based on the primitive model and the 3D depth data obtained based on the first deep learning neural network and   performing AR object tracking based on the 3D integrated depth data.   
     
     
         9 . A system for providing an AR object tracking service based on deep learning comprising:
 at least one memory storing a tracking application; and   at least one processor providing an AR object tracking service based on deep learning by reading the tracking application stored in the memory,   wherein commands of the tracking application include commands for performing:   obtaining first image data,   inputting the first image data to a first deep learning neural network,   obtaining 3D depth data including each descriptor of a target object within the first image data and a distance value corresponding to the descriptor from the first deep learning neural network, and   performing AR object tracking based on the 3D depth data.   
     
     
         10 . The system of  claim 9 , wherein the commands of the tracking application further comprise commands for performing:
 generating a 3D definition model based on the 3D depth data,   determining a target virtual object to be augmented and displayed based on the target object,   generating an AR environment model by anchoring the target virtual object and the 3D definition model, and   performing the AR object tracking based on the AR environment model.