Image processing method and apparatus

Abstract

An image processing method includes: acquiring a video stream of a vehicle by a camera according to a user instruction; obtaining an image corresponding to a frame in the video stream; determining whether the image meets a predetermined criterion by inputting the image into a classification model, the classification model comprising a first convolutional neural network; in response to the image meeting the predetermined criterion, adding at least one of a target box or target segmentation information to the image by inputting the image into a target detection and segmentation model, the at least one of the target box or the target segmentation information corresponding to at least one of a vehicle part or vehicle damage of the vehicle, the target detection and segmentation model comprising a convolutional neural network; and displaying the at least one of the target box or the target segmentation information to the user.

What is claimed is: 
     
         1 . An image processing method for use in a mobile device, comprising:
 acquiring a video stream of a vehicle by a camera of the mobile device according to a user instruction;   obtaining an image corresponding to a frame in the video stream;   determining whether the image meets a predetermined criterion by inputting the image into a classification model, the classification model comprising a first convolutional neural network for use in the mobile device;   in response to the image meeting the predetermined criterion, adding at least one of a target box or target segmentation information to the image by inputting the image into a target detection and segmentation model, the at least one of the target box or the target segmentation information corresponding to at least one of a vehicle part or vehicle damage of the vehicle, the target detection and segmentation model comprising a second convolutional neural network for use in the mobile device; and   displaying to a user the at least one of the target box or the target segmentation information.   
     
     
         2 . The image processing method of  claim 1 , further comprising:
 in response to the image not meeting the predetermined criterion, prompting the user based on a classification result from the classification model.   
     
     
         3 . The image processing method of  claim 1 , wherein the classification model classifies the image based on at least one of: whether the image is blurred, whether the image includes vehicle damage, whether a light intensity is sufficient, whether a shooting angle is skewed, or whether a shooting distance is appropriate. 
     
     
         4 . The image processing method of  claim 1 , further comprising:
 presenting a shooting flow to the user before the video stream of the vehicle is acquired by the camera.   
     
     
         5 . The image processing method of  claim 1 , further comprising:
 prompting the user based on the at least one of the target box or the target segmentation information.   
     
     
         6 . The image processing method of  claim 5 , wherein the prompting the user based on the at least one of the target box or the target segmentation information comprises:
 prompting the user to move forward or backward based on the at least one of the target box or the target segmentation information.   
     
     
         7 . The image processing method of  claim 5 , wherein the prompting the user based on the at least one of the target box or the target segmentation information comprises:
 prompting the user to shoot based on the at least one of the target box or the target segmentation information, to obtain a damage assessment photo corresponding to the image of the frame.   
     
     
         8 . The image processing method of  claim 7 , further comprising:
 uploading the damage assessment photo to a server.   
     
     
         9 . The image processing method of  claim 7 , further comprising:
 obtaining, based on the video stream, an association between the image and a first image, wherein the first image is an image of a first frame before the frame in the video stream.   
     
     
         10 . The image processing method of  claim 9 , wherein the association comprises at least one of: an optical flow, a mapping matrix, or a position and angle transformation relation between the image and the first image. 
     
     
         11 . The image processing method of  claim 9 , further comprising:
 uploading the association to the server.   
     
     
         12 . The image processing method of  claim 1 , wherein the first convolutional neural network and the second convolutional neural network are a same convolutional neural network shared by the classification model and the target detection and segmentation model. 
     
     
         13 . A mobile device, comprising:
 a memory storing instructions; and   a processor configured to execute the instructions to:
 acquire a video stream of a vehicle by a camera according to a user instruction; 
 obtain an image corresponding to a frame in the video stream; 
 determine whether the image meets a predetermined criterion by inputting the image into a classification model, the classification model comprising a first convolutional neural network for use in the mobile device; 
 if the image meets the predetermined criterion, add at least one of a target box or target segmentation information to the image by inputting the image into a target detection and segmentation model, the at least one of the target box or the target segmentation information corresponding to at least one of a vehicle part or vehicle damage of the vehicle, the target detection and segmentation model comprising a second convolutional neural network for use in the mobile device; and 
 display to a user the at least one of the target box or the target segmentation information. 
   
     
     
         14 . The mobile device of  claim 13 , wherein the processor is further configured to execute the instructions to:
 in response to the image not meeting the predetermined criterion, prompt the user based on a classification result from the classification model.   
     
     
         15 . The mobile device of  claim 13 , wherein the classification model classifies the image based on at least one of: whether the image is blurred, whether the image includes vehicle damage, whether a light intensity is sufficient, whether a shooting angle is skewed, or whether a shooting distance is appropriate. 
     
     
         16 . The mobile device of  claim 13 , wherein the processor is further configured to execute the instructions to:
 present a shooting flow to the user before the video stream of the vehicle is acquired by the camera.   
     
     
         17 . The mobile device of  claim 13 , wherein the processor is further configured to execute the instructions to:
 prompt the user based on the at least one of the target box or the target segmentation information.   
     
     
         18 . The mobile device of  claim 17 , wherein the prompting the user based on the at least one of the target box or the target segmentation information comprises:
 prompting the user to move forward or backward based on the at least one of the target box or the target segmentation information.   
     
     
         19 . The mobile device of  claim 17 , wherein the prompting the user based on the at least one of the target box or the target segmentation information comprises:
 prompting the user to shoot based on the at least one of the target box or the target segmentation information, to obtain a damage assessment photo corresponding to the image of the frame.   
     
     
         20 . The mobile device of  claim 19 , wherein the processor is further configured to execute the instructions to:
 upload the damage assessment photo to a server.   
     
     
         21 . The mobile device of  claim 19 , wherein the processor is further configured to execute the instructions to:
 obtain, based on the video stream, an association between the image and a first image, wherein the first image is an image of a first frame before the frame in the video stream.   
     
     
         22 . The mobile device of  claim 21 , wherein the association comprises at least one of: an optical flow, a mapping matrix, or a position and angle transformation relation between the image and the first image. 
     
     
         23 . The mobile device of  claim 21 , wherein the processor is further configured to execute the instructions to:
 upload the association to the server.   
     
     
         24 . The mobile device of  claim 13 , wherein the first convolutional neural network and the second convolutional neural network are a same convolutional neural network shared by the classification model and the target detection and segmentation model. 
     
     
         25 . A non-transitory computer-readable medium storing instructions that, when executed by a processor of a mobile device, cause the mobile device to perform an image processing method, the method comprising:
 acquiring a video stream of a vehicle by a camera according to a user instruction;   obtaining an image corresponding to a frame in the video stream;   determining whether the image meets a predetermined criterion by inputting the image into a classification model, the classification model comprising a first convolutional neural network;   if the image meets the predetermined criterion, adding at least one of a target box or target segmentation information to the image by inputting the image into a target detection and segmentation model, the at least one of the target box or the target segmentation information corresponding to at least one of a vehicle part or vehicle damage of the vehicle, the target detection and segmentation model comprising a second convolutional neural network; and   displaying to a user the at least one of the target box or the target segmentation information.