Method and system for generating video cover based on browser

Abstract

This application provides techniques for generating a video cover. The techniques comprise monitoring whether the browser enters a target page; initializing a main thread and creating a frame extraction thread and an image evaluation thread; monitoring a target action on the target page extracting a plurality of target frames from the local video file using the webassembly video parser running by the frame extraction thread; determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model running by the image evaluation thread; obtaining the image evaluation parameters of each of the plurality of target frames from the image evaluation thread by the main thread, selecting one or more candidate frames from the plurality of target frames based on the image evaluation parameters of each of the plurality of target frames, and generating the video cover based on the one or more candidate frames.

What is claimed is: 
     
         1 . A method for generating a video cover based on a browser executed by a computing device, comprising:
 monitoring whether the browser enters a target page;   initializing a main thread and creating a frame extraction thread and an image evaluation thread in response to determining that the browser enters the target page, wherein the frame extraction thread is configured to load a webassembly video parser in the browser to provide browser-based frame extraction, wherein the image evaluation thread is configured to load a trained image evaluation model, and wherein the trained image evaluation model comprises a trained neural network;   monitoring a target action on the target page, wherein the target action is associated with a local video file;   in response to the target action, performing operations comprising:   extracting a plurality of target frames from the local video file using the webassembly video parser, wherein the webassembly is running by frame extraction thread;   determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model, wherein the trained image evaluation model is running by the image evaluation thread;   obtaining the image evaluation parameters of each of the plurality of target frames from the image evaluation thread by the main thread;   selecting one or more candidate frames from the plurality of target frames based on the image evaluation parameters of each of the plurality of target frames; and   generating the video cover based on the one or more candidate frames.   
     
     
         2 . The method of  claim 1 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 determining whether the local video file is in a portrait orientation using the webassembly video parser;   extracting a plurality of frames from the local video file using the webassembly video parser;   performing image rotation on each of the plurality of frames to obtain frames in the portrait orientation in response to determining that the local video file is in the portrait orientation; and   determining the plurality of frames in the portrait orientation as the plurality of target frames.   
     
     
         3 . The method of  claim 1 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 obtaining N frames corresponding to N time nodes, wherein the obtaining N frames corresponding to N time nodes further comprises:   acquiring a key frame closest to a time node M, and determining the key frame as a frame corresponding to the time node M, wherein 1≤M≤N.   
     
     
         4 . The method of  claim 1 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 performing a detection operation on the local video file; and   determining whether to extract the plurality of target frames from the local video file based on a detection result, wherein the detection operation is configured to detect whether the local video file is a damaged file, whether the local video file contains a video stream, and whether a video format of the video stream is supported by the webassembly video parser.   
     
     
         5 . The method of  claim 1 , wherein the determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model further comprises:
 extracting image features of a target frame M among the plurality of target frames by a feature extraction layer of the trained image evaluation model, wherein 1≤M≤N and N represents a total number of the plurality of target frames;   determining confidence levels of the target frame M based on the image features of the target frame M by a first fully connected layer of the trained image evaluation model, the confidence levels corresponding to a plurality of scene categories;   determining an image quality evaluation value of the target frame M based on the image features of the target frame M by a second fully connected layer of the trained image evaluation model; and   determining the image evaluation parameters of the target frame M based on the confidence levels of the target frame M and the image quality evaluation value of the target frame M.   
     
     
         6 . The method of  claim 5 , wherein the determining the image evaluation parameters of the target frame M based on the confidence levels of the target frame M and the image quality evaluation value of the target frame M further comprises:
 determining the image evaluation parameters P of the target frame M based on a formula:
   P=p2Σ i W i,arg max p1 p1 i  
 
   wherein p1 i  represents a confidence level of the target frame M corresponding to a scene category I, p2 represents the image quality evaluation value of the target frame M, arg max p1 represents a target scene category with a maximum confidence level, and Wi,arg max p1 represents a weight of a degree of association between the target scene category corresponding to target frame M and an ith scene category.   
     
     
         7 . The method of  claim 1 , wherein the generating the video cover based on the one or more candidate frames further comprises:
 displaying the one or more candidate frames in a predetermined area of the target page;   selecting a candidate frame from the one or more candidate frames based on user input; and   generating the video cover based on the selected candidate frame, wherein the video cover is associated with the local video file, and wherein the video cover is sent to a server.   
     
     
         8 . A computing device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable by the processor, wherein the processor executes the computer-readable instructions to perform operations comprising:
 monitoring whether the browser enters a target page;   initializing a main thread and creating a frame extraction thread and an image evaluation thread in response to determining that the browser enters the target page, wherein the frame extraction thread is configured to load a webassembly video parser in the browser to provide browser-based frame extraction, wherein the image evaluation thread is configured to load a trained image evaluation model, and wherein the trained image evaluation model comprising a trained neural network;   monitoring a target action on the target page, wherein the target action is associated with a local video file;   in response to the target action, performing operations comprising:   extracting a plurality of target frames from the local video file using the webassembly video parser, wherein the webassembly is running by frame extraction thread;   determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model, wherein the trained image evaluation model is running by the image evaluation thread;   obtaining the image evaluation parameters of each of the plurality of target frames from the image evaluation thread by the main thread;   selecting one or more candidate frames from the plurality of target frames based on the image evaluation parameters of each of the plurality of target frames; and   generating the video cover based on the one or more candidate frames.   
     
     
         9 . The computing device of  claim 8 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 determining whether the local video file is in a portrait orientation using the webassembly video parser;   extracting a plurality of frames from the local video file using the webassembly video parser;   performing image rotation on each of the plurality of frames to obtain frames in the portrait orientation in response to determining that the local video file is in the portrait orientation; and   determining the plurality of frames in the portrait orientation as the plurality of target frames.   
     
     
         10 . The computing device of  claim 8 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 obtaining N frames corresponding to N time nodes, wherein the obtaining N frames corresponding to N time nodes further comprises:   acquiring a key frame closest to a time node M, and determining the key frame as a frame corresponding to the time node M, wherein 1≤M≤N.   
     
     
         11 . The computing device of  claim 8 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 performing a detection operation on the local video file; and   determining whether to extract the plurality of target frames from the local video file based on a detection result, wherein the detection operation is configured to detect whether the local video file is a damaged file, whether the local video file contains a video stream, and whether a video format of the video stream is supported by the webassembly video parser.   
     
     
         12 . The computing device of  claim 8 , wherein the determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model further comprises:
 extracting image features of a target frame M among the plurality of target frames by a feature extraction layer of the trained image evaluation model, wherein 1≤M≤N, and N represents a total number of the plurality of target frames;   determining confidence levels of the target frame M based on the image features of the target frame M by a first fully connected layer of the trained image evaluation model, the confidence levels corresponding to a plurality of scene categories;   determining an image quality evaluation value of the target frame M based on the image features of the target frame M by a second fully connected layer of the trained image evaluation model; and   determining the image evaluation parameters of the target frame M based on the confidence levels of the target frame M and the image quality evaluation value of the target frame M.   
     
     
         13 . The computing device of  claim 12 , wherein the determining the image evaluation parameters of the target frame M based on the confidence levels of the target frame M and the image quality evaluation value of the target frame M further comprises:
 determining the image evaluation parameters P of the target frame M based on a formula:
   P=p2Σ i W i,arg max p1 p1 i  
 
   wherein p1 i  represents a confidence level of the target frame M corresponding to a scene category I, p2 represents the image quality evaluation value of the target frame M, arg max p1 represents a target scene category with a maximum confidence level, and Wi,arg max p1 represents a weight of a degree of association between the target scene category corresponding to target frame M and an ith scene category.   
     
     
         14 . The computing device of  claim 8 , wherein the generating the video cover based on the one or more candidate frames further comprises:
 displaying the one or more candidate frames in a predetermined area of the target page;   selecting a candidate frame from the one or more candidate frames based on user input; and   generating the video cover based on the selected candidate frame, wherein the video cover is associated with the local video file, and wherein the video cover is sent to a server.   
     
     
         15 . A non-transitory computer-readable storage medium having computer-readable instructions stored therein, the computer-readable instructions being executable by at least one processor to cause the at least one processor to perform operations comprising:
 monitoring whether the browser enters a target page;   initializing a main thread and creating a frame extraction thread and an image evaluation thread in response to determining that the browser enters the target page, wherein the frame extraction thread is configured to load a webassembly video parser in the browser to provide browser-based frame extraction, wherein the image evaluation thread is configured to load a trained image evaluation model, and wherein the trained image evaluation model comprising a trained neural network;   monitoring a target action on the target page, wherein the target action is associated with a local video file;   in response to the target action, performing operations comprising:   extracting a plurality of target frames from the local video file using the webassembly video parser, wherein the webassembly is running by frame extraction thread;   determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model, wherein the trained image evaluation model is running by the image evaluation thread;   obtaining the image evaluation parameters of each of the plurality of target frames from the image evaluation thread by the main thread;   selecting one or more candidate frames from the plurality of target frames based on the image evaluation parameters of each of the plurality of target frames; and   generating the video cover based on the one or more candidate frames.   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 15 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 determining whether the local video file is in a portrait orientation using the webassembly video parser;   extracting a plurality of frames from the local video file using the webassembly video parser;   performing image rotation on each of the plurality of frames to obtain frames in the portrait orientation in response to determining that the local video file is in the portrait orientation; and   determining the plurality of frames in the portrait orientation as the plurality of target frames.   
     
     
         17 . The non-transitory computer-readable storage medium of  claim 15 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 obtaining N frames corresponding to N time nodes, wherein the obtaining N frames corresponding to N time nodes further comprises:   acquiring a key frame closest to a time node M, and determining the key frame as a frame corresponding to the time node M, wherein 1≤M≤N.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 15 , wherein the extracting a plurality of target frames from the local video file using the webassembly video parser further comprises:
 performing a detection operation on the local video file; and   determining whether to extract the plurality of target frames from the local video file based on a detection result, wherein the detection operation is configured to detect whether the local video file is a damaged file, whether the local video file contains a video stream, and whether a video format of the video stream is supported by the webassembly video parser.   
     
     
         19 . The non-transitory computer-readable storage medium of  claim 15 , wherein the determining image evaluation parameters of each of the plurality of target frame using the trained image evaluation model further comprises:
 extracting image features of a target frame M among the plurality of target frames by a feature extraction layer of the trained image evaluation model, wherein 1≤M≤N, and N represents a total number of the plurality of target frames;   determining confidence levels of the target frame M based on the image features of the target frame M by a first fully connected layer of the trained image evaluation model, the confidence levels corresponding to a plurality of scene categories;   determining an image quality evaluation value of the target frame M based on the image features of the target frame M by a second fully connected layer of the trained image evaluation model; and   determining the image evaluation parameters of the target frame M based on the confidence levels of the target frame M and the image quality evaluation value of the target frame M.