Model serving for advanced frequency management

Abstract

Systems and methods for entity detection using artificial intelligence, including: a deep learning model service configured to: select and analyze a set of frames from a media item to determine a set of candidate brand-probability pairs; a voting engine configured to: determining that a first brand-probability pair of a set of candidate brand-probability pairs based on at least one obtained hyperparameter value does not meet a threshold for determining whether candidate brand-probability pairs are to be included in a result set; excluding the first brand-probability pair from the result set based on the determination; sorting the result set; and selecting at least one final brand-probability pair from the result set; and an offline transcoding service configured to: store the final brand-probability pair in a repository with a relation to an identifier of the media item.

1 . A system for entity detection using artificial intelligence, comprising:
 a computer processor;   a deep learning model service configured to analyze a plurality of frames of a media item to generate a set of candidate entity-probability pairs, each candidate entity-probability pair comprising an entity identifier and a probability score representing a likelihood of an entity being associated with the media item;   a voting engine executing on the computer processor and configured to:
 compare the probability score of each candidate entity-probability pair against a score threshold to identify valid detections, the valid detections comprising candidate entity-probability pairs having probability scores meeting or exceeding the score threshold; 
 calculate, for each unique entity identifier among the valid detections, a count representing a number of frames of the media item in which the unique entity identifier was validly detected; 
 compare the count for each unique entity identifier against a count threshold; 
 exclude, from a result set, entity identifiers having counts failing to meet the count threshold; 
 sort the result set based on the counts; and 
 select at least one final entity identifier from the sorted result set; 
   an offline transcoding service configured to store, in a transcoding repository, an association between the at least one final entity identifier and an identifier of the media item; and   a frequency management service configured to manage a frequency of serving the media item to one or more recipients based on the association stored in the transcoding repository.   
     
     
         2 . The system of  claim 1 , wherein the score threshold and the count threshold comprise tuned hyperparameter values, and wherein the tuned hyperparameter values are optimized for the deep learning model service based on precision and recall metrics calculated using validation data. 
     
     
         3 . The system of  claim 1 , wherein the voting engine is further configured to:
 identify a common characteristic among the valid detections, the common characteristic comprising one selected from a group consisting of an industry designation and a category designation; and   exclude, from the result set, entity identifiers not associated with the common characteristic.   
     
     
         4 . The system of  claim 1 , wherein the frequency management service is further configured to:
 store, in a lookup repository, a quantity of impressions associated with the at least one final entity identifier for each of the one or more recipients;   receive a request for media content for a recipient of the one or more recipients;   determine that the quantity of impressions associated with the at least one final entity identifier and the recipient over a duration of time exceeds a predefined frequency threshold; and   exclude the media item from fulfilling the request based on the determination.   
     
     
         5 . The system of  claim 4 , wherein the frequency management service is operatively connected to a realtime bidding service, and wherein the frequency management service is further configured to:
 receive, from the realtime bidding service, a set of candidate media items for the request; and   exclude the media item from the set of candidate media items based on the quantity of impressions exceeding the predefined frequency threshold.   
     
     
         6 . The system of  claim 1 , wherein the deep learning model service is further configured to select the plurality of frames according to a predefined selection procedure, the predefined selection procedure comprising:
 calculating delta values representing differences between consecutive frames of the media item; and   selecting frames from segments of the media item having delta values below a predefined movement threshold.   
     
     
         7 . The system of  claim 1 , further comprising an online transcoding service configured to:
 calculate a fingerprint of the media item using a deterministic algorithm;   query the transcoding repository with the fingerprint; and   determine, based on a response to the query, whether the media item has been previously processed by the offline transcoding service.   
     
     
         8 . The system of  claim 7 , wherein the online transcoding service is further configured to:
 determine that the media item is not tracked in the transcoding repository; and   queue the media item for processing by the offline transcoding service.   
     
     
         9 . The system of  claim 1 , wherein the offline transcoding service is further configured to store, in the transcoding repository, a plurality of entity-probability pairs in association with the identifier of the media item, each of the plurality of entity-probability pairs comprising a respective entity identifier from the result set and a corresponding probability score. 
     
     
         10 . A method for entity detection using artificial intelligence, comprising:
 analyzing, using a trained model executing on at least one computer processor, a plurality of frames of a media item to generate a set of candidate entity-probability pairs, each candidate entity-probability pair comprising an entity identifier and a probability score representing a likelihood of an entity being associated with the media item;   comparing the probability score of each candidate entity-probability pair against a score threshold to identify valid detections, the valid detections comprising candidate entity-probability pairs having probability scores meeting or exceeding the score threshold;   calculating, for each unique entity identifier among the valid detections, a count representing a number of frames of the media item in which the unique entity identifier was validly detected;   comparing the count for each unique entity identifier against a count threshold;   excluding, from a result set, entity identifiers having counts failing to meet the count threshold;   sorting the result set based on the counts;   selecting at least one final entity identifier from the sorted result set;   storing, in a transcoding repository, an association between the at least one final entity identifier and an identifier of the media item; and   managing a frequency of serving the media item to one or more recipients based on the association stored in the transcoding repository.   
     
     
         11 . The method of  claim 10 , wherein the score threshold and the count threshold comprise tuned hyperparameter values, and wherein the method further comprises:
 executing a hyperparameter tuning process using validation data to identify the tuned hyperparameter values based on precision and recall metrics.   
     
     
         12 . The method of  claim 10 , further comprising:
 identifying a common characteristic among the valid detections; and   excluding, from the result set, entity identifiers not associated with the common characteristic.   
     
     
         13 . The method of  claim 10 , further comprising:
 storing, in a lookup repository, a quantity of impressions associated with the at least one final entity identifier for each of the one or more recipients;   receiving a request for media content for a recipient of the one or more recipients;   determining that the quantity of impressions associated with the at least one final entity identifier and the recipient over a duration of time exceeds a predefined frequency threshold; and   excluding the media item from fulfilling the request based on the determination.   
     
     
         14 . The method of  claim 13 , further comprising:
 receiving, from a realtime bidding service, a set of candidate media items for the request; and   excluding the media item from the set of candidate media items based on the quantity of impressions exceeding the predefined frequency threshold.   
     
     
         15 . The method of  claim 10 , further comprising selecting the plurality of frames according to a predefined selection procedure, the predefined selection procedure comprising:
 calculating delta values representing differences between consecutive frames of the media item; and   selecting frames from segments of the media item having delta values below a predefined movement threshold.   
     
     
         16 . The method of  claim 10 , further comprising:
 calculating a fingerprint of the media item using a deterministic algorithm;   querying the transcoding repository with the fingerprint; and   determining, based on a response to the query, whether the media item has been previously processed.   
     
     
         17 . The method of  claim 16 , further comprising:
 determining that the media item is not tracked in the transcoding repository; and   queueing the media item for processing by an offline transcoding service.   
     
     
         18 . The method of  claim 10 , wherein the trained model is trained using augmented training data, and wherein generating the augmented training data comprises:
 identifying an image asset corresponding to an entity;   selecting a subset of frames of a training video;   performing an augmentation technique on the image asset based on characteristics of the subset of frames; and   overlaying the augmented image asset onto the subset of frames to generate the augmented training data.   
     
     
         19 . The method of  claim 10 , wherein the one or more recipients are identified by a recipient identifier comprising one selected from a group consisting of a unique device identifier, an Internet Protocol address, a media access control address, an International Mobile Equipment Identity, and a household identifier. 
     
     
         20 . A non-transitory computer-readable storage medium comprising a plurality of instructions for entity detection using artificial intelligence, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:
 analyze a plurality of frames of a media item to generate a set of candidate entity-probability pairs, each candidate entity-probability pair comprising an entity identifier and a probability score representing a likelihood of an entity being associated with the media item;   compare the probability score of each candidate entity-probability pair against a score threshold to identify valid detections, the valid detections comprising candidate entity-probability pairs having probability scores meeting or exceeding the score threshold;   calculate, for each unique entity identifier among the valid detections, a count representing a number of frames of the media item in which the unique entity identifier was validly detected;   compare the count for each unique entity identifier against a count threshold;   exclude, from a result set, entity identifiers having counts failing to meet the count threshold;   sort the result set based on the counts;   select at least one final entity identifier from the sorted result set;   store, in a transcoding repository, an association between the at least one final entity identifier and an identifier of the media item, wherein the association is provided to a frequency management service for managing a frequency of serving the media item to one or more recipients; and   provide the association to the frequency management service.