Automatic techniques for constructing an evolving interest taxonomy from user-generated content

Abstract

Techniques for creating an interest graph include obtaining content items from multiple content sources and applying tailored (e.g., source-specific) preprocessing to the content items based on their respective content source. Text is extracted and salient keywords and key phrases are identified using unsupervised machine learning models. The keywords and key phrases become nodes in an interest graph, each node comprising an embedding of a keyword or key phrase in a common embedding space, with edges representing semantic similarity based on embeddings or co-engagement patterns. The graph provides an expansive, granular, and dynamic taxonomy easily adaptable to emerging interests. The interest graph overcomes limitations of conventional taxonomies that lack depth, fail to capture niche interests, and cannot adapt to reflect evolving user preferences. The described techniques construct a rich interest graph from diverse content for improved content understanding.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 obtaining a plurality of content items from multiple content sources of an online platform;   extracting text from the plurality of content items;   identifying keywords and key phrases from the extracted text;   constructing a graph structure having nodes representing the keywords and key phrases and edges connecting nodes based on co-engagement patterns determined from user behavior, wherein the co-engagement patterns are identified by detecting when users interact with content items containing different keywords or key phrases; and   using the graph structure to facilitate content understanding for the online platform.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein obtaining the plurality of content items comprises applying distinct preprocessing steps to content items from different content sources, wherein the distinct preprocessing steps are specific to an expected format of each content source. 
     
     
         3 . The computer-implemented method of  claim 2 , wherein the distinct preprocessing steps applied to content items from a first content source comprise:
 identifying one or more hashtags in the content items from the first content source;   separating each hashtag into individual words; and   adding the separated words to the extracted text.   
     
     
         4 . The computer-implemented method of  claim 1 , further comprising:
 for each content item that includes a video clip, processing the video clip using speech-to-text transcription to derive a text-based transcript;   providing the derived text-based transcript as input to a machine learning model that outputs additional keywords and key phrases; and   adding the additional keywords and key phrases to the extracted text.   
     
     
         5 . The computer-implemented method of  claim 1 , wherein constructing the graph structure comprises:
 providing the keywords and key phrases as input to an embedding model;   wherein the embedding model outputs vector representations for the keywords and key phrases in a common embedding space such that keywords and key phrases with similar co-engagement patterns have vector representations that are close in distance to one another.   
     
     
         6 . The computer-implemented method of  claim 1 , further comprising:
 preparing a training dataset by pairing image and video content items with relevant keywords or key phrases from the graph structure; and   training a machine learning model on the training dataset to learn associations between visual content and textual keywords based on the co-engagement patterns.   
     
     
         7 . The computer-implemented method of  claim 1 , wherein obtaining the plurality of content items comprises retrieving previously posted content items from a database, wherein the previously posted content items were submitted to the online platform by users of the online platform. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein detecting when users interact with content items containing different keywords or key phrases comprises:
 monitoring user engagement activities including at least one of viewing, sharing, commenting on, or liking content items on the online platform;   tracking temporal patterns of user interactions with multiple content items within a predetermined time period; and   determining co-engagement strength based on frequency of users engaging with content items containing related keywords or key phrases, wherein higher frequency of co-engagement results in stronger similarity relationships in the graph structure.   
     
     
         9 . The computer-implemented method of  claim 1 , wherein the co-engagement patterns determined from user behavior further comprise:
 identifying cross-content engagement sequences where individual users interact with multiple content items in succession;   analyzing user session data to detect when users engage with content items containing semantically different but behaviorally related keywords or key phrases; and   weighting the co-engagement patterns based on user engagement intensity, wherein engagement intensity is determined by at least one of duration of interaction, type of interaction, or user return patterns to similar content.   
     
     
         10 . A system comprising:
 one or more processors; and   one or more memory storage devices storing instructions thereon, which, when executed by the one or more processors, cause the system to perform operations comprising:   obtaining a plurality of content items from multiple content sources of an online platform;   extracting text from the plurality of content items;   identifying keywords and key phrases from the extracted text;   constructing a graph structure having nodes representing the keywords and key phrases and edges connecting nodes based on co-engagement patterns determined from user behavior, wherein the co-engagement patterns are identified by detecting when users interact with content items containing different keywords or key phrases; and   using the graph structure to facilitate content understanding for the online platform.   
     
     
         11 . The system of  claim 10 , wherein obtaining the plurality of content items comprises applying distinct preprocessing steps to content items from different content sources, wherein the distinct preprocessing steps are specific to an expected format of each content source. 
     
     
         12 . The system of  claim 11 , wherein the distinct preprocessing steps applied to content items from a first content source comprise:
 identifying one or more hashtags in the content items from the first content source;   separating each hashtag into individual words; and   adding the separated words to the extracted text.   
     
     
         13 . The system of  claim 10 , further comprising:
 for each content item that includes a video clip, processing the video clip using speech-to-text transcription to derive a text-based transcript;   providing the derived text-based transcript as input to a machine learning model that outputs additional keywords and key phrases; and   adding the additional keywords and key phrases to the extracted text.   
     
     
         14 . The system of  claim 10 , wherein constructing the graph structure comprises:
 providing the keywords and key phrases as input to an embedding model;   wherein the embedding model outputs vector representations for the keywords and key phrases in a common embedding space such that keywords and key phrases with similar co-engagement patterns have vector representations that are close in distance to one another.   
     
     
         15 . The system of  claim 10 , further comprising:
 preparing a training dataset by pairing image and video content items with relevant keywords or key phrases from the graph structure; and   training a machine learning model on the training dataset to learn associations between visual content and textual keywords based on the co-engagement patterns.   
     
     
         16 . The system of  claim 10 , wherein obtaining the plurality of content items comprises retrieving previously posted content items from a database, wherein the previously posted content items were submitted to the online platform by users of the online platform. 
     
     
         17 . The system of  claim 10 , wherein detecting when users interact with content items containing different keywords or key phrases comprises:
 monitoring user engagement activities including at least one of viewing, sharing, commenting on, or liking content items on the online platform;   tracking temporal patterns of user interactions with multiple content items within a predetermined time period; and   determining co-engagement strength based on frequency of users engaging with content items containing related keywords or key phrases, wherein higher frequency of co-engagement results in stronger similarity relationships in the graph structure.   
     
     
         18 . The system of  claim 10 , wherein the co-engagement patterns determined from user behavior further comprise:
 identifying cross-content engagement sequences where individual users interact with multiple content items in succession;   analyzing user session data to detect when users engage with content items containing semantically different but behaviorally related keywords or key phrases; and   weighting the co-engagement patterns based on user engagement intensity, wherein engagement intensity is determined by at least one of duration of interaction, type of interaction, or user return patterns to similar content.   
     
     
         19 . A computer-readable medium storing instructions thereon, when executed by one or more processors, cause the one or more processors to perform operations comprising:
 obtaining a plurality of content items from multiple content sources of an online platform;   extracting text from the plurality of content items;   identifying keywords and key phrases from the extracted text;   constructing a graph structure having nodes representing the keywords and key phrases and edges connecting nodes based on co-engagement patterns determined from user behavior, wherein the co-engagement patterns are identified by detecting when users interact with content items containing different keywords or key phrases; and   using the graph structure to facilitate content understanding for the online platform.   
     
     
         20 . The computer-readable medium of  claim 19 , wherein detecting when users interact with content items containing different keywords or key phrases comprises:
 monitoring user engagement activities including at least one of viewing, sharing, commenting on, or liking content items on the online platform;   tracking temporal patterns of user interactions with multiple content items within a predetermined time period; and   determining co-engagement strength based on frequency of users engaging with content items containing related keywords or key phrases, wherein higher frequency of co-engagement results in stronger similarity relationships in the graph structure.