Computer-implemented methods and systems for clustering user reviews and ranking clusters

Abstract

Computer-implemented methods and systems are disclosed for organizing user reviews, especially computer app reviews, into clusters and ranking the clusters so that the reviews may be more meaningfully analyzed.

What is claimed is: 
     
         1 . A method of organizing user reviews for data analysis, comprising the steps, each implemented in a computer system, of:
 identifying unique terms in a plurality of reviews from a sample set of reviews;   determining frequency values indicating a number of times the unique terms appears in the plurality of reviews, wherein determining the frequency values includes generating a term/review matrix identifying a respective review and a respective unique term, and wherein the matrix specifies the frequency values for the respective unique terms;   adjusting the frequency values for respective unique terms to account for the rarity of that unique term across the plurality of reviews, wherein adjusting the frequency values includes normalizing the values in the matrix based on values associated with the rarity of the unique term;   calculating similarities between the reviews based at least in part on the frequency values once adjusted; and   grouping the reviews into clusters based at least in part on the similarities of the reviews.   
     
     
         2 . The method of  claim 1 , wherein identifying includes tokenizing text in the plurality of reviews into a set of words and stemming for words not already in root form to generate a set of terms for each review, from which said unique terms are identified. 
     
     
         3 . The method of  claim 1 , wherein adjusting includes calculating tf/idf values for each review, wherein tf=the raw frequency of a term within a review, and idf=log (total number of reviews/total number of reviews containing the term), and wherein the tf/idf values comprise the frequency values adjusted. 
     
     
         4 . The method of  claim 3 , further comprising normalizing the term/review matrix. 
     
     
         5 . The method of  claim 1 , wherein calculating and grouping includes dividing the reviews into a given number of chunks, calculating the similarity of each review against every other review in each chunk, and for each pair of reviews whose similarity exceeds a given threshold, clustering said reviews. 
     
     
         6 . The method of  claim 5 , further comprising using the transitive property for clustering reviews. 
     
     
         7 . The method of  claim 5 , further comprising merging clusters with similarly sufficient reviews. 
     
     
         8 . The method of  claim 7 , wherein the clusters are merged based on cluster centroids within each chunk. 
     
     
         9 . The method of  claim 5 , further comprising performing a post merge process to collapse clusters across chunks. 
     
     
         10 . The method of  claim 1 , further comprising ranking the clusters. 
     
     
         11 . The method of  claim 10 , wherein the clusters are ranked based on a plurality of factors. 
     
     
         12 . A computer system, comprising:
 at least one processor;   memory associated with the at least one processor; and   a program supported in the memory for organizing user reviews for data analysis, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to:   identify unique terms in a plurality of reviews from a sample set of reviews;   determine frequency values indicating a number of times the unique terms appears in the plurality of reviews, wherein determining the frequency values includes generating a term/review matrix identifying a respective review and a respective unique term, and wherein the matrix specifies the frequency values for the respective unique terms;   adjust the frequency values for respective unique terms to account for the rarity of that unique term across the plurality of reviews, wherein adjusting the frequency values includes normalizing the values in the matrix based on values associated with the rarity of the unique term;   calculate similarities between the reviews based at least in part on the frequency values once adjusted; and   group the reviews into clusters based at least in part on the similarities of the reviews.   
     
     
         13 . The computer system of  claim 12 , wherein identifying includes tokenizing text in the plurality of reviews into a set of words and stemming each word not already in root form to generate a set of terms for each review, from which said unique terms are identified. 
     
     
         14 . The computer system of  claim 12 , wherein adjusting includes calculating tf/idf values for each review, wherein tf =the raw frequency of a term within a review, and idf=log (total number of reviews/total number of reviews containing the term), and wherein the tf/idf values comprise the frequency values adjusted in (d). 
     
     
         15 . The computer system of  claim 14 , wherein the program further comprises instructions for normalizing the term/review matrix. 
     
     
         16 . The computer system of  claim 12 , calculating and grouping includes dividing the reviews into a given number of chunks, calculating the similarity of each review against every other review in each chunk, and for each pair of reviews whose similarity exceeds a given threshold, clustering said reviews. 
     
     
         17 . The computer system of  claim 16 , wherein the program further comprises instructions for using the transitive property for clustering reviews. 
     
     
         18 . The computer system of  claim 16 , wherein the program further comprises instructions for merging clusters with similarly sufficient reviews. 
     
     
         19 . The computer system of  claim 18 , wherein the clusters are merged based on cluster centroids within each chunk. 
     
     
         20 . The computer system of  claim 19 , wherein the program further comprises instructions for performing a post merge process to collapse clusters across chunks.