Integration and combination of random sampling and document batching

Abstract

Methods and systems of integrated batching and random sampling of documents for enhanced functionality and quality control, such as validation, within a document review process are provided herein. According to various embodiments, a batching request may be received and may include a population size that corresponds to a total amount of documents available for sampling. The batching request may also include an acceptable margin of error. A random sample size may be calculated based on the batching request, and then a subset of documents corresponding to the random sample size may be selected from the total amount of documents available for sampling. The subset of documents may be grouped into one or more batches, and the one or more batches may be assigned to one or more review nodes.

What is claimed is: 
     
         1 . A method of integrated batching and random sampling of documents for enhanced functionality within document review processes, comprising:
 receiving a batching request, the batching request including:
 a population size that corresponds to a total amount of documents available for sampling; and 
 an acceptable margin of error; 
   computing a random sample size from the batching request;   randomly selecting a subset of documents from the total amount of documents available for sampling, the subset of documents corresponding to the random sample size;   grouping the subset of documents into one or more batches; and   assigning the one or more batches to one or more review nodes.   
     
     
         2 . The method according to  claim 1 , wherein the batching request further includes a selected confidence level. 
     
     
         3 . The method according to  claim 1 , further comprising:
 receiving a statistical query regarding the total amount of documents;   applying a statistical hypothesis test to the subset of documents to calculate a first statistical response to the statistical query for the subset of documents; and   utilizing the first statistical response to calculate a second statistical response to the statistical query for the total amount of documents.   
     
     
         4 . The method according to  claim 1 , further comprising:
 applying a statistical hypothesis test, the statistical hypothesis test including:
 determining a range of excluded but relevant documents within the total amount of documents by determining a population of excluded but relevant documents within the subset of documents; 
 adding the margin of error to the population of excluded but relevant documents to create an upper range boundary and subtracting the margin of error to the population of excluded but relevant documents to create a lower range boundary; and 
 wherein the range of excluded but relevant documents within the total amount of documents extends between the lower range boundary and the upper range boundary, inclusive. 
   
     
     
         5 . The method according to  claim 4 , further comprising:
 determining a range of excluded but relevant documents for both a subset of machine reviewed documents and a subset of human reviewed documents;   comparing the ranges together to determine a difference between machine reviewed documents and human reviewed documents, the difference being expressed as a percentage; and   utilizing machine document review if the difference is less than a threshold amount.   
     
     
         6 . A system of integrated batching and random sampling of documents for enhanced functionality within document review processes, the system comprising:
 a memory for storing executable instructions for batching and random sampling of documents for quality control within document review processes;   a processor for executing the instructions stored in memory, the executable instructions comprising:
 a query module that receives a batching request, the batching request including a population size that corresponds to a total amount of documents available for sampling and an acceptable margin of error; 
 an analysis module communicatively coupled to the query module that computes a random sample size from the batching request and randomly selects a subset of documents from the total amount of documents available for sampling, the subset of documents corresponding to the random sample size; and 
 a batching module communicatively coupled to the analysis module that groups the subset of documents into one or more batches, and assigns the one or more batches to one or more review nodes. 
   
     
     
         7 . The system according to  claim 6 , further comprising a communications module communicatively coupled to the batching module and communicatively coupleable to one or more review nodes, the communications module transmits the one or more batches to the one or more review nodes. 
     
     
         8 . The system according to  claim 6 , further comprising a statistical evaluation module that applies a statistical hypothesis test to the subset of documents to calculate a first statistical response to the statistical query for the subset of documents; and utilizes the first statistical response to calculate a second statistical response to the statistical query for the total amount of documents available for sampling, the statistical evaluation module being communicatively coupled to the query module and the analysis module.