Predicting future performance of multiple workers on crowdsourcing tasks and selecting repeated crowdsourcing workers

Abstract

Systems and methods of a job distribution platform for aggregating performance data in a worker profile for workers in performing crowd sourced tasks are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, of generating and storing performance data when a worker performs one or more tasks distributed online to their respective computing devices by a job distribution platform which crowd sources tasks over a network to remote workers. The tasks can span current jobs and a history of previous jobs distributed to the worker and the job performance data for the worker is collected for current and previous jobs. New jobs can be assigned to a worker selected based on performance data of the worker. Other types of worker information may also be tracked and generated for multiple workers in various worker pools (channels), for example, psych questions or other more subjective data collected from a worker or contributor in addition to performance.

What is claimed is: 
     
         1 . A method to generate profile data for a worker in performing crowd sourced tasks, the method, comprising:
 aggregating, by a job distribution platform which crowd sources tasks to workers, job performance data over a period of time for the worker to generate the profile data;   the job performance data being collected when the worker performs one or more tasks distributed online to their respective computing devices by the job distribution platform.   
     
     
         2 . The method of  claim 1 , wherein, the job performance data is aggregated and stored for multiple workers for future use by the job distribution platform in selecting a suitable worker for a given task. 
     
     
         3 . The method of  claim 1 , wherein, the one or more tasks span current jobs and a history of previous jobs distributed to the worker and the job performance data for the worker is collected for current and previous jobs. 
     
     
         4 . The method of  claim 3 , further comprising, tracking the history of previous jobs having the one or more tasks the worker has worked on. 
     
     
         5 . The method of  claim 1 , wherein, the job performance data includes quantitative metrics indicating rate or number of rejected results in performing the one or more tasks. 
     
     
         6 . The method of  claim 1 , wherein, the job performance data includes quantitative metrics indicating a rate or number of instances of bans from the one or more tasks. 
     
     
         7 . The method of  claim 1 , wherein, the job performance data includes quantitative metrics indicating a rate or number of instances of the worker has been flagged for performance review. 
     
     
         8 . The method of  claim 1 , further comprising, tracking a rate or time with which the worker completes the one or more tasks. 
     
     
         9 . The method of  claim 8 , further comprising, tracking instances when the worker performs or completes a task exceeding a rate or speed. 
     
     
         10 . The method of  claim 1 , wherein, the job performance data includes quantitative metrics indicating a rate or number when the worker has performed or completed one or more tasks exceeding a rate or speed. 
     
     
         11 . The method of  claim 1 , wherein, the job performance data includes quantitative metrics indicating an average rate or speed with which the worker can satisfactorily complete a task. 
     
     
         12 . The method of  claim 1 , further comprising:
 tracking the job performance of the worker for a given job session;   wherein the job performance data includes metrics tracked over the given job session.   
     
     
         13 . The method of  claim 1 , further comprising:
 determining financial value of work generated by the worker;   wherein the job performance data includes quantitative metrics indicating the financial value of the work performed by the worker.   
     
     
         14 . The method of  claim 1 , further comprising:
 determining accuracy of work generated by the worker using test jobs;   wherein the job performance data includes quantitative metrics indicating the accuracy of the work performed by the worker.   
     
     
         15 . The method of  claim 1 , further comprising:
 determining an experience level of the worker based on duration or tenure with the job distribution platform;   wherein the job performance data includes quantitative metrics indicating the experience level of the worker.   
     
     
         16 . The method of  claim 1 , further comprising:
 determining and tracking, one or more of, satisfaction and frustration level with the job distribution platform;   wherein the profile data includes quantitative metrics indicating the satisfaction or frustration levels.   
     
     
         17 . The method of  claim 1 , wherein, the profile data further includes demographic data of the worker. 
     
     
         18 . The method of  claim 1 , wherein, the profile data further includes identification of devices used by the worker to perform tasks distributed by the job distribution platform. 
     
     
         19 . The method of  claim 1 , wherein, the profile data further includes preferences of the worker specified by the worker, the preferences of the worker indicate willingness to perform tasks involving objectionable content or adult content. 
     
     
         20 . The method of  claim 1 , wherein, the profile data further includes preferences of the worker specified by the worker; wherein, the preferences of the worker indicate preferred time of day to perform tasks. 
     
     
         21 . The method of  claim 1 , wherein, the profile data of the worker include compensation preferences which specify preferred payment mechanism including, one or more of, direct deposit, indirect deposit, or mobile credits. 
     
     
         22 . The method of  claim 21 , wherein, the compensation preferences further include, one or more of, preferred wage, preferred currency, and preferred type of currency. 
     
     
         23 . The method of  claim 1 , wherein, the tasks are crowd sourced to workers via multiple channels; wherein, the worker has performed the one or more tasks through multiple channels and is tracked by a unique identifier. 
     
     
         24 . A machine-readable storage medium having stored thereon a set of instructions which when executed causes a processor to perform a method of identifying a suitable worker in a crowd sourced environment, the method, comprising:
 generating and storing performance data when a worker performs one or more tasks distributed online to their respective computing devices by a job distribution platform which crowd sources tasks over a network to remote workers;   wherein, the one or more tasks span current jobs and a history of previous jobs distributed to the worker and the job performance data for the worker is collected for current and previous jobs;   assigning a new job to the worker, wherein the worker is selected based on the performance data of the worker.   
     
     
         25 . The machine-readable storage medium of  claim 24 , further comprising, aggregating profile information for the worker from third party social networks; wherein the profile information includes one or more of, social profile data and professional profile data. 
     
     
         26 . The machine-readable storage medium of  claim 24 , further comprising, aggregating demographic profile information for the worker from input received from the worker. 
     
     
         27 . The machine-readable storage medium of  claim 24 , further comprising, tracking employment status information for the worker. 
     
     
         28 . The machine-readable storage medium of  claim 24 , wherein, the worker is selected to perform the new job based on worker preferences indicating willingness or unwillingness to work on objectionable or controversial content. 
     
     
         29 . The machine-readable storage medium of  claim 24 , further comprising, tracking instances when the worker performs or completes a task exceeding a rate or speed; wherein, the job performance data includes quantitative metrics indicating a rate or number when the worker has performed or completed one or more tasks exceeding a rate or speed. 
     
     
         30 . A method to track performance data for repeat workers over time in performing crowd sourced tasks, the method, comprising:
 aggregating, by a job distribution server which crowd sources tasks to the multiple workers, performance data for the repeat workers;   generating the performance data when the repeat workers perform a task distributed to them by the job distribution server;   wherein, the repeat workers have performed tasks associated with at least two jobs distributed by the job distribution server;   wherein, the job performance data is collected and stored for the repeat workers for future use by the job distribution server in selecting a suitable worker from the repeat workers for a given task.   
     
     
         31 . The method of  claim 30 , wherein, the job performance data is aggregated and stored for multiple workers for future use by the job distribution platform in selecting a suitable worker for a given task. 
     
     
         32 . The method of  claim 30 , further comprising,
 using a test job to compare results generated by multiple workers;   determining accuracy of work generated by a worker based on a comparison of the results to the test job;   wherein the job performance data includes quantitative metrics indicating the accuracy of the work performed by the worker based on the comparison of a result generated by the worker to the results generated by other workers.   
     
     
         33 . A system for tracking performance data for repeat workers over time in performing crowd sourced tasks, the method, comprising:
 means for, aggregating performance data for the repeat workers;   means for, generating the performance data when the repeat workers perform a task distributed to them;   wherein, the repeat workers have performed tasks associated with at least two jobs;   means for, collecting and storing the job performance data for the repeat workers for future use by the job distribution server in selecting a suitable worker from the repeat workers for a given task;   means for, using the job performance data for future use in selecting a suitable worker for a given task.