Full-service research bureau and test center method and apparatus

Abstract

A centralized full-service research bureau and test center method and apparatus is provided, which assists a customer, typically a Web product manager, in designing a user experience and marketing effectiveness test for typical tasks faced by a visitor to the customer's site. The centralized full-service research bureau and test center method and apparatus provides proprietary software to administer the user experience and marketing effectiveness test to a pre-qualified pool of testers meeting desired demographic constraints. The user experience and marketing effectiveness test measures not only the visitor's success in performing tasks, but also prompts a tester for context-specific feedback ranging from aesthetics to behavior. Statistics are aggregated across the testing population and presented not just as data, but with recommended actions backed up by analysis.

1 . A method of testing the user experience and marketing effectiveness of a World Wide Web site, comprising the steps of: 
 providing a first proprietary software, said first proprietary software comprising a user experience and marketing effectiveness test comprising a set of tasks to be performed by a tester in connection with World Wide Web sites that are unmodified, and which may be independently selected by the tester while completing said task;    said tester performing actions necessary to perform said tasks in a manner chosen by the said tester in connection with said tester selected World Wide Web sites;    providing a second proprietary software to administer said user experience and marketing effectiveness test to a plurality of testers qualified for inclusion in a testing sample, wherein said qualification is based on any of: traits, behaviors, and opinions; and    providing a World Wide Web based architecture, wherein said first proprietary software comprises a modified browser and/or software working in conjunction with a browser.    
     
     
         2 . The method of  claim 1 , further comprising the step of: 
 providing a storing and retrieval means to store and retrieve data resulting from said administered user experience and marketing effectiveness test whereby said stored data are aggregatable across a plurality of qualified testers and usable in a future analysis associated with said set of tasks.    
     
     
         3 . The full-service research bureau and test center method of  claim 1 , further comprising the step of: 
 providing a World Wide Web based architecture; and    wherein said first proprietary software comprises a modified browser.    
     
     
         4 . The full-service research bureau and test center method of  claim 1 , wherein said first proprietary software is downloaded and installed on a machine to which said qualified tester has access.  
     
     
         5 . The full-service research bureau and test center method of  claim 1 , wherein said first proprietary software is: 
 adapted to collect a plurality of browser events and properties of the machine comprising one or more of: 
 screen resolution;  
 available memory;  
 installed software and versions; and  
 identifying indicia of hardware components; and  
   adapted to transmit said browser events to said second proprietary software.    
     
     
         6 . The full-service research bureau and test center method of  claim 5 , wherein said plurality of browser events comprises any of: 
 a submission of a page request;    a submission of a form and its associated data;    a fulfillment of a page request;    use of a back, forward, home, or stop button;    access of a page from a history or bookmark list;    use of browser functions, including any of adding a bookmark searching for a text string, or viewing HTML source;    a page-scrolling event; and    a mouse-over event.    
     
     
         7 . The full-service research bureau and test center method of  claim 5 , wherein said transmitted browser events are in a log form.  
     
     
         8 . The full-service research bureau and test center method of  claim 1 , wherein said first proprietary software is adapted to pose a question to said qualified tester, said question associated with a browser event from said plurality of browser events, and to transmit a response to said question to said second proprietary software.  
     
     
         9 . The full-service research bureau and test center method of  claim 1 , wherein said first proprietary software is adapted to pose a question to said qualified tester, said question associated with a message from said second proprietary software and to transmit a response to said message to said second proprietary software.  
     
     
         10 . The full-service research bureau and test center method of  claim 1 , wherein first proprietary software comprises: 
 a plurality of additional buttons, any of said additional buttons used by said qualified tester to provide comments or interact with either of a human or automated test moderator; and    wherein said first proprietary software is adapted to transmit said comments to said second proprietary software.    
     
     
         11 . The full-service research bureau and test center method of  claim 1 , further comprising the step of: 
 loading a World Wide Web page directly from a host server, and wherein said first proprietary software is adapted to display said World Wide Web page unmodified, conveying an appearance and a latency as displayed by an unmodified browser.    
     
     
         12 . The full-service research bureau and test center method of  claim 1 , wherein said second proprietary software is a server application, said server application adapted to support a plurality of user experience and marketing effectiveness test sessions, any of said sessions used simultaneously by any of said plurality of qualified testers.  
     
     
         13 . The full-service research bureau and test center method of  claim 1 , wherein said user experience and marketing effectiveness test comprises a series of questions, said series of questions representing all or a part of a set of customer objectives.  
     
     
         14 . The full-service research bureau and test center method of  claim 12 , further comprising the step of: 
 providing an additional question associated with a specific answer to one of said series of questions or associated with particular actions taken during the user experience and marketing effectiveness test session.    
     
     
         15 . The full-service research bureau and test center method of  claim 2 , wherein said storing and retrieving means further comprises the step of: 
 providing a data warehouse comprising a plurality of entries, wherein each entry of said plurality of entries is tagged with a test sequence indicator and a qualified tester identifier, such that said storing and retrieval means is responsive to a complex query.    
     
     
         16 . The full-service research bureau and test center method of  claim 15 , further comprising the step of: 
 providing a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of demographic properties.    
     
     
         17 . The full-service research bureau and test center method of  claim 15 , further comprising the step of: 
 providing a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of test-specific properties.    
     
     
         18 . The full-service research bureau and test center method of  claim 2 , further comprising any of the steps of: 
 providing a test building means, which said customer uses to create and initiate said user experience and marketing effectiveness test;    providing a reporter means, which said customer uses to submit a set of queries to said storing and retrieval means and, thereby to obtain an ad hoc report;    providing a report builder, whereby said report builder is adapted to process all or a part of said stored data to provide analysis of said all or a part of said stored data;    providing a World Wide Web site, whereby said World Wide Web site is adapted to facilitate communication between said customers and said testers; and    providing a test auditor, whereby a predetermined standard of quality of said data is assured.    
     
     
         19 . The full-service research bureau and test center method of  claim 18 , wherein said test building means is automatic; and further comprising the step of: 
 providing a specifying means, whereby said customer can specify any of: 
 a predetermined number of said plurality of testers;  
 a desired demographic cross-section;  
 a set of customer objectives;  
 any of sites and pages to be visited; and  
 other questions.  
   
     
     
         20 . The full-service research bureau and test center method of  claim 18 , wherein said reporter means comprises a World Wide Web application.  
     
     
         21 . The full-service research bureau and test center method of  claim 18 , wherein said reporter means provides interactive interpretive activities.  
     
     
         22 . The full-service research bureau and test center method of  claim 18 , wherein said report builder is adapted to assist in highlighting substantially relevant quotes and statistics.  
     
     
         23 . The full-service research bureau and test center method of  claim 18 , wherein said report builder is adapted to provide an automated personalized report.  
     
     
         24 . The full-service research bureau and test center method of  claim 18 , further comprising any of the steps of: 
 providing a customer educating means on said World Wide Web site for said customer to learn about benefits of said user experience and marketing effectiveness test;    providing a tester educating means on said World Wide Web site for said tester to learn about said benefits of said user experience and marketing effectiveness test;    providing an enrolling means for said tester on said World Wide Web site;    providing an access means for said customer on World Wide Web site to access any of: 
 a pre-generated report;  
 said ad hoc report; and  
 said test builder means;  
   creating a plurality of communities on said World Wide Web site for all or a portion of said plurality of testers and a plurality of customers; and    providing a means for managing a plurality of electronic commercial (e-commerce) transactions.    
     
     
         25 . The full-service research bureau and test center method of  claim 18 , further comprising the step of: 
 coupling said test auditor to said second proprietary software, and wherein said test auditor is adapted to measure an amount of effort spent by said tester on a task of said user experience and marketing effectiveness test and an amount of time spent by said tester on said task of said user experience and marketing effectiveness test.    
     
     
         26 . The full-service research bureau and test center method of  claim 18 , wherein said test auditor provides a bonus to said tester, said tester having provided a substantially high quality response.  
     
     
         27 . An apparatus for testing the user experience and marketing effectiveness of a World Wide Web site, comprising: 
 a first proprietary software, said first proprietary software comprising a user experience and marketing effectiveness test; comprising a set of tasks to be performed by a tester in connection with World Wide Web sites that are unmodified, and which may be independently selected by the tester while completing said task, where said tester performs actions necessary to perform said tasks in a manner chosen by said tester in connection with said tester selected World Wide Web sites;    a second proprietary software executed by a processor to administer said user experience and marketing effectiveness test to a plurality of testers qualified for inclusion in a testing sample, wherein said qualification is based on any of: traits, behaviors, and opinions; and    a World Wide Web based architecture, wherein said first proprietary software comprises a modified browser and/or software working in conjunction with a browser.    
     
     
         28 . The apparatus of  claim 27 , further comprising: 
 a storing and retrieval means to store and retrieve data resulting from said administered user experience and marketing effectiveness test whereby said stored data are aggregatable across a plurality of qualified testers and usable in a future analysis associated with said set of tasks.    
     
     
         29 . The full-service research bureau and test center apparatus of  claim 27 , further comprising: 
 a World Wide Web based architecture; and    wherein said first proprietary software comprises a modified browser.    
     
     
         30 . The full-service research bureau and test center apparatus of  claim 27 , wherein said first proprietary software is downloaded and installed on a personal machine of said qualified tester.  
     
     
         31 . The full-service research bureau and test center apparatus of  claim 28 , wherein said first proprietary software is: 
 adapted to collect a plurality of browser events and properties of the machine comprising one or more of: 
 screen resolution;  
 available memory;  
 installed software and versions; and  
 identifying indicia of hardware components; and  
   adapted to transmit said browser events to said second proprietary software.    
     
     
         32 . The full-service research bureau and test center apparatus of  claim 31 , wherein said plurality of browser events comprises any of: 
 a submission of a page request;    a submission of a form and its associated data;    a fulfillment of a page request;    use of a back, forward, home, or stop button;    access of a page from a history or bookmark list;    use of browser functions, including any of adding a bookmark, searching for a text string, or viewing HTML source;    a page-scrolling event; and    a mouse-over event.    
     
     
         33 . The full-service research bureau and test center apparatus of  claim 31 , wherein said transmitted browser events are in a log form.  
     
     
         34 . The full-service research bureau and test center apparatus of  claim 27 , wherein said first proprietary software is adapted to pose a question to said qualified tester, said question associated with a browser event from said plurality of browser events, and to transmit a response to said question to said second proprietary software.  
     
     
         35 . The full-service research bureau and test center apparatus of  claim 27 , wherein said first proprietary software is adapted to pose a question to said qualified tester, said question associated with a message from said second proprietary software and to transmit a response to said message to said second proprietary software.  
     
     
         36 . The full-service research bureau and test center apparatus of  claim 27 , wherein first proprietary software comprises: 
 a plurality of additional buttons, any of said additional buttons used by said qualified tester to provide comments or interact with either a human or automated test moderator; and    wherein said first proprietary software is adapted to transmit said comments to said second proprietary software.    
     
     
         37 . The full-service research bureau and test center apparatus of  claim 27 , further comprising: 
 a World Wide Web page, said World Wide Web page loaded directly from a host server, and wherein said first proprietary software is adapted to display said World Wide Web page unmodified, conveying an appearance and a latency as displayed by an unmodified browser.    
     
     
         38 . The full-service research bureau and test center apparatus of  claim 27 , wherein said second proprietary software is a server application, said server application adapted to support a plurality of user experience and marketing effectiveness test sessions, any of said sessions used simultaneously by any of said plurality of qualified testers.  
     
     
         39 . The full-service research bureau and test center apparatus of  claim 27 , wherein said user experience and marketing effectiveness test comprises a series of questions, said series of questions representing all or a part of a set of customer objectives.  
     
     
         40 . The full-service research bureau and test center apparatus of  claim 39 , further comprising: 
 an additional question associated with a specific answer to one of said series of questions or associated with particular actions taken during the user experience and marketing effectiveness test session.    
     
     
         41 . The full-service research bureau and test center apparatus of  claim 28 , wherein said storing and retrieving means further comprises: 
 a data warehouse comprising a plurality of entries, wherein each entry of said plurality of entries is tagged with a test sequence indicator and a qualified tester identifier, such that said storing and retrieval means is responsive to a complex query.    
     
     
         42 . The full-service research bureau and test center apparatus of  claim 41 , further comprising: 
 a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of demographic properties.    
     
     
         43 . The full-service research bureau and test center apparatus of  claim 41 , further comprising: 
 a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of test-specific properties.    
     
     
         44 . The full-service research bureau and test center apparatus of  claim 27 , further comprising any of: 
 a test building means, which said customer uses to create and initiate said user experience and marketing effectiveness test;    a reporter means, which said customer uses to submit a set of queries to said storing and retrieval means and, thereby to obtain an ad hoc report;    a report builder, whereby said report builder is adapted to process all or a part of said stored data to provide analysis of said all or a part of said stored data;    a World Wide Web site, whereby said World Wide Web site is adapted to facilitate communication between said customers and said testers; and    a test auditor, whereby a predetermined standard of quality of said data is assured.    
     
     
         45 . The full-service research bureau and test center apparatus of  claim 44 , wherein said test building means is automatic; and further comprising: 
 a specifying means, whereby said customer can specify any of: 
 a predetermined number of said plurality of testers;  
 a desired demographic cross-section;  
 a set of customer objectives;  
 any of sites and pages to be visited; and  
 other questions.  
   
     
     
         46 . The full-service research bureau and test center apparatus of  claim 44 , wherein said reporter means comprises a World Wide Web application.  
     
     
         47 . The full-service research bureau and test center apparatus of  claim 44 , wherein said reporter means provides interactive interpretive activities.  
     
     
         48 . The full-service research bureau and test center apparatus of  claim 44 , wherein said report builder is adapted to assist in highlighting substantially relevant quotes and statistics.  
     
     
         49 . The full-service research bureau and test center apparatus of  claim 44 , wherein said report builder is adapted to provide an automated personalized report.  
     
     
         50 . The full-service research bureau and test center apparatus of  claim 44 , further comprising any of: 
 a customer educating means on said World Wide Web site for said customer to learn about benefits of said user experience and marketing effectiveness test;    a tester educating means on said World Wide Web site for said tester to learn about said benefits of said user experience and marketing effectiveness test;    an enrolling means for said tester on said World Wide Web site;    an access means for said customer on World Wide Web site to access any of: 
 a pre-generated report;  
 said ad hoc report; and  
 said test builder means;  
   a plurality of communities on said World Wide Web site for all or a portion of said plurality of testers and a plurality of customers; and    a means for managing a plurality of electronic commercial (e-commerce) transactions.    
     
     
         51 . The full-service research bureau and test center apparatus of  claim 44 , wherein said test auditor is coupled to said second proprietary software, and wherein said test auditor is adapted to measure an amount of effort spent by said tester on a task of said user experience and marketing effectiveness test and an amount of time spent by said tester on said task of said user experience and marketing effectiveness test.  
     
     
         52 . The full-service research bureau and test center apparatus of  claim 44 , wherein said test auditor provides a bonus to said tester, said tester having provided a substantially high quality response.  
     
     
         53 . A method of testing the user experience and marketing effectiveness of a World Wide Web site, comprising the steps of: 
 providing any of, a modified browser, software working in conjunction with a browser, a plug-in, a browser helper object, an applet, and a proxy;    logging onto a full-service research bureau and test center;    designing a test script for a test site comprising a set of tasks to be performed by a tester in connection with World Wide Web sites that are unmodified, and which may be independently selected by the tester while completing said task, where said tester performs actions necessary to perform said tasks in a manner chosen by said tester in connection with said tester selected World Wide Web sites;    defining a composition of a desired test panel;    defining requirements for a test completion date;    authorizing a test run;    retrieving raw test result data;    analyzing raw test result data;    comparing analyzed test result data with at least a second analyzed test result data of a second test run by a second tester;    wherein said second tester is different from said tester;    retrieving an industry report on a plurality of comparative test runs across an industry segment; and    building at least one report for delivery to a stakeholder in said test site.    
     
     
         54 . The method of  claim 53 , further, comprising the steps of: 
 logging onto said full-service research bureau and test center;    registering to become a paid tester;    downloading a tester client modified browser and/or software working in conjunction with a browser;    searching for a new user experience and marketing effectiveness test session; and    checking a personal account for a payment status and to manage a personal profile.