Method and apparatus for evaluating educational performance

Abstract

An improved method and apparatus for evaluating the performance of an individual. In one aspect, the invention includes a method of encoding questions used in an examination in order to accurately identify and evaluate deficiencies in an individual's knowledge. In another aspect, an improved method of evaluating responses to the encoded questions is disclosed. Adaptive structuring of subsequent exam questions and/or topics is also provided. A computer program and associated apparatus for administering and evaluating performance is also described.

1 .- 32 . (canceled) 
     
     
         33 . A method of educational evaluation, comprising:
 providing a first plurality of questions relating to a plurality of topical areas to a user;   evaluating responses to the plurality of questions provided by the user;
 based at least on the evaluation, identifying one or more areas of weakness of the user; 
 providing supplemental instruction to the user relating to the one or more areas; and 
 providing a second plurality of questions to the user subsequent to the supplemental instruction. 
   
     
     
         34 . The method of  claim 33 , wherein the first and second pluralities of questions each comprise a plurality of questions encoded in at least two aspects, the encoding enabling identification of particular subject matter within the plurality of topical areas. 
     
     
         35 . The method of  claim 34 , wherein the encoding further enables determination of a level of subject matter difficulty. 
     
     
         36 . The method of  claim 35 , wherein said level of subject matter difficulty is related to subject matter that is taught on multiple instructional levels. 
     
     
         37 . The method of  claim 34 , wherein the encoding further enables identification of a language-based difficulty of the user. 
     
     
         38 . The method of  claim 33 , wherein the identifying one or more areas of weakness comprises identifying one or more areas which are not related to the user's level of understanding of the topical areas. 
     
     
         39 . The method of  claim 33 , further comprising evaluating responses to the second plurality of encoded questions to identify subject matter competency level. 
     
     
         40 . The method of  claim 39 , further comprising iteratively providing one or more subsequent pluralities of questions and evaluating responses thereto, until a desired subject level of competency is achieved by the user, the one or more subsequent pluralities of questions comprising at least some questions different than those of said first and second pluralities, and being selected based at least in part on said response evaluation. 
     
     
         41 . The method of  claim 33 , wherein:
 the identifying one or more areas of weakness of the user further comprises determining the proximity of an incorrect response by the user for a given question to a correct response for that question; and   the providing supplemental instruction to the user relating to the one or more areas comprises selecting the supplemental instruction based at least in part on the determined proximity.   
     
     
         42 . A method of educational competency evaluation of a user, the method comprising:
 providing to a user a first plurality of questions relating to a plurality of topical areas;   evaluating responses provided by the user to the first plurality of questions;   evaluating one or more responses by the user to corresponding encoded prompts for explanations relating to the user's responses to the first plurality of questions; and   based at least on the evaluation of the one or more responses to the corresponding encoded prompts, identifying one or more areas of weakness of the user.   
     
     
         43 . The method of  claim 42 , further comprising determining competency of the user in one or more areas based at least in part on the identified one or more weaknesses. 
     
     
         44 . The method of  claim 42 , further comprising using at least statistical information to determine competency of the user in one or more areas. 
     
     
         45 . The method of  claim 44 , wherein the statistical information relates to a plurality of other user's who have previously been tested or evaluated using at least some of the first plurality of questions. 
     
     
         46 . The method of  claim 42 , wherein the corresponding encoded prompts for explanations relating to the user's responses to the first plurality of questions comprise prompts for the user to explain the user's reasoning for choosing an incorrect response. 
     
     
         47 . The method of  claim 42 , wherein the first plurality of questions comprises a plurality of questions encoded in at least two aspects, the encoding at least enabling statistical evaluation of skill level of the user. 
     
     
         48 . The method of  claim 42 , further comprising using the one or more responses to the corresponding encoded prompts for explanations relating to the user's responses to the first plurality of questions in a statistical determination of skill level. 
     
     
         49 . The method of  claim 42 , wherein the responses both to the one or more encoded prompts and the plurality of first questions are used statistically to enable estimation of the user's skill level. 
     
     
         50 . A method of educational evaluation, comprising:
 providing a first plurality of questions relating to a plurality of topical areas to a user;   evaluating responses to the plurality of questions provided by the user;   based at least on the evaluation, identifying one or more areas of weakness of the user not relating to knowledge level;   providing a second plurality of questions to the user, the second plurality of questions being selected based at least in part on the identified one or more areas of weakness; and   evaluating responses to the second plurality of questions to determine whether the user's performance has improved.   
     
     
         51 . The method of  claim 50 , wherein the identified one or more areas of weakness comprise the user's language skill, and the second plurality of questions selected relate to the subject matter of the first plurality of questions, yet are phrased using less obscure terminology.