Method and apparatus for testing automated valuation models

Abstract

A method and apparatus for real time testing of automated valuation using various indicators of accuracy. These indicators are then weighted according to their value as indicators of accuracy using individualized weighting factors or an equation. A ranking is then computed based upon the factors and their weights. This method is preformed continuously, so as to achieve real-time or periodically updated automated valuation model accuracy rankings.

1 . (canceled) 
     
     
         2 . A method comprising:
 providing ranking criteria to an automated valuation model testing entity;   accessing, by a computer system comprising computer hardware, rankings data and accuracy data associated with a plurality of automated valuation models, and provided by the automated valuation model testing entity, wherein the rankings data comprises rankings of the plurality of automated valuation models based at least in part on the provided ranking criteria and comparisons of valuations for a subject property received from the plurality of automated valuation models without reference to new valuation data associated with the subject property; and   performing, by the computer system, one or more calculations based at least in part on the accessed rankings data.   
     
     
         3 . The method of  claim 2 , wherein the ranking criteria comprises cost preferences associated with automated valuation models. 
     
     
         4 . The method of  claim 2 , wherein said accuracy data comprises data indicative of accuracy associated with each automated valuation model. 
     
     
         5 . The method of  claim 2 , wherein the accuracy data comprises weightings associated with the accuracy data. 
     
     
         6 . The method of  claim 2 , wherein the automated valuation model testing entity provides the rankings data in real-time. 
     
     
         7 . The method of  claim 2 , wherein the automated valuation model testing entity provides the rankings data periodically. 
     
     
         8 . The method of  claim 2 , wherein the new valuation data comprises property sale data. 
     
     
         9 . The method of  claim 2 , wherein the new valuation data comprises an appraisal for the subject property. 
     
     
         10 . The method of  claim 2 , wherein the new valuation data comprises an automated valuation of the subject property. 
     
     
         11 . A system comprising:
 physical data storage configured to store (1) a reference valuation for a subject property and (2) a plurality of automated valuations of the subject property, each automated valuation generated by an automated valuation model (AVM) without use of the reference valuation for the subject property, wherein different automated valuations correspond to different AVMS; and   a computer system in communication with the physical data storage, the computer system comprising computer hardware, the computer system programed to:   provide ranking criteria to an automated valuation model testing entity;   access rankings data and accuracy data associated with a portion of the plurality of automated valuation models, and provided by the automated valuation model testing entity, wherein the rankings data comprises rankings of the portion of the plurality of automated valuation models based at least in part on the provided ranking criteria and comparisons of valuations for the subject property received from the portion of the plurality of automated valuation models; and   perform one or more calculations based at least in part on the accessed rankings data.   
     
     
         12 . The system of  claim 11 , wherein the ranking criteria comprises cost preferences associated with automated valuation models. 
     
     
         13 . The system of  claim 11 , wherein the automated valuation model testing entity provides the rankings data in real-time. 
     
     
         14 . The system of  claim 11 , wherein the automated valuation model testing entity provides the rankings data periodically. 
     
     
         15 . The system of  claim 11 , wherein the reference valuation comprises property sale data. 
     
     
         16 . The system of  claim 11 , wherein the reference valuation comprises an appraisal for the subject property. 
     
     
         17 . The system of  claim 11 , wherein the reference valuation comprises an automated valuation of the subject property. 
     
     
         18 . A tangible computer-readable medium that stores thereon a plurality of computer-executable instructions configured, when executed by a computer processor, to cause computer hardware to perform operations comprising:
 accessing rankings data and accuracy data associated with a plurality of automated valuation models, and provided by the automated valuation model testing entity, wherein the rankings data comprises rankings of the plurality of automated valuation models based at least in part on the provided ranking criteria and comparisons of valuations for a subject property received from the plurality of automated valuation models without reference to new valuation data associated with the subject property;   testing the rankings data to determine accuracy of the rankings data; and   performing one or more calculations based at least in part on the accessed rankings data.   
     
     
         19 . The tangible computer-readable medium of  claim 18 , wherein the automated valuation model testing entity provides the rankings data in real-time. 
     
     
         20 . The tangible computer-readable medium of  claim 18 , wherein the new valuation data comprises property sale data. 
     
     
         21 . The tangible computer-readable medium of  claim 18 , wherein the reference valuation comprises an appraisal for the subject property.