Data quality management using business process modeling

Abstract

A business process modeling framework is used for data quality analysis. The modeling framework represents the sources of transactions entering the information processing system, the various tasks within the process that manipulate or transform these transactions, and the data repositories in which the transactions are stored or aggregated. A subset of these tasks is associated as the potential error introduction sources, and the rate and magnitude of various error classes at each such task are probabilistically modeled. This model can be used to predict how changes in transactions volumes and business processes impact data quality at the aggregate level in the data repositories. The model can also account for the presence of error correcting controls and assess how the placement and effectiveness of these controls alter the propagation and aggregation of errors. Optimization techniques are used for the placement of error correcting controls that meet target quality requirements while minimizing the cost of operating these controls. This analysis also contributes to the development of business “dashboards” that allow decision-makers to monitor and react to key performance indicators (KPIs) based on aggregation of the transactions being processed. Data quality estimation in real time provides the accuracy of these KPIs (in terms of the probability that a KPI is above or below a given value), which may condition the action undertaken by the decision-maker.

1 . A data quality management method comprising the steps of: 
 creating a model of a new or existing business process;    utilizing a modeling framework, identifying transaction sources, error sources, and audit targets;    running error propagation analysis to estimate error rates and cost of error at the audit targets;    utilizing a control systems model to associate error sources with a set of controls; and    analyzing an impact of selected controls using an assessment technique.    
     
     
         2 . The data quality management method recited in  claim 1 , further comprising the steps for transaction sources of obtaining or estimating a volume of transactions over a given time period and estimating transaction book values.  
     
     
         3 . The data quality management method recited in  claim 2 , wherein estimating transaction book values is based on a simple average book value or a probability distribution based on historical transaction data.  
     
     
         4 . The data quality management method recited in  claim 1 , further comprising the steps for error sources of obtaining a probability of errors prior to application of any controls and a taint of the error sources.  
     
     
         5 . The data quality management method recited in  claim 4 , wherein the probability of errors and the taint of the error sources are obtained from logs of controls that already exist.  
     
     
         6 . The data quality management method recited in  claim 4 , wherein for a new business process or for error sources that do not have logs of past control activity, an estimation is done based on comparable error sources with available data.  
     
     
         7 . The data quality management method recited in  claim 1 , further comprising the steps for audit targets of specifying types of errors of interest and if any error level requirements exist for them.  
     
     
         8 . The data quality management method recited in  claim 1 , further comprising the step for a model with probability distributions of performing a Monte Carlo simulation to estimate error rates and costs in terms of probability distributions.  
     
     
         9 . The data quality management method recited in  claim 1 , further comprising the step for each control of estimating its error detection and correction effectiveness.  
     
     
         10 . The data quality management method recited in  claim 1 , further comprising the step of maximizing the reliability level at audit targets subject to meeting a budget for a cost of controls.  
     
     
         11 . The data quality management method recited in  claim 1 , further comprising the step of minimizing a cost of controls subject to meeting a minimum reliability level at audit targets.