Systems and methods for automated screening and prognosis of cancer from whole-slide biopsy images

Abstract

The invention provides systems and methods for detection, grading, scoring and tele-screening of cancerous lesions. A complete scheme for automated quantitative analysis and assessment of human and animal tissue images of several types of cancers is presented. Various aspects of the invention are directed to the detection, grading, prediction and staging of prostate cancer on serial sections/slides of prostate core images, or biopsy images. Accordingly, the invention includes a variety of sub-systems, which could be used separately or in conjunction to automatically grade cancerous regions. Each system utilizes a different approach with a different feature set. For instance, in the quantitative analysis, textural-based and morphology-based features may be extracted at image- and (or) object-levels from regions of interest. Additionally, the invention provides sub-systems and methods for accurate detection and mapping of disease in whole slide digitized images by extracting new features through integration of one or more of the above-mentioned classification systems. The invention also addresses the modeling, qualitative analysis and assessment of 3-D histopathology images which assist pathologists in visualization, evaluation and diagnosis of diseased tissue. Moreover, the invention includes systems and methods for the development of a tele-screening system in which the proposed computer-aided diagnosis (CAD) systems. In some embodiments, novel methods for image analysis (including edge detection, color mapping characterization and others) are provided for use prior to feature extraction in the proposed CAD systems.

What is claimed is: 
     
         1 . A method, performed by a processing unit, for computing pixels along object edges and producing a deinterlaced image from an interlaced source, the method comprising:
 performing image filtering on a collected image depending on the nature of noise in the collected image;   smoothing the filtered image using shape-dependent filters;   calculating gradient vectors in the image using different kernels;   selecting an edge angle;   determine threshold values within a local dynamic range,   generating several edge maps, and   fusing the generated edge maps together.   
     
     
         2 . The method of  claim 1 , further comprising applying an algorithm for image edge-preserving contrast enhancement which is based on HVS, Parameterized Logarithm Image Processing operations, and is integrated with morphological log-ratio approach in order to come up with an effective edge detection operator that is sensitive to edges of dark areas in the image. 
     
     
         3 . Methods and systems for computation of fractal-like dimension for modifying input data/image into data describing the image, the method comprising:
 performing image filtering depending on the nature of noise in the image;   binarizing the image by using the method described in  claim 1  or  2 , or by using an image bit-plane decomposition method;   calculating the fractal dimension of binary images by using different grid sizes based on a Differential Box Counting (DBC) algorithm; and   fusing resulting fractal dimensions.   
     
     
         4 . A method for computing the fractal dimension of color images comprising:
 performing a color model transformation by using arbitrary functions operating on the original RGB components of the image;   splitting the image in smaller windows and computing for each block the probability of having m points/pixels in a hypercube of the size of the window;   estimating the color fractal dimension by using a weighting function and fitting the curve of logarithm of the window sizes against the logarithm of the total number of boxes of each window size needed to cover the image.   
     
     
         5 . A method for carcinomas color region mapping comprising:
 constructing 2-D projections of the RGB color model of a large variety of biopsy images including normal and cancerous tissue;   computing 2-D histograms of biopsy images;   selecting the most prominent colors by using an algorithm for local maxima location in 3-D surfaces;   constructing a color region mapping, in which each color corresponds to one class or tissue structure.   
     
     
         6 . A method for image color quantization comprising:
 performing an image color standardization procedure;   calculating the distance between each pixel and the colors considered in the color map of  claim 4  by using a distance metric;   minimizing the calculated distance; and   assigning the nearest reference color or class to the each pixel.   
     
     
         7 . A textural-based system and methods for automatically detecting, classifying, and grading cancerous regions of a histology image comprising:
 performing an image color standardization procedure;   forming texture-based feature vectors by using spatial and transforms domain image information along with fractal analysis;   selecting the group of features that best describe the images;   training a classifier by using the generated feature vectors;   classifying histology images according to the Gleason grading system;   using the result of classification to determine the Gleason score of the image;   assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.   
     
     
         8 . The method of  claim 7 , wherein forming texture-based feature vectors comprises extraction of a set of features from Fourier and wavelet transforms such us wavelet energy and entropy, phase information of coefficients, and spatial domain algorithms such as statistical spatial filtering, wavelet-based fractal dimension according to the method of  claim 3  or others, and gray level histogram. 
     
     
         9 . A textural-based system and methods for automatically detect, classify, and grade cancerous regions of a histology image (particularly a prostate biopsy images) comprising:
 performing an image color standardization procedure;   describing the image data and forming texture-based feature vectors by using spatial and transforms domain image information along with color fractal analysis as claimed in  claim 4 ;   selecting the group of features that best describe the images;   training a classifier by using the generated feature vectors;   classifying histology images according to the Gleason grading system;   using the result of classification to determine the Gleason score of the image;   assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.   
     
     
         10 . The description step of the system of  claim 9 , further comprising the extraction of a set of features from real, logarithmic, or complex wavelets and multi-wavelets and a new color fractal dimension algorithm developed in this invention. Particularly, transform features include joint probability of detail coefficients, phase information of coefficients, energy distribution of detail coefficients, and energy of color channels of the biopsy image. A new color fractal dimension is also extracted to complete the feature space. 
     
     
         11 . A morphology- and architectural-based system and methods for automatically detect, classify, and grade cancerous regions of a histology image (particularly a prostate biopsy images) comprising:
 performing an image color standardization procedure;   segmenting the image by applying any proper algorithm including the method proposed in  claims 5  and  6 ;   describing the image data and forming morphology and architectural-based feature vectors by extracting the shape, size and arrangement of tissue structures;   selecting the group of features that best describe the images;   training a classifier by using the generated feature vectors;   classifying cancerous images according to grading systems, for example Gleason grading system for prostate carcinomas;   using the result of classification to determine the Gleason score of the image;   assessing the accuracy of the Gleason grading/scoring system by using cross-validation methods.   
     
     
         12 . The description step of the system of  claim 11 , further comprising the extraction of features: color information such as color distribution entropy and 2-D color histograms along with tissue morphology and architectural features from segmented tissue structures. 
     
     
         13 . Systems and methods for automatically detecting, classifying and grading cancerous regions of 3-D histological images (particularly a prostate image) comprising:
 performing an image color standardization procedure,   segmenting image by applying any proper algorithm including the method proposed in  claims 5  and  6 ;   mapping of 2-D images or an image into 3-D space by using pre-developed mapping algorithms,   describing the image data and forming color-, texture-, morphology-, and architectural-based feature vectors;   training a classifier within an framework by using generated feature vectors;   accessing the cancer grade or scale, for example, according to the Gleason grading scale;   accessing the most prominent, the second most prominent pattern and the third prominent pattern in said image data and computing a Gleason 3-D score as a sum of Gleason 3-D grades of said patterns;   accessing the accuracy of Gleason 3-D grading/scoring system;   estimating volume of a tumor region.   
     
     
         14 . The systems as claimed in  13 , wherein the mapping from 2-D image to 3-D images includes algorithms for 3-D reconstruction from a single 2-D image or from several 2-D slides. The processed images nay be gray scaled or colored images. 
     
     
         15 . The systems of  claims 7 ,  9 ,  11  and  13  wherein automatically classifying the biopsy images is based on machine learning algorithms: linear discriminants, Gaussian models, Multivariate Adaptive Regression Splines (MARS), Classification an Regression trees (CART), decision trees, k-nearest neighbor, Bayesian, neural networks, and/or SVM. Boosting algorithms such as AdaBoost, SVM Boost, etc. can be used to improve the classification performance. 
     
     
         16 . A method to obtain a multidimensional Gleason grading wherein a revised Gleason value is generated as follows:
   Revised Gleason Value=2D Gleason Grade+3rd Dimensional Core Grade   
     
     
         17 . A system and method for automatically detecting, classifying and grading cancerous regions from digitized Whole-Slide (particularly a prostate image) comprising:
 performing an image color standardization procedure;   splitting image into smaller regions;   determining and grading a location of a tumor in the prostate using one or more of the systems of  claim 7  or  9  or  11  or  13 ;   accessing a cancer stage;   accessing the most prominent, the second most prominent pattern and third prominent pattern in said image data and computing a Gleason score as a sum of Gleason grades of said patterns;   generating a cancer map from whole digitized slides according to Gleason grading/scoring levels associated with slides processing block-elements;   accessing the accuracy of Gleason grading/scoring system;   accessing the estimating size of a tumor region.   
     
     
         18 . The system according to  claim 17 , wherein said grade estimating module is configured for estimating a size of a tumor in said at least one tissue region or whole digitized slides. 
     
     
         19 . The system of  claim 17 , wherein a complete diagnosis is performed for providing useful information for pathologists about the disease severity and the localization, size and volume of cancerous tissue. 
     
     
         20 . The system of  claim 17  wherein complete diagnosing is performed, the outputs of the systems include: tumor extent in the needle biopsy, number and identification of positive needle cores, percentage of positive cores for cancer, length of the tumor in positive cores and length of the core, percentage of needle core tissue affected by carcinoma and area of tissue positive for cancer, Gleason grade of each selected cancerous region of interest, percentage of each Gleason pattern in biopsy specimens, Gleason score of each positive core, including tertiary pattern information, the most frequent Gleason score in slides, numbers of identical Gleason scores in the positive cores, revised Gleason value ( claim 16 ), and localized cancer map including the grade of each cancerous patch in all needle biopsy slides. 
     
     
         21 . The system of  claim 17  further comprising methods for surgery quality control, which allow a surgeon to be immediately aware of whether the surrounding tissue of a specimen is or not positive for cancer, prior the patient is closed up. 
     
     
         22 . A system for cancer prognosis, wherein the pathology report including all the outputs of systems claimed in  7 ,  9 ,  11  and  13  is integrated with patient information and PSA analysis results to construct feature vectors. The resulting feature vectors are used to aid pathologists in cancer prognosis, risk assessment and treatment planning. 
     
     
         23 . The system of  claim 22 , further comprising the use of feature vectors for prostate cancer risk assessment by implementing risk calculators according to the patient's available information. 
     
     
         24 . The system of  claim 22  further comprising a method for cancer prediction based on 2-D and 3-D histopathology images using cancer map or Gleason graded block and pixel features to predict the state of the tissue in missing blocks within the scope of the image or to produce extrapolated values to know the cancer grade in the neighborhoods outside the core extracted from the prostate gland during a needle biopsy procedure. 
     
     
         25 . The method of  claim 24  wherein Kalman filtering or similar techniques are used for estimation and prediction of cancer grade (Gleason grade) outside of the scope of observed histopathology biopsy images. 
     
     
         26 . A system and methods for cancer tele-screening and diagnosis based on biopsy image and an adjudication scheme, wherein expert pathologists and CAD systems are integrated. 
     
     
         27 . The adjudication methods for prostate cancer diagnoses used in the system of  claim 26  consist of replacing one or more pathologists by CAD systems and evaluate the agreement among them to produce a faster and less expensive prostate cancer diagnosis. 
     
     
         28 . The systems of  claim 26  wherein said adjudication methods is configure for using morphology- or textural-based 2-D classification systems for cancer detection and grading, such that the CADs integrated into the systems perform independent tissue assessment. 
     
     
         29 . The tele-screening system of  claim 26  uses communication networks such as internet for image and patient data acquisition, pathologists' report acquisition. The final assessment results are stored in a centralized database available for pathologist consultation. 
     
     
         30 . A method and system constructing an optimal signal detector using a learning algorithm in tandem with a prediction algorithm (which may be one in the same) and data compression algorithm. 
     
     
         31 . A method and system for detecting carcinomas in a sample image based on compression rates obtained using a detector trained on previously classified carcinoma images as specified in  claim 30 . 
     
     
         32 . A method and system for using detection results to compute a hard, integer Gleason score of a query image through selection of a best compressing pattern compressors (PCs) using one of the systems of  claims 30  and  31 . 
     
     
         33 . A method and system for using detection results to compute a hard, integer Gleason scores of query image pixels and regions through selection of a best compressing pattern compressors (PCs) using one of the systems of  claims 30  and  31 . 
     
     
         34 . A method and system for using detection results to compute a soft, continuous-scale Gleason score of an image (query), comprising:
 compressing query image data using specific pattern PCs and optionally a self PC according to systems in  claims 30  and  31 ;   computing pattern weights determined by:
 PC compression rates, 
 PC compression rate statistics, 
 and/or functions of each pattern PC compression rates or statistics and 
 query image self PC compression rates or statistics; 
   combining pattern values and weights into a single Gleason score.   
     
     
         35 . A method and system for using detection results to compute a soft, continuous-scale Gleason scores of query image pixels and regions, comprising:
 compressing query image pixels using specific pattern PCs and optionally a self PC according to systems in  claims 30  and  31 ;   computing pattern weights determined by either by pattern PC compression rate or by comparison of each pattern PC prediction to a query image's self PC prediction on a pixel by pixel basis;   combining pattern values and weights into a single, soft Gleason score on a pixel by pixel basis.   
     
     
         36 . A system and method for determining the most likely distribution of Gleason patterns within a query image, comprising:
 the systems of claim in  30 ,  31  and  33  or  35 ;   determining pairwise uncertainty statistics between pixel scores or grades;   forming the uncertainty statistics into a matrix;   computing the eigenvalues of said matrix;   computing the eigenvectors of said matrix;   using all or some of the above features to estimate the most likely current distribution of Gleason patterns within the sample;   using all or some of the above features to estimate the most likely future distribution of Gleason patterns within the sample;   
     
     
         37 . A system and method for automatically detecting, classifying and grading cancerous regions from digitized, Whole-Slides (particularly a prostate image) comprising:
 segmenting the image into prominent features;   optionally downsizing the image;   converting the image into a 1 dimensional signal;   compressing the signal using the systems of  claims 30  and  31 ;   assigning Gleason scores to each pixel using the system of  claims 33  or  35 , and  36 ;   determining, grading, and scoring a location of a tumor in the prostate using one or more of the systems of  claim 33 ,  35  or  36 ;   assessing the and grading a location of a tumor in the prostate using one or more of the systems of  claim 33 ,  35  or  36 ;   estimating the current carcinoma pattern distribution of a selected region using the systems of  claim 33 ,  35  or  36 ;   estimating the future carcinoma pattern distribution of a selected region using the systems of  claim 33 ,  35  or  36 ;   generating a cancer map from whole, digitized slides according to the systems of  claim 33 ,  35  or  36 ;   assessing the accuracy of Gleason grading/scoring system;   
     
     
         38 . The system according to  claim 37 , wherein said grade estimating module is configured for estimating a size of a tumor in said at least one tissue region or whole digitized slides. 
     
     
         39 . The system of  claim 37 , wherein a complete diagnosis is performed for providing useful information for pathologists about the disease severity and the localization, size and volume of cancerous tissue. 
     
     
         40 . The system of  claim 37  wherein complete diagnosing is performed, the outputs of the systems include: tumor extent in the needle biopsy, number and identification of positive needle cores, percentage of positive cores for cancer, length of the tumor in positive cores and length of the core, percentage of needle core tissue affected by carcinoma and area of tissue positive for cancer, Gleason grade of each selected cancerous region of interest, percentage of each Gleason pattern in biopsy specimens, Gleason score of each positive core, including tertiary pattern information, the most frequent Gleason score in slides, numbers of identical Gleason scores in the positive cores, revised Gleason value ( claim 16 ), and localized cancer map including the grade of each cancerous patch in all needle biopsy slides. 
     
     
         41 . The system of  claim 37  further comprising methods for surgery quality control, which allow a surgeon to be immediately aware of whether the surrounding tissue of a specimen is or not positive for cancer, prior the patient is closed up. 
     
     
         42 . A system for cancer prognosis, wherein the pathology report including all the outputs of systems claimed in  30  through  41  is integrated with patient information and PSA analysis results to construct feature vectors. The resulting feature vectors are used to aid pathologists in cancer prognosis, risk assessment and treatment planning.