Generating artificial hyperspectral images using correlated analysis of co-registered images

Abstract

High-resolution digital images of adjacent slices of a tissue sample are acquired, and tiles are defined in the images. Values associated with image objects detected in each tile are calculated. The tiles in adjacent images are co-registered. A first hyperspectral image is generated using a first image, and a second hyperspectral image is generated using a second image. A first pixel of the first hyperspectral image has a first pixel value corresponding to a local value obtained using image analysis on a tile in the first image. A second pixel of the second hyperspectral image has a second pixel value corresponding to a local value calculated from a tile in the second image. A third hyperspectral image is generated by combining the first and second hyperspectral images. The third hyperspectral image is then displayed on a computer monitor using a false-color encoding generated using the first and second pixel values.

What is claimed is: 
     
       1. A method comprising:
 (a) defining a first tile on a first digital image of a first slice of a tissue sample, wherein the first tile has a first position in the first digital image; 
 (b) defining a second tile on a second digital image of a second slice of the tissue sample, wherein the second tile has a second position in the second digital image, and wherein the second slice originated from the tissue sample adjacent to the first slice; 
 (c) calculating a first value associated with image objects detected in the first tile; 
 (d) calculating a second value associated with image objects detected in the second tile; 
 (e) generating a first hyperspectral image using the first digital image, wherein a first pixel of the first hyperspectral image has a first pixel value corresponding to the first value and a first pixel location corresponding to the first position; 
 (f) generating a second hyperspectral image using the second digital image, wherein a second pixel of the second hyperspectral image has a second pixel value corresponding to the second value and a second pixel location corresponding to the second position; 
 (g) generating a third hyperspectral image using the first hyperspectral image and the second hyperspectral image; and 
 (h) displaying the third hyperspectral image on a computer monitor using a false-color encoding generated using the first pixel value and the second pixel value. 
 
     
     
       2. The method of  claim 1 , wherein the first slice is stained with a first biomarker, wherein the first value corresponds to a histopathological score of the first biomarker, wherein the second slice is stained with a second biomarker, and wherein the second value corresponds to a histopathological score of the second biomarker. 
     
     
       3. The method of  claim 2 , wherein the false-color encoding is based on a difference between the first value and the second value. 
     
     
       4. The method of  claim 2 , wherein the third hyperspectral image depicts a heterogeneity of a tumor in the tissue sample. 
     
     
       5. The method of  claim 1 , wherein the third hyperspectral image is generated in (g) by co-registering the first hyperspectral image with the second hyperspectral image. 
     
     
       6. The method of  claim 1 , further comprising:
 (i) generating an image object by segmenting the third hyperspectral image using an image analysis process. 
 
     
     
       7. The method of  claim 6 , further comprising:
 (j) determining a value of a property of the image object of the third hyperspectral image; and 
 (k) storing the value of the property in a non-volatile memory. 
 
     
     
       8. The method of  claim 1 , wherein the first slice is stained with an H&E biomarker, and wherein the first value corresponds to a number of mitotic objects stained by the H&E biomarker. 
     
     
       9. The method of  claim 1 , wherein the second slice is immunohistochemically (IHC) stained, and wherein the second value corresponds to a first Allred score. 
     
     
       10. The method of  claim 9 , wherein the second slice is stained using an estrogen receptor antibody. 
     
     
       11. The method of  claim 10 , wherein the first slice is immunohistochemically (IHC) stained using a progesterone receptor antibody, and wherein the first value corresponds to a second Allred score. 
     
     
       12. The method of  claim 1 , wherein the second slice is processed with in-situ hybridization, and wherein the second value indicates gene amplification. 
     
     
       13. The method of  claim 1 , wherein the first value is calculated using a factor, and wherein the factor is taken from the group consisting of: a mean, a median, a minimum, a maximum, a quantile, and a standard deviation of a property of a subset of the image objects detected in the first tile. 
     
     
       14. The method of  claim 13 , wherein the first value is calculated also using the property of image objects detected in tiles adjacent to the first tile. 
     
     
       15. The method of  claim 13 , wherein the subset of the image objects detected in the first tile are those image objects classified as nuclei of tumor cells, and wherein the property is an intensity of staining of the subset of the image objects. 
     
     
       16. The method of  claim 1 , further comprising:
 (i) selecting a pixel of the third hyperspectral image; and 
 (j) displaying on the computer monitor a tile of the first digital image that corresponds to the selected pixel of the third hyperspectral image. 
 
     
     
       17. The method of  claim 1 , further comprising:
 (i) selecting the first tile of the first digital image; and 
 (j) displaying the first pixel value on the computer monitor. 
 
     
     
       18. The method of  claim 17 , wherein the first pixel value is displayed as part of a bar chart. 
     
     
       19. The method of  claim 13 , wherein the tissue sample is taken from a patient, further comprising:
 (i) generating a therapy response for the patient based on the first value. 
 
     
     
       20. The method of  claim 13 , wherein the tissue sample is taken from a patient with cancer, further comprising:
 (i) determining a probability of recurrence of the cancer based on the first value. 
 
     
     
       21. The method of  claim 1 , wherein the first value is calculated using a distance weighted mean of a property of a subset of the image objects detected in the first tile, and wherein the distance weighted mean is based on distances from a center of the first tile to a center of each of the image objects in the subset.