Methods and Software for Screening and Diagnosing Skin Lesions and Plant Diseases

Abstract

Provided herein are portable imaging systems, for example, a digital processor-implemented system for the identification and/or classification of an object of interest on a body, such as a human or plant body. The systems comprise a hand-held imaging device, such as a smart device, and a library of algorithms or modules that can be implemented thereon to process the imaged object, extract representative features therefrom and classify the object based on the representative features. Also provided are methods for the identifying or classifying an object of interest on a body that utilize the algorithms and an automated portable system configured to implement the same.

What is claimed is: 
     
         1 . A portable imaging system, comprising:
 a hand-held imaging device having a digital camera, a display, a memory, a processor and a network connection; and   a library of algorithms tangibly stored in the memory and executable by the processor, said algorithms configured for identification of an object of interest present on a body.   
     
     
         2 . The portable imaging system of  claim 1 , further comprising algorithms tangibly stored and processor executable algorithms configured to display the object of interest and results of the classification thereof. 
     
     
         3 . The portable imaging system of  claim 1 , wherein the algorithms comprise processor-executable instructions to:
 segment the imaged object to detect a border of the object;   extract features from the segmented object image; and   classify the object based on the extracted features.   
     
     
         4 . The portable imaging system of  claim 3 , wherein the processor-executable instructions to segment the object function to:
 determine an initial contour of the imaged object;   classify pixels as contained within the initial contour as foreground, as contained without the initial contour as background or as remaining pixels; and   apply a classifier to the remaining pixels for classification as foreground or background.   
     
     
         5 . The portable imaging system of  claim 4 , wherein the processor-executable instructions to extract features function to:
 divide the segmented object image into regions based on saliency values calculated for at least one patch within the segmented object;   divide the regions into two regions of higher or lower intensity based on average intensity values thereof; and   extract feature representations from a sampling of patches within the intensity regions based on sampling percentages determined for the regions.   
     
     
         6 . The portable imaging system of  claim 5 , wherein the processor-executable instructions to classify the object function to:
 input the extracted feature representations into a support vector machine trained with manually segmented objects; and   classify the object based on a comparison of the inputted extracted features with those in the trained support vector machine.   
     
     
         7 . The hand-held imaging system of  claim 1 , wherein the hand-held imaging device is a smart device. 
     
     
         8 . The hand-held imaging system of  claim 1 , wherein the body is a human body or a plant body. 
     
     
         9 . The hand-held imaging system of  claim 1 , wherein the object of interest is a lesion, an ulcer, or a wound. 
     
     
         10 . A method for identifying an object of interest present on a body, comprising:
 acquiring an image of the object of interest on the body via the imaging device comprising the portable imaging system of  claim 1 ;   processing the acquired object image via the algorithms tangibly stored in the imaging device; and   identifying the object in the image based on patterns of features present in the imaged object, thereby identifying the object of interest on the body.   
     
     
         11 . The method of  claim 10 , further comprising:
 displaying the results of image processing as each result occurs.   
     
     
         12 . The method of  claim 10 , wherein identifying the object occurs in real time. 
     
     
         13 . The method of  claim 10 , wherein the object of interest is a melanoma or a Buruli ulcer. 
     
     
         14 . A digital processor-implemented system for classifying an object of interest on an animal or plant body in real time, comprising:
 a portable smart device comprising the processor, a memory and a network connection; and   modules tangibly stored in the memory comprising:
 a module for segmentation of an imaged object; 
 a module for feature extraction within the segmented object image; and 
 a module for classification of the object based on extracted features. 
   
     
     
         15 . The digital processor-implemented system of  claim 14 , further comprising a module tangibly stored in the memory for display of the object of interest and results of the classification thereof. 
     
     
         16 . The digital processor-implemented system of  claim 14 , wherein the segmentation module comprises processor executable instructions to:
 obtain luminance and color components of the imaged object;   classify pixels comprising the image as object pixels, if they belong to a common luminance and color foreground, as background pixels if they belong to a common luminance and color background or as remaining pixels; and   apply a classifier to the remaining pixels to classify them as object or foreground.   
     
     
         17 . The digital processor-implemented system of  claim 16 , wherein the feature extraction module comprises processor executable instructions to:
 calculate a saliency value for a plurality of patches within the segmented object and separate the patches into regions based on the saliency values;   calculate an average intensity for the regions to identify them as a higher or as a lower intensity region;   determine a sampling percentage for the intensity regions;   sample patches within the intensity regions by corresponding sampling percentages; and   extract one or more feature representations for the object.   
     
     
         18 . The digital processor-implemented system of  claim 16 , wherein the feature extraction module comprises processor executable instructions to:
 read input white light image as RGB and the segmentation result of the region;   read input multispectral images in color channels and transform to gray scale;   register multispectral images via maximization of mutual information with white light image as reference;   extract feature representations within the ROI of multispectral images and within white light images; and   select one or more relevant features from a pool of the extracted features.   
     
     
         19 . The digital processor-implemented system of  claim 16 , wherein the feature extraction module comprises processor executable instructions to:
 read input white light image as RGB and the segmentation result of the region;   read input multispectral images in color channels and transform to gray scale;   register multispectral images via maximization of mutual information with white light image as reference;   determine V mel , V blood , and V oxy  for each ROI pixel to reconstruct maps of melanin, blood and oxygenating percentage;   extract feature representations within the ROI from the reconstructed maps; and   select one or more relevant features from a pool of the extracted features.   
     
     
         20 . The digital processor-implemented system of  claim 17 , wherein the classification module comprises processor executable instructions to:
 train a support vector machine (SVM) with known manually segmented objects; and   classify the object based on the extracted feature representations inputted into the SVM.   
     
     
         21 . The hand-held imaging system of  claim 14 , wherein the object of interest is a lesion, an ulcer, a wound, or skin. 
     
     
         22 . A digital processor-implemented method for classifying an object of interest on an animal or plant body in real time, comprising the processor executable steps of:
 digitally imaging the object of interest with the smart device comprising the digital processor-implemented system of  claim 14 ;   processing the digital image through the system modules, said modules comprising algorithms configured for:
 segmenting the image based on saliency values to identify pixels thereof as comprising the imaged object or the background of the image to obtain an object boundary; 
 extracting features from regions within the object boundary; and 
 comparing the extracted features to known object features in a support vector machine trained on the known features to obtain a classification of the object; and 
   displaying the processed images and classification results on a display comprising the smart device.   
     
     
         23 . The digital processor-implemented method of  claim 20 , wherein the support vector machine is trained on features comprising a melanoma or a Buruli ulcer. 
     
     
         24 . A digital processor-readable medium tangibly storing processor-executable instructions to perform the digital processor implemented method of  claim 20 . 
     
     
         25 . A computer-readable medium tangibly storing a library of algorithms to classify an object of interest on a human or plant body, said algorithms comprising processor-executable instructions operable to:
 obtain luminance and color components of the imaged object;   classify pixels comprising the image as object pixels, if they belong to a common luminance and color foreground, as background pixels if they belong to a common luminance and color background or as remaining pixels;   apply a classifier to the remaining pixels to classify them as object or foreground;   extract one or more feature representations for the object;   train a support vector machine (SVM° with known manually segmented objects; and   classify the object based on the extracted feature representations inputted into the SVM.   
     
     
         26 . The computer-readable medium of  claim 25 , wherein the instructions to extract one or more feature representations for the object comprise:
 calculate a saliency value for a plurality of patches within the segmented object and separate the patches into regions based on the saliency values;   calculate an average intensity for the regions to identify them as a higher or as a lower intensity region;   determine a sampling percentage for the intensity regions;   sample patches within the intensity regions by corresponding sampling percentages; and   extract the one or more feature representations for the object.   
     
     
         27 . The computer-readable medium of  claim 25 , wherein the instructions to extract one or more feature representations for the object comprise:
 read input white light image as RGB and the segmentation result of the region;   read input multispectral images in color channels and transform to gray scale;   register multispectral images via maximization of mutual information with white light image as reference;   extract feature representations within the ROI of multispectral images and within white light images; and   select one or more relevant features from a pool of the extracted features.   
     
     
         28 . The computer-readable medium of  claim 25 , wherein the object is the skin, said instructions to extract one or more feature representations for the object comprising:
 read input white light image as RGB and the segmentation result of the region;   read input multispectral images in color channels and transform to gray scale;   register multispectral images via maximization of mutual information with white light image as reference;   determine V mel , V blood , and V oxy  for each ROI pixel to reconstruct maps of melanin, blood and oxygenating percentage;   extract feature representations within the ROI from the reconstructed maps; and   select one or more relevant features from a pool of the extracted features.