System and method of pill identification

Abstract

A system and method for identifying pills by determining a size and shape of each pill in a digital image. The system includes a background grid organized as a grid of alternating-colored shapes. The system also includes a digital camera, a processor and a memory. The processor is used to receive and process the digital image taken by the digital camera so as to determine contours for each pill in the image. The contour determination is refined and is used to determine size and shape information for each pill.

What is claimed: 
     
         1 . A method of identifying a contour of one or more pills, the method comprising:
 receiving a digital image representing one or more pills on a background grid, the background grid organized as a grid of alternating-colored shapes;   using the grid of alternating-colored shapes to determine an initial estimation of contours for the imaged pills, each contour enclosing a pill part; and   creating an improved estimation of contours by combining pill parts representing a single pill in the initial estimation of contours into a single contour for a pill represented by multiple pill parts.   
     
     
         2 . The method of  claim 1 , wherein receiving the digital image further comprises:
 using a background grid having at least four colors.   
     
     
         3 . The method of  claim 1 , wherein using the grid of alternating-colored shapes to determine the initial estimation of contours further comprises:
 using an edge density-based approach to determine the initial estimation of contours.   
     
     
         4 . The method of  claim 3 , wherein using the edge density-based approach comprises:
 finding edges in the digital image;   joining edges to nearby edges to remove the edges in the background grid; and   determining the initial estimation of contours from the edges that remain after nearby edges are joined.   
     
     
         5 . The method of  claim 1 , further comprising using the grid of alternating colored shapes to create a refined binary image that adds additional pill parts to the pill parts enclosed by the initial estimation of contours. 
     
     
         6 . The method of  claim 5 , wherein using the grid of alternating-colored shapes to create the refined binary image further comprises:
 determining a plurality of binary images from the digital image, each binary image representing the presence and absence of a color in the digital image, the color being one of the colors in the background grid;   modifying the plurality of binary images by expanding regions in each of the binary images that represent the presence of the color corresponding to each binary image;   intersecting the modified binary images to create a multiple colored background image; and   combining the multiple colored background image with the initial estimation of contours to create the refined binary image.   
     
     
         7 . The method of  claim 5 , wherein creating the improved estimation of contours further comprises:
 identifying combinations of pill parts in the refined binary image that potentially represent single pills;   finding a convex hull for identified combinations of pill parts;   analyzing the convex hulls to determine which combinations of pill parts are likely to represent single pills; and   for combinations of pill parts that are likely to represent a single pill, removing contours internal to the convex hull and replacing the convex hull with a new contour, the improved estimation of contours being comprised of the new contours.   
     
     
         8 . The method of  claim 7 , wherein the step of analyzing the convex hulls comprises:
 comparing the sizes and shapes of the convex hulls with the sizes and shapes of known pills.   
     
     
         9 . A system of identifying contours for pills, comprising:
 a background grid organized as a grid of alternating-colored shapes;   a digital camera;   a processor for receiving from the digital camera a digital image representing one or more pills on the background grid and process the digital image in order to identify contours for each pill in the digital image;   a pill stitching module for combining pill parts representing a single pill into a single contour for a pill represented by multiple pill parts; and   a memory for storing the identified contours.   
     
     
         10 . The system of  claim 9 , wherein the background grid includes at least four colors. 
     
     
         11 . The system of  claim 9 , further comprising:
 an initial contour estimation module for determining an initial estimation of contours for the imaged pills, each contour enclosing a pill part; and   a pill part addition module for using the background grid to create a refined binary image that adds additional pill parts to the pill parts enclosed by the initial estimation of contours.   
     
     
         12 . The system of  claim 11 , wherein the initial contour estimation module is configured to determine the initial estimation of contours by using an edge density-based approach. 
     
     
         13 . The system of  claim 11 , wherein the initial contour estimation module is configured to:
 find edges in the digital image;   join edges to nearby edges to remove the edges in the background grid; and   determine the initial estimation of contours from the edges that remain after nearby edges are joined.   
     
     
         14 . The system of  claim 11 , wherein the pill part addition module comprises:
 one or more binary image modules for determining a plurality of binary images from the digital image, each binary image representing the presence and absence of a color in the digital image, the color being one of the colors in the background grid;   a modifying module for modifying the plurality of binary images by expanding regions in each of the binary images that represent the presence of the color corresponding to each binary image;   an intersecting module for intersecting the modified binary images to create a multiple colored background image; and   a combining module for combining the multiple colored background image with the initial estimation of contours to create the refined binary image.   
     
     
         15 . The system of  claim 11 , wherein the pill stitching module is configured to:
 identify combinations of pill parts in the refined binary image that potentially represent single pills;   find a convex hull for each identified combination of pill parts;   analyze the convex hulls to determine which combinations of pill parts are likely to represent single pills; and   for combinations of pill parts that is likely to represent a single pill, remove contours internal to the convex hull and replacing the convex hull with a new contour.   
     
     
         16 . A mobile device for identifying pills, comprising:
 a digital camera;   a processor for receiving from the digital camera a digital image representing one or more pills on a background grid organized as a grid of alternating-colored shapes and process the digital image in order to identify each pill in the digital image;   a pill stitching module for combining pill parts representing a single pill into a single contour for a pill represented by multiple pill parts; and   a memory for storing the identity of each pill.   
     
     
         17 . The mobile device of  claim 16 , further comprising a user interface for displaying to a user of the mobile device the identity of each pill. 
     
     
         18 . The mobile device of  claim 16 , further comprising a transmitter for transmitting the identity of each pill to a remote location. 
     
     
         19 . The mobile device of  claim 16 , wherein the background grid includes at least four colors. 
     
     
         20 . The mobile device of  claim 16 , further comprising:
 an initial contour estimation module for determining an initial estimation of contours for the imaged pills, each contour enclosing a pill part;   a pill part addition module for using the background grid to create a refined binary image that adds additional pill parts to the pill parts enclosed by the initial estimation of contours;   a final contour estimation module for creating an improved estimation of contours generated by the pill stitching module by combining pill parts representing a single pill in the refined binary image; and   a pill identification module for identifying pills in the digital image using the contours in the improved estimation of contours.   
     
     
         21 . The mobile device of  claim 20 , wherein the initial contour estimation module is configured to determine the initial estimation of contours by using an edge density-based approach. 
     
     
         22 . The mobile device of  claim 20 , wherein the initial contour estimation module is for determining the initial estimation of contours by:
 finding edges in the digital image;   joining edges to nearby edges to remove the edges in the background grid; and   determining the initial estimation of contours from the edges that remain after nearby edges are joined.   
     
     
         23 . The mobile device of  claim 20 , wherein the pill part addition module comprises:
 one or more binary image modules for determining a plurality of binary images from the digital image, each binary image representing the presence and absence of a color in the digital image, the color being one of the colors in the background grid;   a modifying module for modifying the plurality of binary images by expanding regions in each of the binary images that represent the presence of the color corresponding to each binary image;   an intersecting module for intersecting the modified binary images to create a multiple colored background image; and   a combining module for combining the multiple colored background image with the initial estimation of contours to create the refined binary image.   
     
     
         24 . The mobile device of  claim 20 , wherein the pill stitching module creates the improved estimation of contours by:
 identifying combinations of pill parts in the refined binary image that potentially represent single pills;   finding a convex hull for each identified combination of pill parts;   analyzing the convex hulls to determine which combinations of pill parts are likely to represent single pills; and   for each combination of pill parts that is likely to represent a single pill, removing contours internal to the convex hull and replacing the convex hull with a new contour, the improved estimation of contours being comprised of the new contours.   
     
     
         25 . The mobile device of  claim 20 , wherein the pill identification module uses the contours in the improved estimation of contours to determine the size and shape of each pill. 
     
     
         26 . The mobile device of  claim 25 , wherein the pill identification module further uses at least one of each pill's color scheme, scoring and imprint pattern with the determined size and shape of each pill to determine the identity of each pill.