Automated object recognition kiosk for retail checkouts

Abstract

A system, method, and apparatus for automated object recognition at checkout is provided. One checkout system includes a base, a head portion, a support for the head portion above the base, an illumination device in the head portion, at least one imaging device in the head portion, a processor coupled to the at least one imaging device, and a display. An examination space is defined between the base and the head portion for accommodating food items. The illumination device illuminates the food items, and the at least one imaging device captures color images of the food items. The processor applies a machine-learning model for performing image recognition of the food items in the color images to identify each food item. The image recognition is based on features of the food items that include shape, size, and color. An identification of each of the food items is presented on the display.

What is claimed is: 
     
         1 . A checkout system comprising:
 a base;   a head portion;   a support that supports the head portion above the base, an examination space being defined between the base and the head portion for accommodating a plurality of food items in the examination space;   an illumination device, disposed in the head portion, for illuminating the plurality of food items;   at least one imaging device, disposed in the head portion, for capturing a plurality of color images of the plurality of food items;   a hardware processor, coupled to the at least one imaging device, for applying a machine-learning model for performing image recognition of the plurality of food items in the plurality of color images to identify each food item of the plurality of food items, the image recognition being based on a plurality of features of the food items that include shape, size, and color; and   a display device for presenting an identification of each of the food items.   
     
     
         2 . The checkout system of  claim 1 , wherein the plurality of features comprises a texture of the food items. 
     
     
         3 . The checkout system of  claim 1 , wherein a machine-learning program formulates the machine-learning model with supervised learning using a training dataset comprising reference images of food items. 
     
     
         4 . The checkout system of  claim 1 , wherein the hardware processor calculates a price for each food item and a total price for all the food items in the examination space, the checkout system comprising a card reader to effect payment of the total price for all the food items. 
     
     
         5 . The checkout system of  claim 1 , comprising:
 a food item database including food item names and prices, wherein the hardware processor accesses the food item database for each identified food item and causes presentation, on the display device, of a name and a price of each identified food item.   
     
     
         6 . The checkout system of  claim 5 , comprising
 a weight sensor disposed in the base for measuring a weight of one or more food items, wherein the price of the one or more food items is based on the measured weight.   
     
     
         7 . The checkout system of  claim 1 , wherein the at least one imaging device comprises at least one color three-dimensional camera or at least one color two-dimensional camera. 
     
     
         8 . The checkout system of  claim 1 , wherein the at least one imaging device comprises at least one color two-dimensional camera and at least one color three-dimensional camera. 
     
     
         9 . The checkout system of  claim 1 , wherein the illumination device generates white light. 
     
     
         10 . The checkout system of  claim 1 , wherein the checkout system adaptively tunes the illumination device to generate light that illuminates the plurality of food items with a predetermined level of brightness. 
     
     
         11 . The checkout system of  claim 1 , wherein a top surface of the base comprises a calibration pattern, wherein the hardware processor determines a position of the at least one imaging device relative to the base based on images, taken by the at least one imaging device, of the calibration pattern. 
     
     
         12 . The checkout system of  claim 1 , wherein each feature is associated with a property of each food item, the food items being at least one of fresh food items and packaged food items. 
     
     
         13 . The checkout system of  claim 1 , wherein the at least one imaging device captures color images of food items placed in one or more orientations in the examination space for training the machine-learning model. 
     
     
         14 . The checkout system of  claim 1 , wherein the machine-learning model is trained to recognize food items selected from a group consisting of packaged goods, fruits, vegetables, and fresh food, the fresh food comprising one or more of curries, breads, pasta, salads, and burgers. 
     
     
         15 . The checkout system of  claim 1 , wherein the plurality of color images is captured after placing the plurality of food items in the examination space. 
     
     
         16 . A method comprising:
 illuminating an examination space for accommodating a plurality of food items in a checkout system, the examination space being defined between a base and a head portion of the checkout system;   capturing, by at least one imaging device mounted in the head portion, a plurality of color images of the plurality of food items;   applying, by a hardware processor of the checkout system, a machine-learning model to perform image recognition of the plurality of food items in the plurality of color images to identify each food item of the plurality of food items, the image recognition being based on a plurality of features of the food items that include shape, size, and color; and   presenting, on a display device of the checkout system, an identification of each of the identified food items.   
     
     
         17 . The method of  claim 16 , wherein the plurality of features comprises a texture of the food items. 
     
     
         18 . The method of  claim 16 , wherein a machine-learning program generates the machine-learning model by supervised learning using a training dataset comprising reference images of food items. 
     
     
         19 . The method of  claim 16 , comprising:
 calculating, by the hardware processor, a price for each food item and a total price for all the food items in the examination space, the checkout system comprising a card reader to effect payment of the total price for all the food items.   
     
     
         20 . The method of  claim 16 , comprising:
 accessing, by the hardware processor, a food item database including food item names and prices to obtain a name and price of each identified food item; and   presenting, on the display device, the name and the price of each identified food item.   
     
     
         21 . The method of  claim 16 , comprising:
 weighting, by a weight sensor disposed in the base, at least one food item; and   calculating the price of the at least one food items based on the measured weight.   
     
     
         22 . The method of  claim 16 , comprising capturing at least one three-dimensional color image with at least one three-dimensional color camera. 
     
     
         23 . The method of  claim 16 , comprising capturing at least one two-dimensional color image with at least one two-dimensional color camera. 
     
     
         24 . The method of  claim 16 , wherein the illumination device generates white light, the method comprising:
 adaptively tunings the illumination device to generate light that illuminates the plurality of food items with a predetermined level of brightness.   
     
     
         25 . The method of  claim 16 , comprising:
 determining, by the hardware processor, a position of the at least one imaging device relative to the base using a calibration pattern of the base.   
     
     
         26 . The method of  claim 16 , wherein each feature is associated with a property of each food item, the food items being at least one of fresh food items and packaged food items. 
     
     
         27 . The method of  claim 16 , wherein the at least one image capturing device captures color images of food items placed in one or more positions in the examination space for training the machine-learning model. 
     
     
         28 . The method of  claim 16 , wherein the machine-learning model is trained to recognize food items selected from a group consisting of packaged goods, fruits, vegetables, and fresh food, the fresh food comprising one or more of curries, breads, pasta, salads, and burgers. 
     
     
         29 . The method of  claim 16 , wherein the plurality of color images is captured after placing the plurality of food items in the examination space. 
     
     
         30 . A non-transitory machine-readable storage medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising:
 illuminating an examination space for accommodating a plurality of food items in a checkout system, the examination space being defined between a base and a head portion of the checkout system;   capturing, by at least one imaging device mounted in the head portion, a plurality of color images of the plurality of food items;   applying, by a hardware processor of the checkout system, a machine-learning model to perform image recognition of the plurality of food items in the plurality of color images to identify each food item of the plurality of food items, the image recognition being based on a plurality of features of the food items that include shape, size, and color; and   presenting, on a display device of the checkout system, an identification of each of the identified food items.