System and method for identifying items

Abstract

In variants, a method for item recognition can include: optionally calibrating a sampling system, determining visual data using the sampling system, determining a point cloud, determining region masks based on the point cloud, generating a surface reconstruction for each item, generating image segments for each item based on the surface reconstruction, and determining a class identifier for each item using the respective image segments.

We claim: 
     
         1 . A checkout system, comprising:
 a set of sensors configured to sample a set of measurements of a set of items; and   a processing system configured to:
 determine a set of geometric features for each item based on the set of measurements; 
 determine a set of appearance features for each item based on the set of measurements; and 
 identify each item based on the respective set of geometric features and the respective set of appearance features. 
   
     
     
         2 . The checkout system of  claim 1 , wherein the set of appearance features for each item is determined from a set of visual segments, determined from the set of measurements, for the item. 
     
     
         3 . The checkout system of  claim 2 , wherein the set of visual segments for the item is determined by:
 determining a geometric representation of each item; and   segmenting each visual segment for the item from each measurement of the set of measurements, using the geometric representation.   
     
     
         4 . The checkout system of  claim 3 , wherein segmenting the measurement comprises reprojecting the geometric representation of the item into the measurement. 
     
     
         5 . The checkout system of  claim 4 , wherein the measurement comprises a color image. 
     
     
         6 . The checkout system of  claim 3 , wherein the geometric representation comprises a convex hull of at least a portion of the item depicted in the set of measurements. 
     
     
         7 . The checkout system of  claim 1 , wherein the set of geometric features comprises a geometric shape of the item. 
     
     
         8 . The checkout system of  claim 1 , wherein the set of appearance features comprises a color of the item. 
     
     
         9 . The checkout system of  claim 1 , further comprising a housing comprising:
 a static base configured to support the set of items during measurement sampling; and   a head portion statically connected to and overhanging the base, wherein the set of sensors are mounted to the head portion.   
     
     
         10 . The checkout system of  claim 9 , wherein the set of measurements are sampled from at least three different angles relative to the static base. 
     
     
         11 . The checkout system of  claim 9 , wherein the set of sensors comprises a first upper corner camera set, a second upper corner camera set, and a top-down camera set. 
     
     
         12 . The checkout system of  claim 1 , further comprising:
 determining checkout information for each identified item; and   interfacing with a point of sale system to check out the set of items based on the respective checkout information.   
     
     
         13 . A checkout system, comprising:
 a housing, comprising:
 a static base configured to support a set of items; and 
 a head portion statically connected to and overhanging the static base; 
   a set of sensors statically mounted to the head portion and configured to sample a set of measurements of the set of items; and   a processing system configured to:
 determine a geometric representation for each item based on the set of measurements; and 
 identify each item within the set of items based on the respective geometric representation. 
   
     
     
         14 . The checkout system of  claim 13 , wherein identifying each item based on the respective geometric representation comprises:
 determining a set of visual segments from the set of measurements using the geometric representation for the item;   extracting a set of appearance features for the item from the set of visual segments; and   identifying the item based on the set of appearance features.   
     
     
         15 . The checkout system of  claim 14 , further comprising extracting a set of geometric features from the geometric representation for each item, wherein the item is further identified based on the set of geometric features. 
     
     
         16 . The checkout system of  claim 15 , wherein the set of geometric features comprises shape information for the respective item. 
     
     
         17 . The checkout system of  claim 13 , wherein determining the geometric representation for each item comprises:
 determining a height map for the set of items;   segmenting the height map into individual item segments corresponding to each of the set of items; and   generating the geometric representation for each item based on the respective item segment.   
     
     
         18 . The checkout system of  claim 13 , wherein the set of sensors comprise a first color camera set mounted in a first upper corner of the head portion; a second color camera set mounted in a second upper corner of the head portion, and a color camera mounted to a surface of the head portion proximal the base. 
     
     
         19 . The checkout system of  claim 18 , wherein the first color camera set and the second color camera set each comprise stereocameras. 
     
     
         20 . The checkout system of  claim 13 , wherein the processing system is configured to connect to a point of sale system, wherein the point of sale system is configured to check out the set of items.