Materials and methods for long-term tracking of group-housed livestock

Abstract

The invention relates to a computer-implemented method of tracking animals is provided. Such a method typically includes recognizing, by using at least one data processor, individual animals in images of a plurality of the animals; and tracking the animals using a probabilistic tracking-by-detection process. In another aspect, a system for recognizing animals is provided. Such a system typically includes an instance detection and part localization module; a visual marker classification module; a fixed-cardinality track interpolation module; and a maximum a posteriori estimation of animal identity module.

What is claimed is: 
     
         1 . A computer-implemented method of tracking animals, the method comprising:
 recognizing, by using at least one data processor, individual animals in images of a plurality of the individual animals;   tracking the individual animals using a probabilistic tracking-by-detection process;   using association vectors generated by a classification network to evaluate a probability that an ear tag belongs to a particular animal in the plurality of the individual animals; and   initializing the probability with a uniform probability, wherein for each ear tag and each detected instance of an animal in the plurality of the individual animals, the probability is modified using a weighted summation of an output of the classification network and the uniform probability.   
     
     
         2 . The method of  claim 1 , further comprising:
 using fixed cardinality of the individual animals to generate a continuous set of tracks of the plurality of the individual animals, and   using a forward-backward algorithm to assign ear-tag identification probabilities to each detected animal in the plurality of the individual animals, wherein the plurality of the individual animals have ear tags and the method further comprises using the classification network to assign unique identification to each detected animal.   
     
     
         3 . The method of  claim 1 , further comprising using a deep, fully-convolutional network to detect each animal in the plurality of the individual animals as a collection of anatomical features. 
     
     
         4 . The method of  claim 1 , further comprising:
 using ear tags for visual identification of each animal in the plurality of the individual animals; and   using a maximum a posteriori (MAP) forward-backward process to assign ear tag identities to each animal by merging ear tag classification probabilities with identified frame-to-frame movements for the animal.   
     
     
         5 . The method of  claim 1 , further comprising processing frames of a video using an instance detection and part localization module to detect target animals in the plurality of the individual animals and determine image coordinates of each detected target animal. 
     
     
         6 . The method of  claim 1 , wherein recognizing individual animals comprises (i) recognizing a plurality of body parts of the plurality of the individual animals based on processing image data of the plurality of the individual animals and (ii) determining first estimated positions of the plurality of body parts, wherein the plurality of body parts include a plurality of types of body parts. 
     
     
         7 . The method of  claim 6 , further comprising (i) recognizing a plurality of first associations of body parts based on processing the image data and (ii) determining relative positions of body parts in each first association, wherein each first association identifies a relationship between a body part of an animal and at least one other body part of the same animal. 
     
     
         8 . The method of  claim 1 , wherein the individual animals are tracked with an average precision and recall greater than 50%. 
     
     
         9 . The method of  claim 1 , further comprising providing a probabilistic framework for merging classification likelihoods to detections of the individual animals. 
     
     
         10 . The method of  claim 1 , further comprising:
 applying at least one recognition module to at least one image of animals to recognize body parts of the animals, in which the body parts include a plurality of types of body parts, and the at least one recognition module outputs first estimated positions of the recognized body parts in the at least one image;   applying the at least one recognition module to the at least one image of animals to recognize first associations of body parts of the animals, in which each first association of body parts associates a body part of an animal with at least one other body part of the same animal, and the at least one recognition module outputs relative positions of the body parts in each recognized first association of body parts;   determining, based on the first estimated positions of the recognized body parts and the relative positions of the body parts in the recognized first associations of body parts, second associations of body parts in which each second association of body parts associates a recognized body part of an animal with at least one other recognized body part of the same animal; and   recognizing individual animals in the at least one image based on the second associations of body parts of the animals.   
     
     
         11 . The method of  claim 1 , further comprising:
 applying at least one recognition module to at least one image of animals to recognize individual body parts of the animals,
 wherein the at least one recognition module outputs first estimated locations of the recognized individual body parts in the at least one image; 
   applying the at least one recognition module to the at least one image of animals to recognize groups of body parts of the animals,
 wherein the at least one recognition module outputs relative positions of the body parts in each recognized group of body parts; 
   determining associations of recognized individual body parts based on (i) the first estimated locations of the recognized individual body parts of the animals and (ii) the relative positions of the body parts in the recognized groups of body parts; and   recognizing individual animals in the at least one image based on the associations of recognized individual body parts of the animals.   
     
     
         12 . The method of  claim 1 , further comprising:
 applying at least one recognition module to at least one image of pigs to recognize body parts of the pigs, in which the body parts include shoulder portions, tail portions, left ears, and right ears of the pigs,
 wherein the at least one recognition module outputs first estimated locations of the recognized shoulder portions, the recognized tail portions, the recognized left ears, and the recognized right ears in the at least one image; 
   applying the at least one recognition module to the at least one image of pigs to recognize pairs of body parts of the pigs, including recognizing a pair of shoulder portion and tail portion of each of at least some of the pigs, recognizing a pair of shoulder portion and left ear of each of at least some of the pigs, and recognizing a pair of shoulder portion and right ear of each of at least some of the pigs, and
 wherein the at least one recognition module outputs a position of the tail portion relative to the corresponding shoulder portion in each recognized pair of shoulder portion and tail portion, a position of the left ear relative to the corresponding shoulder portion in each recognized pair of shoulder portion and left ear, and a position of the right ear relative to the corresponding shoulder portion in each recognized pair of shoulder portion and right ear; 
   determining, for each of at least some of the recognized shoulder portions, an association with a recognized tail portion, a recognized left ear, and a recognized right ear of the same pig based on (i) the first estimated positions of the recognized shoulder portions, tail portions, left ears and right rears, and (ii) the relative positions of the tail portion and the corresponding shoulder portion in each recognized pair of shoulder portion and tail portion, the relative positions of the left ear and the corresponding shoulder portion in each recognized pair of shoulder portion and left ear, and the relative position of the right ear and the corresponding shoulder portion in each recognized pair of shoulder portion and right ear; and   recognizing individual pigs in the at least one image of pigs based on the associations of recognized shoulder portions with recognized tail portions.   
     
     
         13 . The method of  claim 1 , wherein recognizing individual animals comprises using a convolutional detector to recognize the individual animals to provide visible key points of the individual animals, and
 wherein the probabilistic tracking-by-detection process comprises using, as input, the visible key points of the individual animals provided by the convolutional detector to track the animals over a period of time.   
     
     
         14 . The method of  claim 7 , further comprising:
 determining, based on the first estimated positions of the plurality of body parts and the relative positions of body parts in the first associations, a plurality of second associations of body parts, wherein each second association identifies a relationship between a body part of an animal and at least one other body part of the same animal; and   recognizing the individual animals in the image data based on the second associations.   
     
     
         15 . The method of  claim 1 , further comprising:
 determining a first probability of observation given a specific identity for the particular animal;   determining a second probability of the particular animal transitioning between frames from one location to another; and   using the first and second probabilities to calculate a Maximum A-Posteriori (MAP) estimate of an identity of the particular animal.   
     
     
         16 . A system for tracking animals, comprising:
 at least one data processor; and   at least one storage device storing instructions that when executed by the at least one data processor, cause the system to perform operations that include:
 recognizing, by using the at least one data processor, individual animals in images of a plurality of the individual animals; 
 tracking the individual animals using a probabilistic tracking-by-detection process; 
 using association vectors generated by a classification network to evaluate a probability that an ear tag belongs to a particular animal in the plurality of the individual animals; and 
 initializing the probability with a uniform probability, wherein for each ear tag and each detected instance of an animal in the plurality of the individual animals, the probability is modified using a weighted summation of an output of the classification network and the uniform probability.