Systems and methods for detection, analysis, isolation and/or harvesting of biological objects

Abstract

Systems and methods provide for detection and controlled interaction with one or more objects. The system can include an imaging subsystem ( 20 ), a tool subsystem ( 26 ) containing one or more tools, a stage subsystem ( 16 ) and a control system ( 40 ). The control system ( 40 ) can integrate controls for each of the other subsystems, which controls can be implement desired functions over a variety of process parameters to perform the controlled interaction.

What is claimed is: 
     
         1 . A computer-implemented method, comprising:
 identifying at least one biological object of interest in image data representing an image of the object of interest residing in media;   analyzing the image data to determine a distribution of pixels or voxels of the image that corresponds to the object of interest;   interacting with the object of interest in the media through the use of at least one tool, wherein the at least one tool includes a hollow orifice through which fluid and/or medium can flow in a controllable manner; and   controlling the interaction of the at least one tool with the object of interest based on the distribution of pixels or voxels, wherein the distribution of pixels or voxels is updated based on the image data to enable corresponding adjustments to the controlling of the interaction.   
     
     
         2 . The method of  claim 1 , wherein the image data includes images acquired over time and analyzing the image further comprises:
 dividing each of a plurality of images into respective discrete spatial regions based on location metadata that is embedded in the image data;   analyzing a respective distribution of pixels or voxels within each of the spatial regions to determine changes in each respective distribution of pixels or voxels during the interaction or in a sequence of multiple interactions; and   adjusting process parameters to control the at least one tool to interact with a given spatial region containing the object of interest based on the determined changes in the respective distribution of pixels or voxels for the given spatial region.   
     
     
         3 . The method of  claim 2 , further comprising:
 determining a given interaction protocol for each spatial region of the media containing an object of interest based on the corresponding distribution of pixels or voxels for each respective spatial region and/or another feature of an object of interest determined based on an analysis of the distribution of pixels or voxels, in which the interaction protocol describes one or more actions, each having a defined set of parameters that characterize each action that is to be performed by the at least one tool in a sequence or concurrently.   
     
     
         4 . The method of  claim 1 , further comprising:
 storing a library of interaction protocols;   selecting a given interaction protocol from the library of interaction protocols in response to a user input defining a desired type of interaction; and   setting process parameters for the given interaction protocol based on the distribution of pixels or voxels, wherein the given interaction protocol is selected from the library of interaction protocols to implement an interaction scheme based on an analysis of the image data acquired for a plurality of objects located within the media on a stage and in response to the user input, in which the user input specifies at least two of the desired type of interaction, one or more process parameters associated with the interaction, a minimum size of the object, a type of the media in which the objects are being cultured, and material properties of the media in which the objects are being cultured.   
     
     
         5 . The method of  claim 4 , further comprising adjusting the process parameters for the given interaction protocol for a subsequent interaction with one or more objects based on evaluating an effectiveness of executing the given interaction protocol to harvest the object of interest based on changes in the image data determined from the analysis of the image data acquired after executing the given interaction protocol. 
     
     
         6 . The method of  claim 1 , wherein the object of interest is a cell or group of cells, and the analysis of the image data further comprises determining one or more morphological features for the object of interest based on values of pixels or voxels and/or the distribution of pixels or voxels for the object of interest. 
     
     
         7 . The method of  claim 6 , further comprising:
 acquiring a sequence of images over time for a localized set of one or more cells or group of cells at respective time intervals;   determining changes in the distribution of pixels or voxels that occur between two or more images in the sequence of images;   detecting at least one biological event that occurs for the localized set of cells or group of cells based on the changes, wherein the at least one biological event includes at least one of proliferation, migration, a change in physical properties, a change in chemical properties, a change in anabolic properties, a change in catabolic properties or a change in secretory properties; and   controlling the at least one tool to interact with the object of interest based on the at least one biological event that is detected.   
     
     
         8 . The method of  claim 6 , wherein the morphological features determined for a given cell or group of cells include at least one of a size of the given cell or group of cells, a shape of the given cell or group of cells, an optical density of the given cell or group of cells, auto-fluorescence of the given cell or group of cells, presence or absence of cell surface markers in the given cell or group of cells, presence or absence of specific extracellular matrix components for the given cell or group of cells, a presence of specific enzymatic activity for the given cell or group of cells, supercellular features located near the given cell or group of cells, minerals formed near the given cell or group of cells, extracellular matrix or proteins formed near the given cell or group of cells, and the size and configuration of junction points between cells. 
     
     
         9 . The method of  claim 1 , wherein the at least one tool comprises an aspiration tool having a tip member and controlling the interaction further comprises:
 controlling a flow rate and/or volume of the object of interest and/or the media into or out of the aspiration tool based on the distribution of pixels or voxels for a region of the image that includes the object of interest and/or the media.   
     
     
         10 . The method of  claim 1 , wherein the interaction includes one of picking the object of interest, changing a composition of the media, sampling a portion of the object of interest, adding bioactive agents to the media, or moving the object of interest to a different location, in which the respective interaction occurs over multiple action phases, and the method further comprises:
 analyzing the distribution of pixels and voxels acquired before a previous action phase and after the previous action phase to determine process parameters for a next action phase, in which the process parameters for the respective interaction include object location, a flow rate, a volume of material to draw into the tool, and a distance between the tip and the object of interest; and   controlling the at least one tool in the next action phase based on the process parameters.   
     
     
         11 . The method of  claim 1 , wherein the interaction includes a series of interactions over time, the method further comprising:
 quantitatively analyzing image data acquired prior to each interaction relative to image data acquired after each interaction to provide quantitative comparison data;   storing process parameters associated with each interaction; and   storing documentation data to describe each interaction based on the quantitative comparison data and information describing process parameters for each respective interaction.   
     
     
         12 . The method of  claim 11 , wherein the method further comprises configuring the process parameters for a subsequent interaction based on an evaluation of the documentation data from one or more prior interactions. 
     
     
         13 . A system comprising:
 a stage;   a stage motion system adapted to move the stage along orthogonal axes of the stage;   at least one tool supported above a surface of the stage, in which the tool includes a body extending toward the stage and terminating in a distal end thereof and the body has a hollow orifice through which fluid and/or medium can flow in a controllable manner;   an imaging device supported above the surface of the stage and having a known spatial position relative to the stage and spaced apart from the tool, the imaging device configured to capture at least one image for a field of view and provide image data representative of the at least one image; and   a control system configured to at least:   identify at least one biological object of interest in the image data representing an image of the object of interest residing in media;   analyze the image data to determine a distribution of pixels or voxels of the image that corresponds to the object of interest; and   control the at least one tool to interact with the object of interest in the media on the stage based on the distribution of pixels or voxels, wherein the distribution of pixels or voxels is updated based on the image data to enable corresponding adjustments to the control.   
     
     
         14 . The system of  claim 13 , wherein the image data includes images acquired over time and the control system is further configured to:
 divide each of a plurality of images into respective discrete spatial regions based on location metadata that is embedded in the image data;   analyze a respective distribution of pixels or voxels within each of the spatial regions to determine changes in each respective distribution of pixels or voxels during a given interaction or in a sequence of multiple interactions; and   adjust process parameters to control the at least one tool to interact with a given spatial region containing the object of interest based on the determined changes in the respective distribution of pixels or voxels for the given spatial region.   
     
     
         15 . The system of  claim 14 , the control system is further configured to:
 determine a given interaction protocol for each spatial region of the media containing an object of interest based on the corresponding distribution of pixels or voxels for each respective spatial region and/or another feature of an object of interest determined based on an analysis of the distribution of pixels or voxels, in which the interaction protocol describes one or more actions, each having a defined set of parameters that characterize each action that is to be performed by the at least one tool in a sequence or concurrently.   
     
     
         16 . The system of  claim 13 , wherein the control system includes non-transitory memory to store a library of interaction protocols, and the control system is further configured to:
 select a given interaction protocol from the library of interaction protocols in response to a user input defining a desired type of interaction between the tool and the object of interest; and   set process parameters for the given interaction protocol based on the distribution of pixels or voxels, wherein the given interaction protocol is selected from the library of interaction protocols to implement an interaction scheme based on an analysis of the image data acquired for a plurality of objects located within the media on the stage and in response to the user input, in which the user input specifies at least two of the desired type of interaction, one or more process parameters associated with the control of the interaction between the tool and the object of interest, a minimum size of the object, a type of the media in which the objects are being cultured, and material properties of the media in which the objects are being cultured; and   adjust the process parameters for the given interaction protocol for a subsequent interaction with one or more objects based on evaluating an effectiveness of executing the given interaction protocol to harvest the object of interest based on changes in the image data determined from the analysis of the image data acquired after executing the given interaction protocol.   
     
     
         17 . The system of  claim 13 , wherein the object of interest includes a cell or group of cells, and the analysis of the image data further comprises determining one or more morphological features for the object of interest based on values of pixels or voxels and/or the distribution of pixels or voxels for the object of interest, and the control system is further configured to:
 acquire a sequence of images over time for a localized set of one or more cells or group of cells at respective time intervals;   determine changes in the distribution of pixels or voxels that occur between two or more images in the sequence of images;   detect at least one biological event that occurs for the localized set of cells or group of cells based on the changes, wherein the at least one biological event includes at least one of proliferation, migration, a change in physical properties, a change in chemical properties, a change in anabolic properties, a change in catabolic properties or a change in secretory properties; and   control the at least one tool to interact with the object of interest based on the at least one biological event that is detected.   
     
     
         18 . The system of  claim 13 , wherein the interaction includes one of picking the object of interest, changing a composition of the media, sampling a portion of the object of interest, moving the object of interest to different location in the media, in which the respective interaction occurs over multiple action phases, and the control system is further configured to:
 analyze the distribution of pixels and voxels acquired before a previous action phase and after the previous action phase to determine process parameters for a next action phase, in which the process parameters for the respective interaction include object location, a flow rate, a volume of material to draw into the tool, and a distance between the tip and the object of interest; and   control the at least one tool to interact with the object of interest in the next action phase based on the process parameters.   
     
     
         19 . The system of  claim 13 , wherein the control system is further configured to:
 quantitatively analyze image data acquired prior to each interaction between the tool and a given object of interest relative to image data acquired after each respective interaction to provide quantitative comparison data;   store process parameters associated with each interaction;   store documentation data to describe each interaction based on the quantitative comparison data and information describing process parameters for each respective interaction.   
     
     
         20 . A computer-implemented method, comprising:
 identifying a plurality of objects of interest in image data representing an image of the objects of interest residing in media on a stage;   analyzing a distribution of pixels or voxels in the image data for each of the plurality of objects of interest;   determining an object profile for each of the plurality of objects of interest based on the distribution of pixels or voxels, in which the object profile includes object classification data for each of the plurality of objects of interest that describes at least one of size, morphology, density, brightness, texture, gradients, proximity, shape, and pattern of the respective object of interest; and   configuring process parameters for controlling an interaction between at least one tool and a given object of interest of the plurality of objects of interest based on the object profile determined for the given object of interest; and   controlling the at least one tool to interact with the given object of interest based on the process parameters.