Microscope system and control method thereof

Abstract

A microscope system comprises a microscope body, an image sensor mounted on the microscope body, and a stage that moves in an X-axis direction and a Y-axis direction, and places a slide as an observation object. The stage includes a mark used to indicate a stage reference position. The microscope system obtains positions of the stage in the X-axis direction and the Y-axis direction, and detects the stage reference position indicated by a mark provided on the stage and a slide reference position indicated by a mark provided on a slide placed on the stage. The microscope system manages the position of the stage by coordinates based on the slide reference position if the slide reference position is detected, and manages the position of the stage by coordinates based on the stage reference position if the slide reference position is not detected.

The invention claimed is: 
     
       1. A microscope system comprising:
 a microscope body; 
 an image sensor mounted on the microscope body via a mounting unit and configured to capture an observation image of an observation object under a microscope; 
 a translational stage configured to move translationally in an X-axis direction and a Y-axis direction, which are orthogonal to each other, and place a slide as the observation object, the transitional stage including a mark used to indicate a translational stage reference position; 
 an obtaining unit configured to obtain positions of the translational stage in the X-axis direction and the Y-axis direction; 
 an adjusting unit configured to adjust an alignment of an axial direction indicated by a mark provided on the slide placed on the translational stage to any one of the X-axis direction and the Y-axis direction of the translational stage; 
 a detecting unit configured to detect, from an image captured by the image sensor, the translational stage reference position indicated by the mark provided on the translational stage and a slide reference position indicated by the mark provided on the slide placed on the translational stage; and 
 a managing unit configured to, in a case in which the slide reference position is detected, manage the positions of the translational stage obtained by the obtaining unit by stage coordinate value based on the slide reference position, and in a case in which the slide reference position is not detected, managing the positions of the translational stage by the stage coordinate value based on the translational stage reference position, 
 wherein the adjusting unit adjusts an alignment of the axial direction indicated by the mark provided on the slide to align with an axial direction defined by a pixel arrangement of the image sensor, and the detecting unit detects the slide reference position after the adjustment is performed. 
 
     
     
       2. The system according to  claim 1 , wherein the managing unit uses one of the translational stage reference position and the slide reference position according to an objective lens used for observation by the microscope body. 
     
     
       3. The system according to  claim 1 , wherein the adjusting unit comprises:
 a rotary stage configured to rotate on the translational stage around an axis in a Z-axis direction orthogonal to the X-axis direction and the Y-axis direction, with the slide being placed on the translational stage via the rotary stage; and 
 a control unit configured to rotate the rotary stage so as to cause an axial direction obtained from an image of the mark provided on the slide, which is captured by the image sensor, to align with one of an X-axis direction and a Y-axis direction of the image sensor. 
 
     
     
       4. The system according to  claim 3 , wherein the mounting unit includes a rotation mechanism configured to rotate the mounted image sensor about an axis in a Z direction orthogonal to the X-axis direction and the Y-axis direction, and
 wherein the adjusting unit causes an axial direction obtained from an image of the mark provided on the translational stage, which is captured by the image sensor, to align with one of the X-axis direction and the Y-axis direction of the image sensor before the alignment adjustment. 
 
     
     
       5. The system according to  claim 3 , wherein the adjusting unit divides an image captured by the image sensor into strip-shaped partial regions, each strip-shaped partial region having a width not less than one pixel, and detects the axial direction indicated by the mark provided on the slide based on a barycentric position of each strip-shaped partial region. 
     
     
       6. The system according to  claim 5 , wherein the mark that provides the axial direction is formed from a plurality of lines having different widths, and the plurality of lines are arranged to be axisymmetrical. 
     
     
       7. The system according to  claim 1 , further comprising a first calculation unit configured to detect positions of two marks provided on one of the translational stage and the slide and having a known actual distance of an interval or two patterns included in one mark, thereby calculating a first coefficient used for conversion between the stage coordinate values and the actual distance. 
     
     
       8. The system according to  claim 7 , wherein the first calculation unit calculates the first coefficient used for conversion between the stage coordinate values and the actual distance based on the positions of the translational stage obtained by the obtaining unit when a predetermined position of an imaging field of the image sensor is caused to align with each of the two marks having the known actual distance of the interval or the two patterns included in the one mark. 
     
     
       9. The system according to  claim 1 , further comprising a second calculation unit configured to calculate a second coefficient used for conversion between a pixel distance and the actual distance from an image captured while putting, in an imaging field of the image sensor, two patterns provided on one of the translational stage and the slide and having the known actual distance of the interval. 
     
     
       10. The system according to  claim 1 , further comprising a recording unit configured to record the image captured by the image sensor and additional information including stage coordinate values based on the slide reference position, which represent the positions of the translational stage when capturing the image, in association with each other. 
     
     
       11. The system according to  claim 10 , wherein the stage coordinate values based on the slide reference position, which are included in the additional information, represent an actual distance from the slide reference position. 
     
     
       12. The system according to  claim 10 , wherein the additional information further includes a first coefficient used for conversion between stage coordinate values and an actual distance, and a second coefficient used for conversion between a pixel distance and the actual distance in the image. 
     
     
       13. The system according to  claim 12 , further comprising a control unit configured to, in accordance with display of the image, move the translational stage to an observation position when capturing the image based on the actual distance obtained by converting, using the first coefficient, the stage coordinate values based on the slide reference position included in the additional information of the image. 
     
     
       14. The system according to  claim 10 , wherein the recording unit records one of the additional information and link information to the additional information in one of a header and a footer of an image file that records the image. 
     
     
       15. The system according to  claim 1 , further comprising a recording unit configured to record the image captured by the image sensor and additional information including stage coordinate values based on one of the slide reference position and the translational stage reference position, which represent the position of the translational stage when capturing the image, and information representing which one of the slide reference position and the translational stage reference position is a reference position in association with each other. 
     
     
       16. The system according to  claim 1 , wherein the image is associated with additional information including coordinate values of the translational stage based on the slide reference position when capturing the image and a coefficient used for conversion between a pixel distance and an actual distance of the image sensor, and
 wherein the system further comprises:
 a calculation unit configured to convert, out of a moving direction and a moving amount designated on a screen of a display that displays the image, the moving amount into the pixel distance and calculate the actual distance corresponding to the moving amount using the coefficient; and 
 a moving unit configured to move the translational stage based on the moving direction and the actual distance calculated by the calculation unit. 
 
 
     
     
       17. The system according to  claim 16 , further comprising a connection unit configured to connect an image displayed on the display before the movement of the translational stage and an image captured after the movement of the translational stage. 
     
     
       18. The system according to  claim 1 , wherein the image is associated with additional information including coordinate values of the stage when capturing the image and a coefficient used for conversion between a pixel distance and an actual distance of the image sensor, and
 wherein the system further comprises:
 a calculation unit configured to calculate a moving direction and a moving amount of the translational stage based on the position obtained by the obtaining unit in accordance with movement of the translational stage in the X direction and the Y direction; and 
 an updating unit configured to convert the moving amount into the pixel distance based on the coefficient and the actual distance corresponding to the moving amount, and updating display contents of a display that displays the image based on the actual distance and the moving direction. 
 
 
     
     
       19. The system according to  claim 1 , further comprising a generation unit configured to connect a plurality of images captured by the image sensor based on the stage coordinate values representing the positions of the translational stage when capturing each image, and generate an image in a range larger than an imaging field of the image sensor. 
     
     
       20. The system according to  claim 1 , wherein the obtaining unit includes an X-axis sensor configured to detect a position of the translational stage in the X-axis direction, and a skew detecting sensor arranged at a predetermined interval in the Y-axis direction from the X-axis sensor, and
 wherein the system further comprises a skew detecting unit configured to detect a skew based on a difference between the positions of the translational stage detected by the X-axis sensor and the positions of the translational stage obtained by the skew sensor. 
 
     
     
       21. The system according to  claim 20 , further comprising a skew correction unit configured to correct the skew by rotating the image obtained by the image sensor based on the skew detected by the skew detecting unit. 
     
     
       22. The system according to  claim 1 , further comprising a display control unit configured to cause a display unit to display, in a superimposed manner, a plurality of images of the observation object obtained by the microscope system, which are images of a plurality of slides obtained using a plurality of adjacent tissue slices as the observation object. 
     
     
       23. The system according to  claim 1 , further comprising a display control unit configured to cause a display unit to display, in a superimposed manner, a plurality of images of an observation object obtained by the microscope system, which are a plurality of images in different staining states. 
     
     
       24. A control method by a controller that controls a microscope including:
 a microscope body; 
 an image sensor mounted on the microscope body via a mounting unit and configured to capture an observation image of an observation object under the microscope; and 
 a translational stage configured to move transitionally in an X-axis direction and a Y-axis direction, which are orthogonal to each other, and place a slide as the observation object and the translational stage including a mark used to indicate a translational stage reference position, the method comprising: 
 obtaining positions of the translational stage in the X-axis direction and the Y-axis direction; 
 adjusting an alignment of an axial direction indicated by a mark provided on the slide placed on the translational stage to any one of the X-axis direction and the Y-axis direction of the translational stage; 
 detecting, from an image captured by the image sensor, the translational stage reference position indicated by the mark provided on the translational stage and a slide reference position indicated by the mark provided on the slide placed on the translational stage; and 
 in a case in which the slide reference position is detected, managing the obtained position of the translational stage by a stage coordinate value based on the slide reference position, and in a case in which the slide reference position is not detected, managing the positions of the translational stage by a stage coordinate value based on the translational stage reference position, 
 wherein the adjusting includes adjusting the axial direction indicated by the mark provided on the slide to align with an axial direction defined by a pixel arrangement of the image sensor, and the detecting is performed after the adjustment. 
 
     
     
       25. A non-transitory computer readable storage medium storing a program that causes a computer to execute a control method by a controller that controls a microscope including:
 a microscope body; 
 an image sensor mounted on the microscope body via a mounting unit and configured to capture an observation image of an observation object under the microscope; and 
 a translational stage configured to move translationally in an X-axis direction and a Y-axis direction, which are orthogonal to each other, and place a slide as the observation object and including a mark used to indicate a translational stage reference position, the method comprising: 
 obtaining positions of the translational stage in the X-axis direction and the Y-axis direction; 
 adjusting an alignment of an axial direction indicated by a mark provided on the slide to any one of the X-axis direction and the Y-axis direction of the translational stage; 
 detecting, from an image captured by the image sensor, the translational stage reference position indicated by the mark provided on the translational stage and a slide reference position indicated by the mark provided on the slide placed on the translational stage; and 
 in a case in which the slide reference position is detected, managing the obtained positions of the translational stage by a stage coordinate value based on the slide reference position, and in a case in which the slide reference position is not detected, managing the positions of the translational stage by a stage coordinate value based on the translational stage reference position, 
 wherein the adjusting includes adjusting the axial direction indicated by the mark provided on the slide to align with an axial direction defined by a pixel arrangement of the image sensor, and the detecting is performed after the adjustment.