Object observation system and method of controlling object observation system

Abstract

An object observation system of the present invention has an observation apparatus for observing a body to be examined, a three-dimensional image recording apparatus for recording three-dimensional images, which are obtained in advance, of the body to be examined, and an image constructing apparatus for constructing a three-dimensional image based on images in synchronization with the observation apparatus, which are recorded in the three-dimensional image recording apparatus.

1 . An object observation system, comprising: 
 an observation apparatus for observing a body to be examined;    a three-dimensional image recording apparatus for recording three-dimensional images, which are obtained in advance, of the body to be examined; and    an image constructing apparatus for constructing a three-dimensional image based on images in synchronization with the observation apparatus, which are recorded in the three-dimensional image recording apparatus.    
     
     
         2 . An object observation system according to  claim 1 , further comprising a positional relationship detecting portion for detecting a relative positional information of the observation apparatus with respect to the body to be examined.  
     
     
         3 . An object observation system according to  claim 2 , 
 wherein the observation apparatus is an endoscope, and    the image constructing apparatus constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting portion.    
     
     
         4 . An object observation system according to  claim 3 , further comprising a recording portion for recording the three-dimensional image constructed by the image constructing apparatus.  
     
     
         5 . An object observation system according to  claim 4 , 
 wherein the recording portion records an image resulting from synthesis of an image of the endoscope and the three-dimensional image.    
     
     
         6 . An object observation system according to  claim 2 , 
 wherein the positional relationship detecting portion has at least one of an amount-of-insertion detecting section for detecting an amount of insertion of the observation apparatus into a body to be examined, an insertion position detecting portion for detecting a position of insertion of the observation apparatus into a body to be examined, an inclination-angle-of-insertion detecting portion for detecting an inclination angle of insertion of the observation apparatus into a body to be examined, a position-of-point-of-vision specifying portion for specifying a position of a point of vision with respect to a part of concern of the body to be examined, and an observation direction detecting portion for detecting information indicating an observation direction of the observation apparatus.    
     
     
         7 . An object observation system according to  claim 6 , comprising: 
 a first three-dimensional image data creating portion for creating three-dimensional image data relating to the body to be examined corresponding to an image observed by the observation apparatus based on angle information of the inclination angle of insertion detected by the inclination-angle-of-insertion detecting portion; and    a second three-dimensional image data creating portion for creating three-dimensional image data relating to the body to be examined corresponding to a plane having a predetermined angle with respect to an observation image plane of the observation apparatus.    
     
     
         8 . An object observation system according to  claim 7 , 
 wherein the observation apparatus is an endoscope, and    the image constructing apparatus constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting portion.    
     
     
         9 . An object observation system according to  claim 6 , comprising: 
 a first three-dimensional image data creating portion for creating three-dimensional image data relating to the body to be examined corresponding to an observation image plane of the observation apparatus based on angle information of the inclination angle of insertion detected by the inclination-angle-of-insertion detecting portion; and    a second three-dimensional image data creating portion for creating three-dimensional image data relating to the body to be examined corresponding to an observation image plane of the observation apparatus based on the axial angle specified by the axial angle specifying portion.    
     
     
         10 . An object observation system according to  claim 9 , 
 wherein the observation apparatus is an endoscope, and    the image constructing apparatus constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting portion.    
     
     
         11 . An object observation system according to  claim 1 , further comprising: 
 a three-dimensional image display portion, which can display a three-dimensional image by the image constructing apparatus and an image observed by the observation apparatus; and    a three-dimensional image display control portion for controlling display of the three-dimensional image display portion.    
     
     
         12 . An object observation system according to  claim 11  in which the observation apparatus is an endoscope, the system further comprising: 
 a positional relationship detecting portion for detecting a relative positional relationship of the endoscope with respect to the body to be examined.    
     
     
         13 . An object observation system according to  claim 12 , 
 wherein the positional relationship detecting portion is provided in a trocar holding the endoscope therethrough.    
     
     
         14 . An object observation system according to  claim 12 , 
 wherein the positional relationship detecting portion is provided in a member provided to the arm of an operator.    
     
     
         15 . An object observation system, comprising: 
 an endoscope having an insert portion, which can be used for observing a body to be examined;    at least one treating device for performing a treatment on the body to be examined;    a first detecting portion for detecting information indicating an observation direction of the insert portion of the endoscope;    a second detecting portion for detecting information indicating a treatment direction of the treating device;    a three-dimensional image data storing portion for storing three-dimensional image data relating to the body to be examined;    a three-dimensional image data processing portion for creating first and second three-dimensional image data corresponding to the first and second detecting portions by processing the three-dimensional image data based on information detected by the first and second detecting portions;    first and second three-dimensional image display portions, which can display first and second three-dimensional images based on the first and second three-dimensional image data and an endoscopic observation image by the endoscope;    a switching section, which can selectively switch and output, to the first and second three-dimensional image display portion, the first and second three-dimensional image data from the three-dimensional image data processing portion; and    a control portion for controlling the switching section.    
     
     
         16 . An object observation system according to  claim 15 , 
 wherein the first and second detecting portions have operation portions for instructing display modes of the first and second three-dimensional image display portions.    
     
     
         17 . An object observation system according to  claim 1 , 
 wherein the image constructing apparatus has:    an image extracting portion for extracting a desired image from images recorded in the three-dimensional image recording portion; and    an image processing portion for processing the extracted image.    
     
     
         18 . An object observation system according to  claim 17 , further comprising a positional relationship detecting portion for detecting a relative positional relationship of the observation apparatus with respect to the body to be examined.  
     
     
         19 . An object observation system according to  claim 18 , 
 wherein the observation apparatus is an endoscope, and    the image constructing apparatus constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting portion.    
     
     
         20 . An object observation system according to  claim 19 , further comprising: 
 a synthesis processing portion for creating a synthesized image by synthesizing multiple extracted images created by the image extracting portion.    
     
     
         21 . An object observation system according to  claim 1 , further comprising: 
 a display portion for displaying a three-dimensional image constructed by the image constructing apparatus; and    a three-dimensional image change processing portion for performing image processing for changing an image being displayed on the display portion.    
     
     
         22 . An object observation system according to  claim 21 , 
 wherein the three-dimensional image change processing portion performs the image processing based on a position, direction or display scale of the observation apparatus.    
     
     
         23 . An object observation system according to  claim 1 , further comprising an observation information input portion for inputting focus-point information of the observation apparatus with respect to the body to be examined.  
     
     
         24 . An object observation system according to  claim 23 , 
 wherein the three-dimensional image constructing apparatus processes a three-dimensional image stored in the three-dimensional image recording apparatus based on information input from the observation information input portion.    
     
     
         25 . An object observation system according to  claim 3 , further comprising: 
 another image constructing apparatus, which is connected to the positional relationship detecting portion through a communications line, for constructing a three-dimensional image of the body to be examined based on information of the positional information received from the positional relationship detecting portion.    
     
     
         26 . An object observation system according to  claim 25 , further comprising: 
 a display portion for displaying an observation image of the observation apparatus received through the communications line; and    a support information output portion for creating support information based on the three-dimensional image constructed by the other image constructing apparatus, adding the support information to the observation image and outputting the result through the communications line.    
     
     
         27 . A control method of an object observation system, comprising the steps of: 
 observing a body to be examined by an observation apparatus;    recording three-dimensional images, which are obtained in advance, of the body to be examined in a three-dimensional image recording apparatus; and    constructing a three-dimensional image based on images in synchronization with the observation apparatus, which are recorded in the three-dimensional image recording apparatus.    
     
     
         28 . A control method of an object observation system according to  claim 27 , further comprising the step of detecting a relative positional information of the observation apparatus with respect to the body to be examined.  
     
     
         29 . A control method of an object observation system according to  claim 28 , 
 wherein the observation apparatus is an endoscope, and    the step of constructing a three-dimensional image constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting portion.    
     
     
         30 . A control method of an object observation system according to  claim 29 , further comprising the steps of recording the three-dimensional image constructed in the image constructing step in a recording portion.  
     
     
         31 . A control method of an object observation system according to  claim 30 , 
 wherein the recording step includes a step of recording an image resulting from synthesis of an image of the endoscope and the three-dimensional image.    
     
     
         32 . A control method of an object observation system according to  claim 28 , 
 wherein the positional relationship detecting step detects at least one of an amount of insertion of the observation apparatus into the body to be examined, a position of insertion of the observation apparatus into the body to be examined, an inclination angle of insertion of the observation apparatus into the body to be examined, a position of a point of vision with respect to a part of concern of the body to be examined and information indicating an observation direction of the observation apparatus.    
     
     
         33 . A control method of an object observation system according to  claim 32 , comprising: 
 a first step of creating three-dimensional image data relating to the body to be examined corresponding to an image observed by the observation apparatus based on information of the inclination angle of insertion detected by the step of detecting a positional relationship; and    a second step of creating three-dimensional image data relating to the body to be examined corresponding to a plane having a predetermined angle with respect to an observation image plane of the observation apparatus.    
     
     
         34 . A control method of an object observation system according to  claim 33 , 
 wherein the observation apparatus is an endoscope, and    the step of constructing a three-dimensional image constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting step.    
     
     
         35 . A control method of an object observation system according to  claim 32 , comprising: 
 a first step of creating three-dimensional image data relating to the body to be examined corresponding to an observation image plane of the observation apparatus based on angle information of the inclination angle of insertion detected by the positional relationship detecting portion; and    a second step of creating three-dimensional image data relating to the body to be examined corresponding to an observation image plane of the observation apparatus based on the axial angle specified by the axial angle specifying portion.    
     
     
         36 . A control method of an object observation system according to  claim 35 , 
 wherein the observation apparatus is an endoscope, and    the step of constructing a three-dimensional image constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting step.    
     
     
         37 . A control method of an object observation system according to  claim 27 , further comprising the steps of: 
 displaying a three-dimensional image by the three-dimensional image constructing step and an image observed by the observation apparatus; and    controlling display of the display step.    
     
     
         38 . A control method of an object observation system according to  claim 37  in which the observation apparatus is an endoscope, the method further comprising the step of: 
 detecting a relative positional relationship of the endoscope with respect to the body to be examined.    
     
     
         39 . A control method of an object observation system according to  claim 38 , 
 wherein the positional relationship detecting step is performed by using a detecting portion included in a trocar holding the endoscope therethrough.    
     
     
         40 . A control method of an object observation system according to  claim 38 , 
 wherein the positional relationship detecting step is performed by using a member detecting portion provided to the arm of an operator.    
     
     
         41 . A control method of an object observation system including an endoscope having an insert portion, which can be used for observing a body to be examined and at least one treating device for performing a treatment on the body to be examined, the method comprising: 
 a first step of detecting information indicating an observation direction of the insert portion of the endoscope;    a second step of detecting information indicating a treatment direction of the treating device;    a step of storing three-dimensional image data relating to the body to be examined;    a step of creating first and second three-dimensional image data corresponding to the first and second detecting steps by processing the three-dimensional image data based on information detected by the first and second detecting steps;    a step of displaying first and second three-dimensional images based on the first and second three-dimensional image data and an endoscopic observation image by the endoscope;    a step of selectively switching the first and second three-dimensional image data for the display step; and    a step of controlling the switching step.    
     
     
         42 . A control method of an object observation system according to  claim 41 , 
 wherein the first and second steps have operation steps for instructing display modes of the display steps.    
     
     
         43 . A control method of an object observation system according to  claim 27 , 
 wherein the three-dimensional image constructing step has:    a step of extracting a desired image from images recorded in the three-dimensional image recording portion; and    a step of processing the extracted image.    
     
     
         44 . A control method of an object observation system according to  claim 43 , further comprising a step of detecting a relative positional relationship of the observation apparatus with respect to the body to be examined.  
     
     
         45 . A control method of an object observation system according to  claim 44 , 
 wherein the observation apparatus is an endoscope, and    the three-dimensional image constructing step constructs the three-dimensional image in accordance with the distal end of the endoscope based on information on the positional relationship detected by the positional relationship detecting step.    
     
     
         46 . A control method of an object observation system according to  claim 45 , further comprising: 
 a step of creating a synthesized image by synthesizing multiple extracted images created by the extracting step.    
     
     
         47 . A control method of an object observation system according to  claim 27 , further comprising the steps of: 
 displaying a three-dimensional image constructed by the three-dimensional image constructing step; and    performing image processing for changing an image being displayed by the display step.    
     
     
         48 . A control method of an object observation system according to  claim 47 , 
 wherein the image processing step performs the image processing based on a position, direction or display scale of the observation apparatus.    
     
     
         49 . A control method of an object observation system according to  claim 27 , further comprising the step of inputting focus-point information of the observation apparatus with respect to the body to be examined.  
     
     
         50 . A control method of an object observation system according to  claim 49 , 
 wherein the three-dimensional image constructing step processes a three-dimensional image stored in the three-dimensional image recording apparatus based on information input at the focus point information input step.    
     
     
         51 . A control method of an object observation system according to  claim 29 , further comprising: 
 another image constructing apparatus, which is connected to the positional relationship detecting portion through a communications line, for constructing a three-dimensional image of the body to be examined based on information of the positional information received from the positional relationship detecting portion.    
     
     
         52 . A control method of an object observation system according to  claim 51 , further comprising: 
 a display portion for displaying an observation image of the observation apparatus received through the communications line; and    a support information output portion for creating support information based on the three-dimensional image constructed by the other image constructing apparatus, adding the support information to the observation image and outputting the result through the communications line.