Three-dimensional microscopic imaging method and system

Abstract

Provided are a 3D microscopic imaging method and a 3D microscopic imaging system. The method includes: acquiring a first PSF of a 3D sample from an object plane to a plane of a main camera sensor and a second PSF of the 3D sample from the object plane to a plane of a secondary camera sensor, and generating a first forward projection matrix corresponding to the first PSF and a second forward projection matrix corresponding to the second PSF; acquiring a light field image captured by the main camera sensor and a high resolution image captured by the secondary camera sensor; generating a reconstruction result of the 3D sample by reconstructing the light field image, the first forward projection matrix, the high resolution image and the second forward projection matrix according to a preset algorithm.

What is claimed is: 
     
       1. A 3D microscopic imaging method, comprising:
 acquiring a first Point Spread Function (PSF) of a 3D sample from an object plane to a plane of a main camera sensor and a second PSF of the 3D sample from the object plane to a plane of a secondary camera sensor, and generating a first forward projection matrix corresponding to the first PSF and a second forward projection matrix corresponding to the second PSF; 
 acquiring a light field image captured by the main camera sensor and a high resolution image captured by the secondary camera sensor; and 
 generating a reconstruction result of the 3D sample by reconstructing the light field image, the first forward projection matrix, the high resolution image and the second forward projection matrix according to a preset algorithm, wherein the preset algorithm is an iterative algorithm, comprising: 
 acquiring an initial matrix g 0  of the 3D sample with a uniformly distributed grayscale; 
 performing an iteration process for a preset number of times, the iteration process comprising: 
 performing a light field reconstruction according to a formula of
     g   new   (k) =diag( H   1   T 1) −1  diag( H   1   T  diag( H   1   g   (k) ) −1   f   1 ) g   (k) , and 
 
 performing a high resolution reconstruction according to a formula of
     g   (k+1) =diag( H   2   T 1) −1  diag( H   2   T  diag( H   2   g   new   (k) ) −1   f   2 ) g   new   (k) , 
 
 where f 1  represents the light field image and f 2  represents the high resolution image, H 1  represents the first forward projection matrix and H 2  represents the second forward projection matrix. 
 
     
     
       2. The method according to  claim 1 , wherein acquiring a first PSF of a 3D sample from an object plane to a plane of a main camera sensor and a second PSF of the 3D sample from the object plane to a plane of a secondary camera sensor, and generating a first forward projection matrix corresponding to the first PSF and a second forward projection matrix corresponding to the second PSF comprises:
 calculating the second PSF of the 3D sample from the object plane to a plane where a micro lens array is located, 
 acquiring the first PSF by multiplying the second PSF by a transmission coefficient of the micro lens array, 
 squaring a modulus of the first PSF and performing a discretization process to acquire the first forward projection matrix and squaring a modulus of the second PSF and performing the discretization process to acquire the second forward projection matrix. 
 
     
     
       3. The method according to  claim 1 , wherein acquiring a light field image captured by the main camera sensor and a high resolution image captured by the secondary camera sensor comprises:
 focusing on a plane of the 3D sample; 
 dividing a light beam from a tube lens into a first part of the light beam and a second part of the light beam by a light splitter downstream of the tube lens; 
 recording the high resolution image by using the secondary camera sensor to collect the first part of the light beam; 
 recording the light field image of a 3D scene of the 3D sample in each viewing direction by using the main camera sensor to collect the second part of the light beam. 
 
     
     
       4. A 3D microscopic imaging system, comprising:
 a microscope, 
 a light splitter, 
 a main camera sensor, 
 a secondary camera sensor, 
 a micro lens array, and 
 a controller, wherein 
 the microscope comprises a laser source, a translation stage, a dichroscope, an objective lens and a tube lens, 
 the secondary camera sensor is arranged at a first side of the light splitter and located at a first microscope image plane, 
 the light splitter, the micro lens array and the main camera sensor are arranged in line, 
 the micro lens array is arranged at a second side of the light splitter and located at a second microscope image plane, 
 the main camera sensor is located at a focal plane of the micro lens array, 
 the main camera sensor and the secondary camera sensor are connected to the controller through data lines, and 
 the controller is configured to:
 acquire a first Point Spread Function (PSF) of a 3D sample from an object plane to a plane of the main camera sensor and a second PSF of the 3D sample from the object plane to a plane of the secondary camera sensor, and generate a first forward projection matrix corresponding to the first PSF and a second forward projection matrix corresponding to the second PSF; 
 acquire a light field image captured by the main camera sensor and a high resolution image captured by the secondary camera sensor; and 
 generate a reconstruction result of the 3D sample by reconstructing the light field image, the first forward projection matrix, the high resolution image and the second forward projection matrix according to a preset algorithm. 
 
 
     
     
       5. The system according to  claim 4 , wherein the 3D sample is placed on the translation stage of the microscope, and the microscope focuses on a plane of the 3D sample. 
     
     
       6. The system according to  claim 4 , wherein the light splitter is placed downstream of the tube lens and configured to divide a light beam from the tube lens into a first part of the light beam and a second part of the light beam. 
     
     
       7. The system according to  claim 6 , wherein the first part of the light beam is collected by the secondary camera sensor and the second part of the light beam is collected by the main camera sensor. 
     
     
       8. The system according to  claim 4 , wherein the main camera sensor is configured to collect the second part of the light beam to acquire the light field image. 
     
     
       9. The system according to  claim 4 , wherein the secondary camera sensor is configured to collect the first part of the light beam to acquire the high resolution image. 
     
     
       10. The system according to  claim 4 , wherein the controller acquires the first PSF of the 3D sample from the object plane to the plane of the main camera sensor and the second PSF of the 3D sample from the object plane to the plane of the secondary camera sensor, and generates the first forward projection matrix corresponding to the first PSF and the second forward projection matrix corresponding to the second PSF by:
 calculating the second PSF of the 3D sample from the object plane to a plane where a micro lens array is located, 
 acquiring the first PSF by multiplying the second PSF by a transmission coefficient of the micro lens array, 
 squaring a modulus of the first PSF and performing a discretization process to acquire the first forward projection matrix and squaring a modulus of the second PSF and performing the discretization process to acquire the second forward projection matrix. 
 
     
     
       11. The system according to  claim 4 , wherein the preset algorithm is an iterative algorithm, comprising:
 acquiring a 3D matrix g 0  of the 3D sample with a uniformly distributed grayscale; 
 performing an iteration process for a preset number of times, the iteration process comprising: 
 performing a light field reconstruction according to a formula of PS
     g   new   (k) =diag( H   1   T 1) −1  diag( H   1   T  diag( H   1   g   (k) ) −1   f   1 ) g   (k) , 
 
 performing a high resolution reconstruction according to a formula of
     g   (k+1) =diag( H   2   T 1) −1  diag( H   2   T  diag( H   2   g   new   (k) ) −1   f   2 ) g   new   (k) , 
 
 where f 1  represents the light field image and f 2  represents the high resolution image, H 1  represents the first forward projection matrix and H 2  represents the second forward projection matrix.