Identification and quantification of needle and seed displacement departures from treatment plan

Abstract

A placement plan is developed for the placement of radioactive seeds or other therapeutic agents in a prostate for brachytherapy. The placement plan is made available to an intraoperative tracking interface which also shows a live ultrasound image of the needle or catheter placement in the prostate. Position errors can be detected and corrected. Techniques for merging of orthogonal ultrasound image sets, classification of malignancy through neural networks, and statistical detection of seeds in an ultrasound image can be used.

We claim:  
     
         1 . A method for identifying and quantifying departures in placement of needles or catheters from intended placements in a treatment plan for treating a bodily organ, the needles or catheters carrying therapeutic agents for insertion into the bodily organ for use in the treatment plan, the method comprising: 
 (a) inputting the intended placements into an intraoperative tracking interface;    (b) for at least one needle or catheter, calculating a difference in an x-y plane between the intended placement of that needle or catheter and an actual placement of that needle or catheter;    (c) calculating, from the difference calculated in step (b), a position error for each of the therapeutic agents;    (d) adjusting positions of therapeutic agents along each needle or catheter by an amount determined in accordance with the position error calculated in step (c); and    (e) determining positions of a deposited number of therapeutic agents through imaging.    
     
     
         2 . The method of  claim 1 , further comprising: 
 (f) assigning a confidence level to each of the positions determined in step (e); and    (g) for each therapeutic agent whose position is assigned a confidence level exceeding a predetermined threshold, recalculating a dosimetry associated with that therapeutic agent.    
     
     
         3 . The method of  claim 2 , further comprising: 
 (h) adjusting positions of the therapeutic agents having low confidence values assigned in step (t) in accordance with x-ray imaging;    (i) recalculating a dosimetry associated with all of the therapeutic agents; and    (j) assigning a confidence level to the dosimetry recalculated in step (i).    
     
     
         4 . The method of  claim 1 , wherein the intraoperative tracking interface permits an operator to select a needle or catheter for step (b).  
     
     
         5 . The method of  claim 1 , wherein the intraoperative tracking interface displays at least one isodose plot of the bodily organ.  
     
     
         6 . The method of  claim 1 , wherein step (e) is performed through real-time ultrasound imaging of the bodily organ.  
     
     
         7 . The method of  claim 6 , wherein: 
 the intraoperative tracking interface permits an operator to select a needle or catheter; and    the real-time ultrasound imaging is performed in a direction of the needle or catheter selected by the operator.    
     
     
         8 . The method of  claim 7 , wherein the real-time ultrasound imaging results in a column of ultrasound images along the needle or catheter selected by the operator.  
     
     
         9 . The method of  claim 8 , wherein the column of ultrasound images undergoes gray-scale preprocessing.  
     
     
         10 . The method of  claim 9 , wherein the gray-scale preprocessing corrects a gray-scale histogram of the column of ultrasound images to produce gray-scale corrected ultrasound images.  
     
     
         11 . The method of  claim 10 , wherein the gray-scale corrected ultrasound images are used to find locations of the therapeutic agents along the needle or catheter selected by the operator.  
     
     
         12 . The method of  claim 11 , wherein the locations of the therapeutic agents found in the gray-scale corrected ultrasound images are used for further correction of the positions of the therapeutic agents.  
     
     
         13 . The method of  claim 11 , wherein a number of therapeutic agents whose locations are found in the gray-scale corrected ultrasound images is compared to a number of therapeutic agents which are actually along the needle or catheter selected by the operator.  
     
     
         14 . The method of  claim 13 , wherein, when the numbers of therapeutic agents are not equal, a column width of the column of ultrasound images is changed, and the column of ultrasound images is taken again.  
     
     
         15 . The method of  claim 1 , wherein the bodily organ is a prostate.  
     
     
         16 . A system for identifying and quantifying departures in the placement of needles or catheters from intended placements in a treatment plan for treating a bodily organ, the needles or catheters carrying therapeutic agents for insertion into the bodily organ for use in the treatment plan, the system comprising: 
 an imaging device for imaging the bodily organ;    an intraoperative tracking interface comprising a display; and    a computing device, in electronic communication with the intraoperative tracking interface, for: 
 (a) inputting the intended placements into the intraoperative tracking interface;  
 (b) for at least one needle or catheter, calculating a difference in an x-y plane between the intended placement of that needle or catheter and an actual placement of that needle or catheter;  
 (c) calculating, from the difference calculated in step (b), a position error for each of the therapeutic agents;  
 (d) adjusting positions of therapeutic agents along each needle or catheter by an amount determined in accordance with the position error calculated in step (c); and  
 (e) determining positions of a deposited number of therapeutic agents through imaging carried out through the imaging device.  
   
     
     
         17 . The system of  claim 16 , wherein the computing device also performs the following: 
 (f) assigning a confidence level to each of the positions determined in step (e); and    (g) for each therapeutic agent whose position is assigned a confidence level exceeding a predetermined threshold, recalculating a dosimetry associated with that therapeutic agent.    
     
     
         18 . The system of  claim 16 , wherein the imaging device comprises an x-ray imaging device, and wherein the computing device further performs the following: 
 (h) adjusting positions of the therapeutic agents having low confidence values assigned in step (f) in accordance with x-ray imaging carried out through the x-ray imaging device;    (i) recalculating a dosimetry associated with all of the therapeutic agents; and    (j) assigning a confidence level to the dosimetry recalculated in step (i).    
     
     
         19 . The system of  claim 16 , wherein the intraoperative tracking interface comprises an input device which permits the operator to select a needle or catheter for step (b).  
     
     
         20 . The system of  claim 16 , wherein the intraoperative tracking interface displays at least one isodose plot of the bodily organ.  
     
     
         21 . The system of  claim 16 , wherein: 
 the imaging device comprises a real-time ultrasound imaging device; and    the computing device performs step (e) through real-time ultrasound imaging of the bodily organ by the real-time ultrasound imaging device.    
     
     
         22 . The system of  claim 21 , wherein: 
 the intraoperative tracking interface permits an operator to select a needle or catheter; and    the real-time ultrasound imaging is performed in a direction of the needle or catheter selected by the operator.    
     
     
         23 . The system of  claim 22 , wherein the real-time ultrasound imaging results in a column of ultrasound images along the needle or catheter selected by the operator.  
     
     
         24 . The system of  claim 23 , wherein the computing device performs gray-scale preprocessing on the column of ultrasound images.  
     
     
         25 . The system of  claim 24 , wherein the gray-scale preprocessing corrects a gray-scale histogram of the column of ultrasound images to produce gray-scale corrected ultrasound images.  
     
     
         26 . The system of  claim 25 , wherein the gray-scale corrected ultrasound images are used to find locations of the therapeutic agents along the needle or catheter selected by the operator.  
     
     
         27 . The system of  claim 26 , wherein the locations of the therapeutic agents found in the gray-scale corrected ultrasound images are used for further correction of the positions of the therapeutic agents.  
     
     
         28 . The system of  claim 26 , wherein a number of therapeutic agents whose locations are found in the gray-scale corrected ultrasound images is compared to a number of therapeutic agents which are actually along the needle or catheter selected by the operator.  
     
     
         29 . The system of  claim 28 , wherein, when the numbers of therapeutic agents are not equal, a column width of the column of ultrasound images is changed, and the column of ultrasound images is taken again.  
     
     
         30 . A method for generating a 3D orthogonal compound ultrasound image volume of a patient prostate, with high contrast and detailed information of prostate anatomy, tumor lesions, and implanted radioactive sources if any, comprising the steps of: 
 (a) providing a bi-plane ultrasound transducer having a transverse imaging function and a longitudinal imaging function;    (b) using the transverse imaging function of the bi-plane ultrasound transducer, moving the transducer linearly within the patient rectum in the cranio-caudal direction to acquire a first image set of images of transverse view of the prostate;    (c) using the longitudinal imaging function of the bi-plane transducer, rotating the transducer within the rectum to acquire a second image set of images of sagittal-coronal views of the prostate, which are orthogonal to the transverse view images obtained in step (b);    (d) measuring the positions and orientations of the ultrasound transducer in real-time while the images are taken in steps (b) and (c);    (e) reconstructing the first and second image sets to 3D image volumes with a common voxel size;    (f) registering the two 3D image volumes reconstructed in step (e); and    (g) combining the two 3D image volumes which have been registered in step (f) by using an intelligent compounding method.    
     
     
         31 . The method of  claim 30 , wherein step (d) is performed by using an electromagnetic sensing system.  
     
     
         32 . The method of  claim 30 , wherein step (d) is performed by using a mechanical encoder.  
     
     
         33 . The method of  claim 30 , wherein step (d) is performed by using an optical positioning system.  
     
     
         34 . The method of  claim 30 , in which the transducer comprises a switch for switching the transducer between the transverse imaging function and the longitudinal imaging function, and wherein at least one of steps (b) and (c) comprises pressing the switch.  
     
     
         35 . The method of  claim 30 , step (d) comprises registering actual coordinates and orientations of the transducer with each image at the instant of acquisition of said each image, whereby mis-registration is eliminated even when the transducer movement is very rapid.  
     
     
         36 . The method of  claim 30 , wherein the longitudinal images are reconstructed into a 3D image volume by using the destination-oriented method for every x-y plane, and then built up by using a look-up table repeatedly for each successive plane.  
     
     
         37 . The method of  claim 30 , wherein the two 3D image volumes are registered by using non-rigid image registration methodology of grayscale data sets.  
     
     
         38 . The method of  claim 30 , wherein the two 3D image volumes are registered by using a maximization of mutual information (MMI) of a joint histogram formed between the two 3D image volumes.  
     
     
         39 . The method of  claim 38 , wherein a feature of high image intensity is used to reduce the number of grayscale bins when computing the joint histogram.  
     
     
         40 . The method of  claim 30 , wherein step (g) is performed by use of a compounding weight, the compounding weight being based on regional image information content.  
     
     
         41 . A method for generating a 3D orthogonal compound ultrasound image volume of a human organ, the method comprising: 
 (a) providing an ultrasound transducer capable of taking a first set of images in a first direction and a second set of images in a second direction which is orthogonal or at an angle to the first direction;    (b) taking the first set of images of the organ;    (c) taking the second set of images of the organ;    (d) registering the first set of images with the second set of images; and    (e) combining the first and second sets of images which have been registered in step (d) to produce the ultrasound image volume.    
     
     
         42 . An ultrasound imaging system for generating 3D orthogonal compound ultrasound images for the diagnosis and treatment of prostate cancer, comprising: 
 (a) a bi-plane ultrasound transducer capable of transverse imaging and longitudinal imaging of the prostate to produce a transverse image set and a longitudinal image set;    (b) a measuring system for measuring the position and orientation of thetransducer;    (c) a rigid bio-compatible sheath which covers the ultrasound transducer to isolate motion of the transducer from the prostate; and    (d) an image processing work station for acquiring the transverse image set and the longitudinal image set and for producing an image of the prostate from the transverse image set and the longitudinal image set.    
     
     
         43 . A normalized statistical tumor probability model (NSTPM) aimed to help effective biopsy for tumor detection in an organ of a patient, the model comprising: 
 an input layer for receiving raw data concerning the patient;    at least one hidden layer for extracting features from the raw data; and    an output layer for weighting and combining the features extracted by the at least one hidden layer to produce a classification of the organ for the biopsy.    
     
     
         44 . The normalized statistical tumor probability model (NSTPM) of  claim 43 , wherein the organ is a prostate.  
     
     
         45 . A method to re-optimize a dosimetry plan for implantation of therapeutic agents into an organ of a patient after partial or complete implantation of an original implant plan, the method comprising: 
 (a) determining locations of therapeutic agents which have been implanted in the partial or complete implantation;    (b) evaluating a dosimetry of the locations determined in step (a); and    (c) re-optimizing the dosimetry plan in accordance with the dosimetry evaluated in step (b).    
     
     
         46 . The method of  claim 45 , wherein the locations are determined in step (a) such that each of the locations has an uncertainty level.  
     
     
         47 . The method of  claim 46 , wherein step (a) is performed automatically using a computer.  
     
     
         48 . The method of  claim 46 , wherein step (a) is performed by taking an ultrasound image and performing a human inspection of the ultrasound image.  
     
     
         49 . The method of  claim 45 , wherein step (c) is performed by a computer algorithm.  
     
     
         50 . The method of  claim 45 , wherein step (c) is performed by human inspection through trial-and-error.  
     
     
         51 . A method of programming an artificial neural network to recognize objects in images, the method comprising: 
 (a) providing first and second sets of image data, the objects being present in at least one of the first and second sets of image data;    (b) calculating numerical values of features in the first and second sets of image data;    (c) supplying the numerical values of the features to the artificial neural network and controlling the artificial neural network to recognize the objects;    (d) identifying correct and incorrect identifications of the objects made by the artificial neural network in step (c); and    (e) using the correct and incorrect identifications to train the artificial neural network through feedback.    
     
     
         52 . The method of  claim 51 , wherein the correct and incorrect identifications comprise true positives, false negatives, true negatives and false positives.  
     
     
         53 . The method of  claim 51 , wherein the objects are therapeutic agents in an organ of a patient.  
     
     
         54 . The method of  claim 53 , wherein the numerical values of the features comprise a correlation coefficient of one of the objects with a template.  
     
     
         55 . The method of  claim 54 , wherein a plurality of templates are provided for different orientations of the objects.  
     
     
         56 . The method of  claim 53 , wherein the numerical values of the features comprise statistical parameters of selected sub-images in the first and second sets of image data.  
     
     
         57 . The method of  claim 56 , wherein the statistical parameters comprise frequency domain parameters.  
     
     
         58 . The method of  claim 56 , wherein the statistical parameters comprise geometry parameters.  
     
     
         59 . The method of  claim 1 , wherein step (a) comprises receiving a manual adjustment of a position of at least one of the needles or catheters.  
     
     
         60 . The method of  claim 59 , wherein the manual adjustment comprises an expansion of a pattern of the intended placements.  
     
     
         61 . The method of  claim 59 , wherein the manual adjustment comprises a shift of a pattern of the intended placements.  
     
     
         62 . The method of  claim 59 , wherein the manual adjustment comprises an adjustment of a single one of the needles or catheters.  
     
     
         63 . The method of  claim 1 , wherein step (e) is performed using a global search method.  
     
     
         64 . The method of  claim 63 , wherein the global search method is used to provide candidate locations of the therapeutic agents to correct a bias in a tracking or segmentation algorithm.  
     
     
         65 . The method of  claim 63 , wherein the global search method is used to provide candidate locations of the therapeutic agents, and wherein the candidate locations of the therapeutic agents are used with an original plan of the needles or catheters to locate actual positions of the needles or catheters.  
     
     
         66 . The method of  claim 1 , wherein the global search method is performed on a grayscale transform of an ultrasound image.  
     
     
         67 . A method for adjusting a treatment plan for treating a bodily organ, the treatment plan comprising insertion of needles or catheters carrying therapeutic agents into the bodily organ, the method comprising: 
 (a) providing a computing device with a user interface;    (b) entering positions of the needles or catheters into the computing device;    (c) taking at least one image of the bodily organ and inputting the at least one image into the computing device; and    (d) using the user interface to adjust at least one of the positions entered in step (b) in accordance with the at least one image input in step (c).    
     
     
         68 . The method of  claim 67 , wherein the at least one image comprises an ultrasound image.  
     
     
         69 . The method of  claim 67 , wherein step (d) comprises expanding a pattern of the positions.  
     
     
         70 . The method of  claim 67 , wherein step (d) comprises shifting a pattern of the positions.  
     
     
         71 . The method of  claim 67 , wherein step (d) comprises adjusting a single one of the positions.  
     
     
         72 . The method of  claim 67 , further comprising: 
 (e) after step (d), performing an automatic tracking of the needles or catheters in accordance with the at least one position adjusted in step (d).    
     
     
         73 . A method for locating a therapeutic agent which have been inserted into a bodily organ, the method comprising: 
 (a) providing a computing device with a user interface;    (b) taking at least one image of the bodily organ and inputting the at least one image into the computing device;    (c) drawing a boundary of the bodily organ;    (d) performing a global search in a portion of the at least one image contained within the boundary for at least one local extremum; and    (e) determining a location of the therapeutic agent in accordance with the at least one local extremum.    
     
     
         74 . The method of  claim 73 , wherein step (c) is performed manually.  
     
     
         75 . The method of  claim 73 , wherein step (c) is performed automatically.  
     
     
         76 . The method of  claim 73 , wherein the at least one image comprises at least one ultrasound image.  
     
     
         77 . The method of  claim 73 , wherein the at least one local extremum is a local maximum.  
     
     
         78 . The method of  claim 73 , wherein step (e) comprises: 
 (i) determining a candidate location of the therapeutic agent; and    (ii) using the candidate location to correct a bias in a location of the therapeutic agent determined by another technique.    
     
     
         79 . The method of  claim 78 , wherein the other technique comprises a needle tracking and segmentation algorithm.  
     
     
         80 . The method of  claim 73 , wherein step (e) comprises: 
 (i) determining a candidate location of the therapeutic agent; and    (ii) using the candidate location and a planned needle pattern to determine a final needle pattern.