Systems, methods, and media for presenting biophysical simulations in an interactive mixed reality environment

Abstract

Systems, methods, and media for presenting biophysical simulations in an interactive mixed reality environment are provided. In some embodiments, a system comprises: a head mounted display comprising: a transparent display; sensors; and a processor programmed to: receive medical imaging data associated with a subject; receive, from a server, information useable to visualize a simulation of biophysical processes and a subject-specific anatomical model based on the medical imaging data; cause a visualization of the simulation to be presented, via the transparent display, in connection with the medical imaging data and an instrument in a first position; receive, from the server, updated information useable to visualize an updated simulation with the instrument in a second position; and cause a visualization of the updated simulation to be presented with the instrument presented in the second position.

The invention claimed is: 
     
         1 . A system for presenting biophysical simulations in an interactive mixed reality environment, comprising:
 a head mounted display comprising:
 a transparent display; 
 a plurality of sensors; and 
 at least one processor, wherein the at least one processor is programmed to:
 receive medical imaging data associated with a subject, the medical imaging data comprising T1-weighted magnetic resonance imaging (MRI) data, T2-weighted MRI data, or computed tomography (CT) data corresponding to the subject's brain tissue; 
 receive, from a server, information useable to visualize a simulation of one or more biophysical processes associated with the subject in connection with a subject-specific anatomical model associated with the subject based on the medical imaging data with an instrument in a first position, wherein the server generates the subject-specific anatomical model based on the medical imaging data by using a nonlinear transformation matrix and a warp field with an anatomical atlas; 
 cause a visualization of the simulation to be presented, via the transparent display, in connection with the medical imaging data with the instrument presented in the first position; 
 receive, from the server, updated information useable to visualize an updated simulation of the one or more biophysical processes in connection with the subject-specific anatomical model based on the medical imaging data with the instrument in a second position; and 
 cause a visualization of the updated simulation to be presented, via the transparent display, in connection with the medical imaging data with the instrument presented in the second position. 
 
   
     
     
         2 . The system of  claim 1 , wherein the visualization includes a representation of the instrument. 
     
     
         3 . The system of  claim 1 , wherein the instrument is a deep brain stimulation (DBS) electrode comprising multiple electrode contacts. 
     
     
         4 . The system of  claim 3 , wherein the at least one processor is further programmed to:
 receive, via an input device, input to manipulate an activation state of one or more of the contacts of the DBS electrode.   
     
     
         5 . The system of  claim 1 , wherein the instrument comprises;
 a stereo-encephalography (SEEG) electrode;   a convection enhanced delivery (CED) probe; or   a laser interstitial thermal therapy (LITT) probe.   
     
     
         6 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 receive, via an input device, input to manipulate a position of the instrument; and   transmit, to the server, instructions based on the input.   
     
     
         7 . The system of  claim 6 , wherein:
 the head mounted display further comprises the input device;   the input device comprises a sensor; and   the at least one processor is further programmed to:
 receive, from the sensor, information indicative of a position of an index finger of a wearer of the head mounted display, wherein the input to manipulate the position of the instrument comprises movement of the index finger from a first position to a second position. 
   
     
     
         8 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 transmit, to the server, instructions to adjust the instrument to the second position.   
     
     
         9 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 receive, via an input device, input to manipulate a position of a portion of the subject-specific anatomical model.   
     
     
         10 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 transmit, to the server, instructions to adjust a position of a portion of the subject-specific anatomical model.   
     
     
         11 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 receive, via an input device, input to change a portion of the simulation that is visualized.   
     
     
         12 . The system of  claim 1 , wherein the at least one processor is further programmed to:
 transmit, to the server, instructions to change a portion of the simulation that is visualized.   
     
     
         13 . The system of  claim 1 , wherein the anatomical atlas comprises a brain atlas. 
     
     
         14 . A method for presenting biophysical simulations in an interactive mixed reality environment, comprising:
 receiving medical imaging data associated with a subject, the medical imaging data comprising T1-weighted magnetic resonance imaging (MRI) data, T2-weighted MRI data, or computed tomography (CT) data corresponding to the subject's brain tissue;   receiving, from a server, information useable to visualize a simulation of one or more biophysical processes associated with the subject in connection with a subject-specific anatomical model associated with the subject based on the medical imaging data with an instrument in a first position, wherein the server generates the subject-specific anatomical model based on the medical imaging data by using a nonlinear transformation matrix and a warp field with an anatomical atlas;   causing a visualization of the simulation to be presented, via a transparent display of a head mounted display device, in connection with the medical imaging data with the instrument presented in the first position;   receiving, from the server, updated information useable to visualize an updated simulation of the one or more biophysical processes in connection with the subject-specific anatomical model based on the medical imaging data with the instrument in a second position; and   causing a visualization of the updated simulation to be presented, via the transparent display, in connection with the medical imaging data with the instrument presented in the second position.   
     
     
         15 . The method of  claim 14 , wherein the instrument is a deep brain stimulation (DBS) electrode comprising multiple electrode contacts, a convection enhanced delivery (CED) probe, or a laser interstitial thermal therapy (LITT) probe. 
     
     
         16 . The method of  claim 14 , wherein the anatomical atlas comprises a brain atlas. 
     
     
         17 . A non-transitory computer readable medium containing computer executable instructions that, when executed by processing circuitry, cause the processing circuitry to:
 receive medical imaging data associated with a subject, the medical imaging data comprising computed tomography (CT) data corresponding to the subject's brain tissue;   receive, from a server, information useable to visualize a simulation of one or more biophysical processes associated with the subject in connection with a subject-specific anatomical model associated with the subject based on the medical imaging data with an instrument in a first position, wherein the server generates the subject-specific anatomical model based on the medical imaging data by using a nonlinear transformation matrix and a warp field with an anatomical atlas;   cause a visualization of the simulation to be presented, via a transparent display of a head mounted display device, in connection with the medical imaging data with the instrument presented in the first position;   receive, from the server, updated information useable to visualize an updated simulation of the one or more biophysical processes in connection with the subject-specific anatomical model based on the medical imaging data with the instrument in a second position; and   cause a visualization of the updated simulation to be presented, via the transparent display, in connection with the medical imaging data with the instrument presented in the second position.   
     
     
         18 . The non-transitory computer readable medium of  claim 17 , wherein the medical imaging data comprises T1-weighted magnetic resonance imaging (MRI) data or T2-weighted MRI data corresponding to the subject's brain tissue. 
     
     
         19 . The non-transitory computer readable medium of  claim 17 , wherein the anatomical atlas comprises a brain atlas. 
     
     
         20 . The non-transitory computer readable medium of  claim 17 , wherein the instrument is a deep brain stimulation (DBS) electrode comprising multiple electrode contacts, a convection enhanced delivery (CED) probe, or a laser interstitial thermal therapy (LITT) probe.