Augmented reality-based system and method to show the location of personnel and sensors inside occluded structures and provide increased situation awareness

Abstract

An augmented reality system provides enhanced situational information to personnel located within an environment. A tracking system obtains viewpoint information corresponding to a real-time view of said environment. A processing system receives information from one or more sensors. Information includes sensor location information and status information about the environment and personnel therein. The processing system generates graphics using the sensor location information and the viewpoint information. Graphics include visual representations of said status information. A display displays the generated graphics on a display at a supervisor station that is outside of said environment such that graphics are superimposed on the real-time view.

1 . A method of providing enhanced situational information to personnel located within an environment comprising the steps of:
 receiving information from one or more sensors located within the environment, where information includes sensor location information and status information about the environment and personnel therein;   obtaining viewpoint information corresponding to an exterior real-time view of said environment;   generating graphics using said sensor location information and viewpoint information, wherein the graphics include visual representations of the status information;   displaying the generated graphics on a display at a supervisor station that is outside of said environment such that the generated graphics are superimposed on the real-time view.   
   
   
       2 . The method of  claim 1 , further comprising the step of:
 communicating situational information from the supervisor station to said personnel based on information visually represented on the display.   
   
   
       3 . The method of  claim 1 , wherein said personnel located within an environment are occluded from the exterior real-time perspective. 
   
   
       4 . The method of  claim 1 , wherein the said personnel are firefighters and said environment is a building. 
   
   
       5 . The method of  claim 1 , wherein the said personnel are soldiers and said environment is a combat zone. 
   
   
       6 . The method of  claim 2 , wherein the step of communicating situational information includes:
 sending video on the display at a supervisor station to said personnel located within the environment.   
   
   
       7 . The method of  claim 1 , wherein said visual representations include symbols representing said personnel. 
   
   
       8 . An augmented reality system for providing enhanced situational information to personnel located within an environment comprising:
 one or more sensors located within the environment;   a tracking system that obtains viewpoint information corresponding to a real-time view of said environment;   a processing system that receives information from said one or more sensors, where the information includes sensor location information and status information about the environment and personnel therein, and generates graphics using said sensor location information and said viewpoint information, wherein the graphics include visual representations of said status information;   a display that displays the generated graphics at a supervisor station that is outside of said environment such that the generated graphics are superimposed on the real-time view.   
   
   
       9 . The augmented reality system of  claim 8 , further comprising:
 a communications device that communicates situational information from the supervisor station to said personnel based on information visually represented on the display.   
   
   
       10 . The method of  claim 8 , wherein the said personnel are firefighters and said environment is a building. 
   
   
       11 . The method of  claim 8 , wherein the said personnel are soldiers and said environment is a combat zone. 
   
   
       12 . The augmented reality system of  claim 8 , wherein the situational information includes video on the display at a supervisor station to said personnel located within the environment. 
   
   
       13 . The method of  claim 8 , wherein said visual representations include symbols representing said personnel. 
   
   
       14 . A method of providing enhanced situational information to personnel located within an environment comprising the steps of:
 receiving information from one or more sensors located within the environment, where the information includes sensor location information and status information about the environment and personnel therein;   obtaining viewpoint information corresponding to a real-time view of said environment;   generating graphics using said sensor location information and viewpoint information, wherein the graphics include visual representations of the status information;   displaying the generated graphics on a display such that the generated graphics are superimposed on the real-time view.   
   
   
       15 . The method of  claim 14 , wherein the said personnel are firefighters and said environment is a building. 
   
   
       16 . The method of  claim 14 , wherein the said personnel are soldiers and said environment is a combat zone. 
   
   
       17 . A method of providing enhanced situational information about an environment comprising the steps of:
 receiving information from one or more sensors located within the environment, where the information includes sensor location information and status information about the environment;   obtaining viewpoint information corresponding to a real-time view of said environment;   generating graphics using said sensor location information and viewpoint information, wherein the graphics include visual representations of the status information;   displaying the generated graphics on a display such that the generated graphics are superimposed on the real-time view.   
   
   
       18 . The method of  claim 17 , wherein said environment is a building, said one or more sensors correspond with one or more alarms in the building, and said status information indicates whether an alarm has been triggered.