Method and a control node for controlling a mining rig

Abstract

A method for controlling a mining rig in a mining environment is provided. The method includes obtaining sensor data from a plurality of sensors. The plurality of sensors includes one or more first sensors for sensing the mining environment and one or more second sensors for sensing the mining rig in the mining environment. The method further includes generating a virtual mining model based on the obtained sensor data. The virtual mining model includes a combined three-dimensional real-time representation of the mining rig and the mining environment. The combined three-dimensional real-time representation of the mining rig and the mining environment includes a coordinate system for establishing a positioning relationship of the mining rig relative to the mining environment in real time. The method further includes triggering one or more operations to be performed by the mining rig in the mining environment based on the generated virtual mining model.

1 . A method for controlling a mining rig in a mining environment, the method comprises:
 obtaining sensor data from a plurality of sensors, wherein the plurality of sensors comprises one or more first sensors for sensing the mining environment and one or more second sensors for sensing the mining rig in the mining environment,   based on the obtained sensor data, generating a virtual mining model, wherein the virtual mining model comprises a combined three-dimensional real-time representation of the mining rig and the mining environment, wherein the combined three-dimensional real-time representation of the mining rig and the mining environment comprises a coordinate system for establishing a positioning relationship of the mining rig relative to the mining environment in real time, and   triggering one or more operations to be performed by the mining rig in the mining environment based on the generated virtual mining model, wherein triggering one or more operations comprises triggering one or more real-time operations.   
     
     
         2 . The method according to  claim 1 , wherein generating the virtual mining model comprises:
 producing a visual representation of the virtual mining model, wherein the visual representation comprises a viewing position and a viewing orientation in the coordinate system, wherein the viewing position and viewing orientation in the coordinate system corresponds to a location and orientation in the mining environment, and wherein producing the visual representation of the virtual mining model fulfils a real-time condition.   
     
     
         3 . The method according to  claim 2 , wherein the method further comprises:
 in response to an operator observing the visual representation of the virtual mining model, obtaining one or more control inputs from the operator, and   wherein triggering the one or more operations to be performed by the mining rig is further based on the one or more control inputs obtained from the operator.   
     
     
         4 . The method according to  claim 3 , wherein the one or more control inputs is obtained from the operator by means of any one out of:
 the operator inputs the one or more control inputs by using a remote operator station comprising a joystick and a plurality of buttons, wherein the virtual representation of the virtual mining model is presented to the operator on at least one display,   the operator inputs the one or more control inputs by using a Virtual Reality (VR) environment, wherein the virtual representation of the virtual mining model is presented to the operator in the VR environment, and   the operator inputs the one or more control inputs by using an Augmented Reality (AR) environment, wherein the virtual representation of the virtual mining model is presented to the operator in the AR environment.   
     
     
         5 . The method according to  claim 3 , wherein the method further comprises:
 based on the one or more control inputs, triggering an update of the visual representation of the virtual mining model such that the updated visual representation comprises any one or both of an updated viewing position and an updated viewing orientation in the coordinate system.   
     
     
         6 . The method according to  claim 3 , wherein the one or more obtained control inputs from the operator is associated with the operator performing one or more virtual operations in the virtual mining model, and wherein the one or more virtual operations correspond to the one or more operations to be performed by the mining rig in the mining environment. 
     
     
         7 . The method according to  claim 3 , wherein generating the virtual mining model, comprises generating the coordinate system comprised in the combined three-dimensional real-time representation of the mining rig and the mining environment by:
 obtaining one or more relative distances between one or more of the plurality of sensors and a reference point of the mining rig,   based on the obtained sensor data and based on the one or more relative distances between the one or more of the plurality of sensors and the reference point of the mining rig, estimating one or more distances from the mining rig and the mining environment, and   based on the one or more distances from the mining rig and the mining environment, determining one or more positions associated with the mining rig and the mining environment, wherein the one or more positions are determined relative to the reference point of the mining rig.   
     
     
         8 . The method according to  claim 1 , wherein the plurality of sensors comprises at least one high resolution sensor and at least one low resolution sensor, wherein the at least one high resolution sensor captures high resolution sensor data of the mining environment, and wherein the at least one low resolution sensor captures low resolution data for monitoring real-time movements of the mining rig. 
     
     
         9 . The method according to  claim 8 , wherein the high resolution sensor data captured by the at least one high resolution sensor is of a higher resolution than the low resolution sensor data captured by the low resolution sensor, and wherein the at least one low resolution sensor captures sensor data at a higher rate than the at least one high resolution sensor. 
     
     
         10 . The method according to  claim 8 , wherein the at least one high resolution sensor comprises any one or more out of:
 a rotating laser scanner,   a plurality of two-dimensional cameras,   a time-of-flight camera,   a solid state Light Detection and Ranging (Lidar) sensor, and   a structured light three-dimensional scanner.   
     
     
         11 . The method according to  claim 8 , wherein the at least one high resolution sensor comprises the plurality of two-dimensional cameras, and wherein generating the virtual mining model further comprises deriving the combined three-dimensional real-time representation of the mining rig and the mining environment, based on a plurality of two-dimensional images captured by the plurality of two-dimensional cameras. 
     
     
         12 . The method according to  claim 1 , wherein the method further comprises:
 obtaining a three-dimensional representation of the mining rig, and wherein generating the virtual mining model is further based on the three-dimensional representation of the mining rig.   
     
     
         13 . The method according to  claim 12 , wherein the obtained three-dimensional representation of the mining rig is a Computer-Aided Design (CAD) drawing of the mining rig. 
     
     
         14 . The method according to  claim 1 , wherein generating the virtual mining model further comprises deriving information of any one or both of the mining rig and the mining environment based on a combination of sensor data obtained from at least two differing sensors in the plurality of sensors. 
     
     
         15 . The method according to  claim 1 , wherein the mining rig comprises one or more booms, and when triggering the one or more operations to be performed by the mining rig comprises operating the one or more booms in the mining environment in real time. 
     
     
         16 . A control node configured to control a mining rig in a mining environment, the control node is further configured to:
 obtain sensor data from a plurality of sensors, wherein the plurality of sensors comprises one or more first sensors for sensing the mining environment and one or more second sensors for sensing the mining rig in the mining environment,   based on the obtained sensor data, generate a virtual mining model, wherein the virtual mining model comprises a combined three-dimensional real-time representation of the mining rig and the mining environment, wherein the combined three-dimensional real-time representation of the mining rig and the mining environment comprises a coordinate system for establishing a positioning relationship of the mining rig relative to the mining environment in real time, and   trigger one or more operations to be performed by the mining rig in the mining environment based on the generated virtual mining model, wherein to trigger one or more operations comprises to trigger one or more real-time operations.   
     
     
         17 . The control node according to  claim 16 , wherein the control node is further configured to perform the operations of:
 obtaining sensor data from a plurality of sensors, wherein the plurality of sensors comprises one or more first sensors for sensing the mining environment and one or more second sensors for sensing the mining rig in the mining environment,   based on the obtained sensor data, generating a virtual mining model, wherein the virtual mining model comprises a combined three-dimensional real-time representation of the mining rig and the mining environment, wherein the combined three-dimensional real-time representation of the mining rig and the mining environment comprises a coordinate system for establishing a positioning relationship of the mining rig relative to the mining environment in real time, and   triggering one or more operations to be performed by the mining rig in the mining environment based on the generated virtual mining model, wherein triggering one or more operations comprises triggering one or more real-time operations.   
     
     
         18 . A computer program comprising instructions, which when executed by a processor, causes the processor to perform actions according to  claim 1 . 
     
     
         19 . (canceled)