System with 3D user interface integration

Abstract

Disclosed is a system comprising a handheld device and at least one display. The handheld device is adapted for performing at least one action in a physical 3D environment; wherein the at least one display is adapted for visually representing the physical 3D environment; and where the handheld device is adapted for remotely controlling the view with which the 3D environment is represented on the display.

The invention claimed is: 
     
       1. A scanning system for scanning a 3D environment, the scanning system comprising:
 a handheld device including an optical scanner, wherein the 3D environment to be scanned is selected by pointing the optical scanner at the 3D environment; and 
 at least one display remotely connected to the handheld device,
 wherein the handheld device is adapted for performing at least one scanning action in a physical 3D environment, and the at least one display is adapted for visually representing the physical 3D environment; and 
 
 the handheld device includes a 3D user interface for remotely controlling the display to adjust the view with which the 3D environment is represented on the display, 
 wherein the handheld device comprises at least one motion sensor, and 
 wherein the at least one motion sensor is a sensor that directly detects motion. 
 
     
     
       2. A The scanning system according to  claim 1 , wherein the handheld device is adapted to record the 3D geometry of the 3D environment. 
     
     
       3. A The scanning system according to  claim 1 , wherein the user interface handheld device includes means an actuator for manually switching between performing the at least one scanning action and remotely controlling the view. 
     
     
       4. The system according to  claim 1 , wherein the handheld device comprises at least one motion sensor. 
     
     
       5. The scanning system according to claim  4  1, wherein the view of the 3D environment represented in the at least one display is at least partly determined by the at least one motion sensor. 
     
     
       6. The scanning system according to claim  4  1, wherein functionality of the 3D user interface comprises a use of gestures. 
     
     
       7. The scanning system according to  claim 6 , wherein the gestures are detected by the at least one motion sensor. 
     
     
       8. The scanning system according to claim  4  1, wherein the user-interface is other than the at least one motion sensor handheld device further comprises at least one of an infrared sensor, a range sensor, or a proximity sensor. 
     
     
       9. The scanning system according to  claim 1 , wherein the handheld device is adapted to change a viewing angle with which the 3D environment is represented on the at least one display. 
     
     
       10. The scanning system according to  claim 1 , wherein the handheld device is adapted to change a magnification factor with which the 3D environment is represented on the at least one display. 
     
     
       11. The scanning system according to  claim 1 , wherein the handheld device is an intra-oral 3D scanner. 
     
     
       12. The scanning system according to  claim 1 , wherein the handheld device includes a surgical instrument. 
     
     
       13. The scanning system according to  claim 1 , wherein the handheld device includes a mechanical tool. 
     
     
       14. The scanning system according to  claim 1 , wherein the handheld device is an in-ear 3D scanner. 
     
     
       15. The scanning system according to  claim 1 , wherein the at least one display is defined as a first display, and where the system further comprises a second display. 
     
     
       16. The scanning system according to  claim 15 , wherein the second display indicates where the handheld device is positioned relative to the 3D environment. 
     
     
       17. The scanning system according to  claim 15 , wherein the first display and/or the second display provides instructions for the operator. 
     
     
       18. The scanning system according to  claim 1 , wherein audible information is provided to the operator. 
     
     
       19. A system comprising:
 a handheld device and at least one display; 
 wherein the handheld device is adapted for switching between performing at least one action in a physical 3D environment, wherein the at least one display is adapted for visually representing the physical 3D environment; and remotely controlling the display to adjust the view with which the 3D environment is represented on the display; 
 wherein the handheld device is an intra-oral 3D scanner and the at least one action performed in the physical 3D environment is scanning and that the view is remotely controlled by at least one motion sensor arranged in the handheld device, and wherein an actuator provided on the handheld device switches between performing the at least one action and remotely controlling the view, and wherein the at least one motion sensor is a sensor that directly detects motion. 
 
     
     
       20. The scanning system according to claim 1, wherein the at least one motion sensor is an accelerometer, gyro, or magnetometer. 
     
     
       21. The scanning system according to claim 1, wherein the at least one motion sensor is adapted for taking the movement of the scanner into account while performing the scanning. 
     
     
       22. The scanning system according to claim 1, wherein the system comprises at least two motion sensors and wherein the at least two motion sensors provide sensor fusion. 
     
     
       23. The scanning system according to claim 1, wherein the at least one motion sensor is the 3D user interface for remotely controlling the display, wherein the view on the display is determined by moving the handheld scanner. 
     
     
       24. The scanning system according to claim 23, wherein moving the handheld scanner to point down causes the view on the display to be a downward viewing angle. 
     
     
       25. The scanning system according to claim 1, wherein the handheld device further comprises a user-interface element. 
     
     
       26. The scanning system according to claim 25, wherein the user-interface element comprises a touch-sensitive element, a button, a scroll-wheel, or a proximity sensor. 
     
     
       27. The scanning system according to claim 25, wherein the user-interface element provides more than one user input. 
     
     
       28. The scanning system according to claim 25, wherein the at least one motion sensor and/or the user-interface element are utilized in a workflow. 
     
     
       29. The system according to claim 19, wherein the handheld device is adapted to change a magnification factor of the view represented on the at least one display which is determined by the motion of the operator's hand holding the handheld device, through the use of the at least one motion sensor. 
     
     
       30. The scanning system according to claim 1, wherein the at least one display is arranged on a cart. 
     
     
       31. The scanning system according to claim 1, wherein the at least one display is divided into multiple regions. 
     
     
       32. The system according to claim 19, wherein the handheld device further comprises a user interface element,
 wherein switching to remotely controlling the view puts the handheld device into a controller mode,   wherein holding the user interface element and/or the actuator on the handheld device in conjunction with moving the handheld device determines the view of the 3D environment on the display in accordance with signals from the motion sensor.   
     
     
       33. A scanning system for scanning a 3D environment, the scanning system comprising:
 a handheld device including an optical scanner, wherein the 3D environment to be scanned is selected by pointing the optical scanner at the 3D environment; and   at least one display remotely connected to the handheld device,
 wherein the handheld device is adapted for performing at least one scanning action in a physical 3D environment, and the at least one display is adapted for visually representing the physical 3D environment; and 
 wherein the handheld device includes at least one motion sensor for remotely controlling the display to adjust the view with which the 3D environment is represented on the display; and 
 wherein the at least one motion sensor is an accelerometer, gyro, or magnetometer. 
   
     
     
       34. The scanning system according to claim 33, wherein the handheld device further comprises at least two user interface elements. 
     
     
       35. The scanning system according to claim 34, wherein the at least two user interface elements comprises at least one button and a touch-sensitive element, and wherein the display is on a cart. 
     
     
       36. The scanning system according to claim 35, wherein the at least one button and the touch-sensitive element provides more than one user input. 
     
     
       37. The scanning system according to claim 36, wherein at least one of the user input provides for manually switching between performing the at least one scanning action and remotely controlling the view. 
     
     
       38. The scanning system according to claim 37, wherein switching to remotely controlling the view puts the handheld device into a controller mode, wherein holding at least one user interface element on the handheld device in conjunction with moving the handheld device determines the view of the 3D environment on the display in accordance with signals from the motion sensor. 
     
     
       39. The scanning system according to claim 37, wherein switching to remotely controlling the view outs the handheld device into a controller mode and wherein when in controller mode, moving the handheld device down results in showing the view of the 3D environment from a downward viewing angle on the display. 
     
     
       40. The scanning system according to claim 1, wherein the at least one display is a 3D display, whereby a 3D representation of the 3D environment is displayed on the 3D display; and wherein the handheld device is an intra-oral 3D scanner or an in-ear 3D scanner. 
     
     
       41. The scanning system according to claim 40, wherein the 3D display is configured to project stereoscopic image pairs of the 3D representation. 
     
     
       42. The scanning system according to claim 40, wherein the 3D display is volumetric display configured to create a 3D effect of the 3D representation. 
     
     
       43. The system according to claim 19, wherein the at least one display is a 3D display, whereby a 3D representation of the 3D environment is displayed on the 3D display; and wherein the 3D display is configured to project stereoscopic image pairs of the 3D representation. 
     
     
       44. The scanning system according to claim 33, wherein the at least one display is a 3D display, whereby a 3D representation of the 3D environment is displayed on the 3D display; wherein the handheld device is an intra-oral 3D scanner or an in-ear 3D scanner; and wherein the 3D display is configured to project stereoscopic image pairs of the 3D representation.