Predictive virtual training systems, apparatuses, interfaces, and methods for implementing same

Abstract

Apparatuses, systems, interfaces, and implementing methods including constructing training programs or routines and predictive training programs and routines implemented in a VR, AR, MR or XR environments, preparing non-predictive and/or predictive tools for use in predictive and/or non-predictive training programs or routines implemented in the VR/AR/MR/XR environments, converting non-computer assisted training programs into predictive and/or non-predictive training programs implemented in a VR and/or AR/MR/XR environments, and implementing avatars to assist trainees in performing training programs routines or any aspects thereof.

1 . A method implemented on an electronic device comprising a process coupled to or associated with a motion sensor, an input device, a display device and an output device, the method comprising:
 loading a non-animated training program comprising a plurality of routines, tasks, or combinations thereof;   gathering information about:
 (a) training program features, attributes, properties, characteristics, or combinations thereof of the non-animated training program; 
 (b) routine features, attributes, properties, and characteristics; 
 (c) task features, attributes, properties, and characteristics; 
 (d) program equipment features, attributes, properties, characteristics, or combinations thereof associated with equipment used in the non-animated training program; 
   generating a VR training program or an AR/MR/XR training program corresponding to the non-animated program, the VR training program or the AR/MR/XR training program including:
 (a) generated training program features, attributes, properties, and characteristics; 
 (b) generated routine features, attributes, properties, and characteristics; 
 (c) generated task features, attributes, properties, and characteristics; 
 (d) generated program equipment features, attributes, properties, and characteristics; 
   analyzing the non-animated training program and the gathered information;   creating content from the analyzed non-animated training program and the gathered information, the content comprising program content, routine content, task content, and equipment content;   generating hot spots or interactive areas from the created content, the hot spots or interactive areas comprising:
 (a) hot spots or interactive areas associated with the program content; 
 (b) hot spots or interactive areas associated with the routine content; 
 (c) hot spots or interactive areas associated with the task content; 
 (d) hot spots or interactive areas associated equipment content; 
   generating a VR environment or an AR/MR/XR environment for the generated VR training program or the generated AR/MR/XR training program;   populating the VR environment or the VR/AR/MR environment with the generated VR training program or the generated AR/MR/XR training program and the hot spots or interactive areas;   capturing trainer performance data as a trainer performs the VR training program or the AR/MR/XR training program, the trainer performance data comprising trainer program competency data, trainer routine competency data, trainer task competency data, trainer hot spot or interact area interaction data, and trainer environment interaction data;   capturing trainee performance data as a trainee performs the VR training program or the AR/MR/XR training program, the trainee performance data comprising trainee program competency data, trainee routine competency data, trainee task competency data, trainee hot spot or interact area interaction data, and trainee environment interaction data;   comparing the trainer performance data and the trainee performance data;   determining differences between the trainer performance data and the trainee performance data;   testing the differences between the trainer performance data and the trainee performance data,   if each of the differences exceeds one or more minimum difference criteria, then:
 displaying the differences; and 
 for each of the routines or tasks having differences that exceeds the one or more minimum difference criteria, repeating:
 the trainee capturing step, 
 the comparing step, 
 the determining differences step; 
 the testing step; 
 until each of the differences is less than or equal to the one or more minimum difference criteria; and 
 
   indicating successful completion of the training program.   
     
     
         2 - 28 . (canceled) 
     
     
         29 . The method of  claim 1 , further comprising:
 storing the gathered information on local databases, remote databases, or both;   storing the generated VR training program or the generated AR/MR/XR training program on local databases, remote databases, or both;   storing the analyses of the non-animated training program and the gathered information on local databases, remote databases, or both;   storing the created content on local databases, remote databases, or both;   storing the generated hot spots or interactive areas on local databases, remote databases, or both;   storing the VR environment or an AR/MR/XR environment on local databases, remote databases, or both;   storing the VR environment or an AR/MR/XR environment populated with the generated VR training program or the generated AR/MR/XR training program and the hot spots or interactive areas on local databases, remote databases, or both;   storing the trainer performance data on the local databases, remote databases, or both;   storing the trainee performance data on the local databases, remote databases, or both;   storing the differences on the local databases, remote databases, or both;   storing the repeated trainee image sequences, computer generated trainee constructions, scaled trainee constructs, the repeated differences, and trainee completion data;   providing the trainer performance data or any part thereof to the trainer;   during and/or after the trainee capturing, providing the trainee performance data to the trainer and/or a supervisor;   providing the differences to the trainer and/or the supervisor; and   during and/or after the trainee repeating step, providing the trainee performance data and the differences to the trainer and/or the supervisor.   
     
     
         30 . The method of  claim 1 , further comprising:
 highlighting the differences according to a highlighting format,   wherein the highlighting format comprising visually highlighting the differences, haptic highlighting the differences via a haptic device, audio highlighting the differences via a audio device, neurofeedback highlighting the differences via a neurofeedback device, or any combination thereof.   
     
     
         31 . The method of  claim 1 , further comprising:
 before, during, and/or after the repeating step, illustrating to the trainee how to adjust the trainee's body and/or any part thereof in the overlaid construct to improve trainee performance.   
     
     
         32 . The method of  claim 1 , wherein values of the one or more minimum difference criteria are:
 less than or equal to a 20% difference between the trainer performance data and the trainee performance data;   less than or equal to a 10% difference between the trainer performance data and the trainee performance data;   less than or equal to a 5% difference between the trainer performance data and the trainee performance data; or   less than or equal to a 1% the difference between the trainer performance data and the trainee performance data.   
     
     
         33 . The method of  claim 1 , wherein:
 the trainer performance data comprising trainer whole body position features, trainer body part position features, trainer device position features, or any combination thereof;   the trainee performance data comprise trainee whole body position features, trainee body part position features, trainee device position features, or any combination thereof.   
     
     
         34 . The method of  claim 1 , further comprising:
 generating one or more avatars, providing the avatar with trainer performance data and trainee performance data, and modifying the one or more avatars based on the training program analysis and modifications.   
     
     
         35 . The method of  claim 34 , wherein the avatar is configured to change form depending on the routine or task being performed and the trainee performance of the routine or task. 
     
     
         36 . The method of  claim 1 , wherein the training program is fully interactive using motion-based processing, hard select processing, gesture processing, voice command processing, neural command processing, or any combination thereof. 
     
     
         37 . The method of  claim 1 , further comprising:
 continuously, periodically, or intermittently analyzing all of the stored data;   developing predictive tools or routines to assist trainees in performing the training program, routines, or tasks;   developing trainee type predictive tools to assist specific types of trainees in performing the training program, routines, or tasks;   modifying one, some, or all aspects, features, attributes, properties, and/or characteristics of the training program, the training routine, the training task, the environment or environments, hot spots or interactive areas, or any combination thereof, during and/or after the trainer performing the generated training program or any part thereof;   updating the training program, routines, tasks, and the hot spots; and   storing the updates on the local databases, remote databases, or both.   
     
     
         38 . The method of  claim 1 , wherein, in the constructing or generating steps, the training program, routines, and/or tasks further comprising:
 explanatory sessions associated with the program, routines and/or tasks;   explanatory sessions associated with the equipment;   question/answer sessions associated with the routines and/or tasks;   question/answer sessions associated with the equipment;   information presentation sessions associated with the program, routines and/or tasks;   information presentation sessions associated with the equipment;   trainee pass/fail explanatory sessions;   trainee evaluation sessions;   trainee performance ranking sessions;   trainee feedback sessions; or   any combination thereof.   
     
     
         39 . The method of  claim 1 , further comprising:
 analyzing the trainer performance data, the trainee performance data, the differences, and/or the repeated trainee performance data, the repeated trainee differences, and the trainee completion data; and   modifying:
 (1) one, some, or all of the generated training program features, attributes, properties, and characteristics; 
 (2) one, some, or all of the generated training routine features, attributes, properties, and characteristics; 
 (3) one, some, or all of the generated training task features, attributes, properties, and characteristics; 
 (4) one, some, or all of the generated program equipment features, attributes, properties, and characteristics; 
 (5) one, some, or all of the hot spots or interactive areas associated with the program content; 
 (6) one, some, or all of the hot spots or interactive areas associated with the routine content; 
 (7) one, some, or all of the hot spots or interactive areas associated with the task content; 
 (8) one, some, or all of the hot spots or interactive areas associated with the equipment content; 
 wherein the modifications improve the training program; and 
   storing the modifications on the local databases, remote databases, or both.   
     
     
         40 . The method of  claim 1 , wherein, in the generating steps:
 the generated VR training program or the generated AR/MR/XR training program or the preexisting VR training program or the preexisting AR/MR/XR training program comprising computer generated 2D, 3D, 4D, or nD components.   
     
     
         41 . The method of  claim 1 , wherein the program content, routine content, task content, and equipment content independently comprising:
 textual content;   non-textual content including:
 visual content; 
 audio content; 
 audiovisual content; 
 haptic content; or 
 any combination; or 
   any combination of textual content and non-textual content.   
     
     
         42 . The method of  claim 1 , further comprising:
 analyzing trainer and trainee historical performance data; and   modifying one, some, or all aspects, features, attributes, properties, and/or characteristics of the training program, the training routine, the training task, the environment or environments, hot spots or interactive areas, or any combination thereof, during and/or after the trainer performing the generated training program or any part thereof.   
     
     
         43 . The method of  claim 1 , further comprising:
 storing the trainer image sequence on the local databases, remote databases, or both;   storing the computer generated trainer construct on the local databases, remote databases, or both;   storing the trainee image sequence on the local databases, remote databases, or both;   storing the computer generated trainee construct on the local databases, remote databases, or both;   storing the scaled trainer construct and the scaled trainee construct on the local databases, remote databases, or both;   storing the differences on the local databases, remote databases, or both;   storing the overlaid construct on the local databases, remote databases, or both;   storing the repeated trainee image sequences, computer generated trainee constructions, scaled trainee constructs, the repeated differences, and trainee completion data;   providing the differences to the trainer and/or a supervisor; and   during and/or after the trainee repeating step, providing the differences to the trainer and/or the supervisor.   
     
     
         44 . The method of  claim 1 , further comprising:
 modifying the trainer construct based on the trainee performance data to improve the trainer construct based on trainee performance data; and   storing the modification on the local databases, remote databases, or both.   
     
     
         45 . The method of  claim 1 , further comprising:
 constructing a specific trainer construct based on trainee learning proclivities; and   storing the specific trainer construct on the local databases, remote databases, or both.   
     
     
         46 . The method of  claim 1 , further comprising:
 continuously, periodically, or intermittently analyzing all of the stored data;   developing predictive tools or routines to assist trainees in performing the training program, routines, or tasks;   developing trainee type predictive tools to assist specific types of trainees in performing the training program, routines, or tasks;   modifying the trainer construct based on the trainee performance data to improve the trainer construct based on trainee performance data;   updating the training program, routines, and tasks; and   storing the updates on the local databases, remote databases, or both.