Crowdsourced on-boarding of digital assistant operations

Abstract

Embodiments described herein are generally directed towards systems and methods relating to a crowd-sourced digital assistant and system. In particular, embodiments facilitate the intuitive creation and distribution of action datasets that include computing events or tasks that can be reproduced when an associated command, stored in an action dataset, is determined received by a digital assistant device. The digital assistant device described herein can generate new action datasets, on-board new action datasets, and receive new action datasets or updates to existing action datasets. Each digital assistant device in the described system can participate in the building of action datasets, so as to crowd-source a dialect that can be understood by a digital assistant device.

What is claimed is: 
     
         1 . A computer-implemented method performed by a digital assistant device, the method comprising:
 generating an action dataset for an application of the digital assistant device, by:
 recording, at the digital assistance device, a set of received inputs and application events that correspond to an application installed on the digital assistant device; 
 receiving, by the digital assistant device, a set of commands to generate a set of command templates associated with the application installed on the digital assistant device; 
   communicating, by the digital assistant device, the generated action dataset to a remote server for distribution to other digital assistant devices.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the generated action dataset causes the application to perform an operation in response the digital assistant device receiving a command that matches the received set of commands of the generated action dataset. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein the set of received inputs includes at least one touch input event. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the application events include a detected GUI element, detected metadata, a detected state, or a detected operation for the application. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein the set of received inputs and application events are recorded in sequence. 
     
     
         6 . The computer-implemented method of claim wherein the set of commands includes a set of command representations that include a corresponding string of text. 
     
     
         7 . The computer-implemented method of  claim 6 , is herein the corresponding string of text is generated based on converted speech data received by the digital assistant device. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein the generated action dataset is associated with causing the digital assistant device to perform a particular operation or invoke a particular output of the application. 
     
     
         9 . The computer-implemented method of  claim 1 , wherein the action datase includes identifying information for the application. 
     
     
         10 . The computer-implemented method of  claim 9 , wherein the application identifying information includes a name of the application and a version of the application. 
     
     
         11 . The computer-implemented method of  claim 9 , wherein the application identifying information includes a name of an operating system installed on the digital assistant device. 
     
     
         12 . A computer-implemented method, comprising:
 receiving, at a server device, an action dataset generated by a digital assistant device and associated with an application of the digital assistant device;   determining, at the server device, that at least a portion of the received action dataset is different than other action dataset portions stored in memory of the server device;   storing, by the server device, the portion of the received action dataset to the memory of the service device;   identifying other digital assistance devices that have the application of the digital assistance device; and   distributing, by the server device, the portion of the received action dataset to the identified other digital assistant devices.   
     
     
         13 . The computer-implemented method of  claim 12 , wherein the action dataset includes application identifying information for the application. 
     
     
         14 . The computer-implemented method of  claim 12 , wherein at least the portion of the received action dataset is determined to be different than other action dataset portions based on a hash generated for the portion of the received action dataset not matching hashes generated for the other action dataset portions stored in the memory. 
     
     
         15 . A non-transitory, computer readable medium whose contents, when executed by a computing system of a digital assistant device, causes the computing system to perform a method, the method comprising:
 receiving a command representation;   determining that the command representation corresponds to a command template stored in an action dataset,
 wherein the action dataset is indexed by the digital assistant device or a server of a crowd-sourced digital assistant network associated with multiple other digital assistant devices; 
   receiving the action dataset that stores the command template corresponding to the received command representation; and   reproducing, via an application of the digital assistant device, a set of recorded events or tasks stored by the received action dataset by interpreting the received action dataset.   
     
     
         16 . The non-transitory, computer readable medium of  claim 15 , wherein the command representation is generated by a speech-to-text engine of the digital assistant device based on received speech data. 
     
     
         17 . The non-transitory, computer readable medium of  claim 15 , wherein the action dataset is indexed based on a hash generated for the action dataset. 
     
     
         18 . The non-transitory, computer readable medium of  claim 15 , wherein the action dataset is one of multiple indexed action datasets stored on the digital assistant device or the server. 
     
     
         19 . The non-transitory, computer readable medium of  claim 15 , wherein the set of recorded events or tasks corresponds to the application installed on the digital assistant device. 
     
     
         20 . The non-transitory, computer readable medium of  claim 15 , wherein the action dataset is received based on a determination that the digital assistant device has installed the application.