Digital assistant for providing handsfree notification management

Abstract

This relates generally to intelligent automated assistants and, more specifically, to provide a handsfree notification management system. An example method includes displaying one or more notifications by the electronic device. In response to displaying the one or more notifications, the method includes detecting a visual interaction of a user with the one or more notifications, identifying a notification from the one or more notifications based on the visual interaction, receiving a speech input related to the notification from the user, determining one or more actions associated with the notification based on the speech input, and performing the one or more actions.

What is claimed is: 
     
       1. A method performed on an electronic device, the method comprising:
 displaying one or more notifications by the electronic device; 
 in response to displaying the one or more notifications:
 detecting a visual interaction of a user with the one or more notifications; 
 identifying a notification from the one or more notifications based on the visual interaction; 
 receiving a speech input related to the notification from the user; 
 determining one or more actions associated with the notification based on the speech input; and 
 performing the one or more actions, wherein performing the one or more actions comprises:
 determining whether the one or more actions are associated with a notification system or an application associated with the notification; and 
 in response to determining that the one or more actions are associated with the notification system:
 invoking the notification system to perform the one or more actions based on determining that the one or more actions are associated with the notification of the notification system. 
 
 
 
 
     
     
       2. The method of  claim 1 , further comprising:
 in response to identifying the notification from the one or more notifications based on the visual interaction:
 determining time-interval of the visual interaction with the notification; 
 determining that the time-interval exceeds at least one predetermined threshold; 
 identifying one or more actions associated with the notification based on determining that the time-interval exceeds the at least one predetermined threshold; and 
 performing the one or more actions. 
 
 
     
     
       3. The method of  claim 1 , wherein each of the one or more notifications is an alert of an event associated with the electronic device. 
     
     
       4. The method of  claim 1 , wherein detecting the visual interaction of the user with the one or more notifications is based on a gaze direction of the user's eyes. 
     
     
       5. The method of  claim 1 , wherein identifying the notification from the one or more notifications based on the visual interaction comprises:
 determining a gaze direction of the user directed at the one or more notifications; 
 determining a time-interval of the gazing direction of the user; and 
 selecting the notification from the one or more notifications based on the time-interval. 
 
     
     
       6. The method of  claim 1 , wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device. 
     
     
       7. The method of  claim 1 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; 
 determining that the speech input contains one or more ambiguous references; 
 resolving the one or more ambiguous references based on a content of the notification; and 
 identifying one or more actions based on resolving the one or more ambiguous references and the speech input. 
 
     
     
       8. The method of  claim 1 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; and 
 identifying the one or more actions based on parsing the speech input. 
 
     
     
       9. The method of  claim 1 , wherein the speech input includes a wake word for enabling a digital assistant associated with the electronic device for identifying the one or more actions. 
     
     
       10. The method of  claim 1 , further comprising:
 in response to determining that the one or more actions are associated with the notification system:
 determining that the one or more actions are associated with manipulating one or more user interface elements of the notification; and 
 manipulating the one or more user interface elements to perform the one or more actions. 
 
 
     
     
       11. The method of  claim 1 , wherein performing the one or more actions comprises:
 in response to determining that the one or more actions are associated with the application:
 invoking the application to perform the one or more actions based on determining that the one or more action is associated with the notification of the notification system. 
 
 
     
     
       12. The method of  claim 11 , further comprising:
 in response to determining that the one or more actions are associated with the application:
 identifying an entity associated with the notification; and 
 invoking the application to perform the one or more actions involving the entity to change a state of the entity associated with the notification of the notification system. 
 
 
     
     
       13. The method of  claim 12 , wherein the one or more notifications are displayed on a portion of a user interface of the electronic device. 
     
     
       14. An electronic device comprising:
 one or more processors; 
 a memory; and 
 one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
 displaying one or more notifications by the electronic device; 
 in response to displaying the one or more notifications:
 detecting a visual interaction of a user with the one or more notifications; 
 identifying a notification from the one or more notifications based on the visual interaction; 
 receiving a speech input related to the notification from the user; 
 determining one or more actions associated with the notification based on the speech input; and 
 performing the one or more actions, wherein performing the one or more actions comprises:
 determining whether the one or more actions are associated with a notification system or an application associated with the notification; and 
 in response to determining that the one or more actions are associated with the notification system: 
  invoking the notification system to perform the one or more actions based on determining that the one or more actions are associated with the notification of the notification system. 
 
 
 
 
     
     
       15. A non-transitory computer-readable storage medium storing
 one or more programs, the one or more programs including instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:
 display one or more notifications by the electronic device; 
 in response to displaying the one or more notifications:
 detect a visual interaction of a user with the one or more notifications; 
 identify a notification from the one or more notifications based on the visual interaction; 
 receive a speech input related to the notification from the user; 
 determine one or more actions associated with the notification based on the speech input; and 
 perform the one or more actions, wherein performing the one or more actions comprises:
 determining whether the one or more actions are associated with a notification system or an application associated with the notification; and 
 in response to determining that the one or more actions are associated with the notification system: 
  invoking the notification system to perform the one or more actions based on determining that the one or more actions are associated with the notification of the notification system. 
 
 
 
 
     
     
       16. The electronic device of  claim 14 , wherein the one or more programs further include instructions for:
 in response to identifying the notification from the one or more notifications based on the visual interaction:
 determining time-interval of the visual interaction with the notification; 
 determining that the time-interval exceeds at least one predetermined threshold; 
 identifying one or more actions associated with the notification based on determining that the time-interval exceeds the at least one predetermined threshold; and 
 performing the one or more actions. 
 
 
     
     
       17. The electronic device of  claim 14 , wherein each of the one or more notifications is an alert of an event associated with the electronic device. 
     
     
       18. The electronic device of  claim 14 , wherein detecting the visual interaction of the user with the one or more notifications is based on a gaze direction of the user's eyes. 
     
     
       19. The electronic device of  claim 14 , wherein identifying the notification from the one or more notifications based on the visual interaction comprises:
 determining a gaze direction of the user directed at the one or more notifications; 
 determining a time-interval of the gazing direction of the user; and 
 selecting the notification from the one or more notifications based on the time-interval. 
 
     
     
       20. The electronic device of  claim 14 , wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device. 
     
     
       21. The electronic device of  claim 14 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; 
 determining that the speech input contains one or more ambiguous references; 
 resolving the one or more ambiguous references based on a content of the notification; and 
 identifying one or more actions based on resolving the one or more ambiguous references and the speech input. 
 
     
     
       22. The electronic device of  claim 14 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; and 
 identifying the one or more actions based on parsing the speech input. 
 
     
     
       23. The electronic device of  claim 14 , wherein the speech input includes a wake word for enabling a digital assistant associated with the electronic device for identifying the one or more actions. 
     
     
       24. The electronic device of  claim 14 , wherein the one or more programs further include instructions for:
 in response to determining that the one or more actions are associated with the notification system:
 determining that the one or more actions are associated with manipulating one or more user interface elements of the notification; and 
 manipulating the one or more user interface elements to perform the one or more actions. 
 
 
     
     
       25. The electronic device of  claim 14 , wherein performing the one or more actions comprises:
 in response to determining that the one or more actions are associated with the application:
 invoking the application to perform the one or more actions based on determining that the one or more action is associated with the notification of the notification system. 
 
 
     
     
       26. The electronic device of  claim 25 , wherein the one or more programs further include instructions for:
 in response to determining that the one or more actions are associated with the application:
 identifying an entity associated with the notification; and 
 invoking the application to perform the one or more actions involving the entity to change a state of the entity associated with the notification of the notification system. 
 
 
     
     
       27. The electronic device of  claim 26 , wherein the one or more notifications are displayed on a portion of a user interface of the electronic device. 
     
     
       28. The non-transitory computer-readable storage medium of  claim 15 , wherein the one or more programs further include instructions, which when executed by the one or more processors of the electronic device, cause the electronic device to:
 in response to identifying the notification from the one or more notifications based on the visual interaction:
 determine time-interval of the visual interaction with the notification; 
 determine that the time-interval exceeds at least one predetermined threshold; 
 identify one or more actions associated with the notification based on determining that the time-interval exceeds the at least one predetermined threshold; and 
 perform the one or more actions. 
 
 
     
     
       29. The non-transitory computer-readable storage medium of  claim 15 , wherein each of the one or more notifications is an alert of an event associated with the electronic device. 
     
     
       30. The non-transitory computer-readable storage medium of  claim 15 , wherein detecting the visual interaction of the user with the one or more notifications is based on a gaze direction of the user's eyes. 
     
     
       31. The non-transitory computer-readable storage medium of  claim 15 , wherein identifying the notification from the one or more notifications based on the visual interaction comprises:
 determining a gaze direction of the user directed at the one or more notifications; 
 determining a time-interval of the gazing direction of the user; and 
 selecting the notification from the one or more notifications based on the time-interval. 
 
     
     
       32. The non-transitory computer-readable storage medium of  claim 15 , wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device. 
     
     
       33. The non-transitory computer-readable storage medium of  claim 15 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; 
 determining that the speech input contains one or more ambiguous references; 
 resolving the one or more ambiguous references based on a content of the notification; and 
 identifying one or more actions based on resolving the one or more ambiguous references and the speech input. 
 
     
     
       34. The non-transitory computer-readable storage medium of  claim 15 , wherein determining one or more actions associated with the notification based on the speech input comprises:
 parsing the speech input; and 
 identifying the one or more actions based on parsing the speech input. 
 
     
     
       35. The non-transitory computer-readable storage medium of  claim 15 , wherein the speech input includes a wake word for enabling a digital assistant associated with the electronic device for identifying the one or more actions. 
     
     
       36. The non-transitory computer-readable storage medium of  claim 15 , wherein the one or more programs further include instructions, which when executed by the one or more processors of the electronic device, cause the electronic device to:
 in response to determining that the one or more actions are associated with the notification system:
 determine that the one or more actions are associated with manipulating one or more user interface elements of the notification; and 
 manipulate the one or more user interface elements to perform the one or more actions. 
 
 
     
     
       37. The non-transitory computer-readable storage medium of  claim 15 , wherein performing the one or more actions comprises:
 in response to determining that the one or more actions are associated with the application:
 invoking the application to perform the one or more actions based on determining that the one or more action is associated with the notification of the notification system. 
 
 
     
     
       38. The non-transitory computer-readable storage medium of  claim 37 , wherein the one or more programs further include instructions, which when executed by the one or more processors of the electronic device, cause the electronic device to:
 in response to determining that the one or more actions are associated with the application:
 identify an entity associated with the notification; and 
 invoke the application to perform the one or more actions involving the entity to change a state of the entity associated with the notification of the notification system. 
 
 
     
     
       39. The non-transitory computer-readable storage medium of  claim 38 , wherein the one or more notifications are displayed on a portion of a user interface of the electronic device.