Application programming interface for setting the prominence of user interface elements

Abstract

In one implementation, a method of determining a visual appearance of a user interface (UI) element overlaid in an underlying physical environment is performed at a device. The method includes: obtaining environmental data corresponding to a physical environment; determining environmental characteristics of the physical environment based on the environmental data; in response to obtaining a request to display a UI element, determining whether a prominence-display value satisfies a prominence criterion; in response to determining that the prominence-display value satisfies the prominence criterion: modifying visual characteristics of the UI element based on the environmental characteristics of the physical environment, and displaying the UI element as an overlay on the physical environment, wherein the UI element includes the modified visual characteristics; and in response to determining that the prominence-display value for the UI element does not satisfy the prominence criterion, foregoing modifying the visual characteristics of the UI element.

What is claimed is: 
     
       1. A method comprising:
 at a device including one or more environmental sensors, one or more processors, a non-transitory memory, and a display: 
 obtaining, from the one or more environmental sensors, environmental data that corresponds to a physical environment; 
 determining environmental characteristics of the physical environment based at least in part on the environmental data; 
 in response to obtaining a request to display a user interface (UI) element, determining a prominence-display value for the UI element; 
 in response to determining that the prominence-display value for the UI element exceeds a prominence threshold:
 modifying one or more visual characteristics of the UI element based on the environmental characteristics of the physical environment, and 
 displaying, via the display, the UI element as a first overlay on the physical environment, wherein the UI element includes the one or more modified visual characteristics while displayed as the first overlay; and 
 
 in response to determining that the prominence-display value for the UI element does not exceed the prominence threshold:
 foregoing modifying the one or more visual characteristics of the UI element; and 
 displaying, via the display, the UI element as a second overlay on the physical environment, wherein the second overlay is different from the first overlay, and wherein the UI element does not include the one or more modified visual characteristics while displayed as the second overlay. 
 
 
     
     
       2. The method of  claim 1 , wherein the prominence-display value for the UI element corresponds to one of a level of criticality or a level of importance associated with the UI element. 
     
     
       3. The method of  claim 1 , wherein the environmental data corresponds to image data associated with a field-of-view of the device, wherein the UI element is overlaid onto the image data. 
     
     
       4. The method of  claim 1 , wherein the display corresponds to an additive display, and wherein the UI element is displayed or projected onto the additive display. 
     
     
       5. The method of  claim 1 , wherein the device corresponds to at least one of a near-eye system, a mobile phone, or a tablet. 
     
     
       6. The method of  claim 1 , wherein the one or more modified visual characteristics includes modifying at least one of a brightness, a contrast, a hue, a saturation, rotation coordinates, translational coordinates, and a size value of the UI element. 
     
     
       7. The method of  claim 1 , wherein the environmental characteristics of the physical environment corresponds to at least one of lighting characteristics, objects within the physical environment, and a background color. 
     
     
       8. The method of  claim 1 , further comprising:
 obtaining an object-specific flag for the UI element, wherein the object-specific flag corresponds to a particular object; and 
 in response to determining that at least one instance of the particular object is present within the environmental data, displaying the UI element in visual proximity to the particular object in the physical environment. 
 
     
     
       9. The method of  claim 8 , wherein displaying the UI element proximate to the particular object in the physical environment includes displaying the UI element proximate to a single instance of the particular object from among two or more instances of the particular object in response to determining that the physical environment includes the two or more instances of the particular object. 
     
     
       10. The method of  claim 1 , wherein modifying the one or more visual characteristics of the UI element is based at least in part on user accessibility parameters. 
     
     
       11. The method of  claim 1 , wherein modifying the one or more visual characteristics of the UI element is based at least in part on maintaining a visual attribute of the UI element. 
     
     
       12. The method of  claim 11 , wherein maintaining the visual attribute of the UI element includes modifying at least one visual characteristic of the UI element to contrast with the physical environment. 
     
     
       13. The method of  claim 12 , wherein the visual attribute corresponds to at least one of a color, a geometric shape, a texture, and a size. 
     
     
       14. The method of  claim 1  further comprising, determining a body pose vector of a user of the device; and
 wherein modifying the one or more visual characteristics of the UI element is based at least in part on the body pose vector of the user, and the environmental characteristics of the physical environment. 
 
     
     
       15. The method of  claim 1  further comprising, determining, using a gaze sensor, a gaze direction of a user of the device; and
 wherein modifying the one or more visual characteristics of the UI element is based at least in part on the gaze direction of the user of the device, and the environmental characteristics of the physical environment. 
 
     
     
       16. A device comprising:
 one or more environmental sensors; 
 a display; 
 one or more processors; 
 a non-transitory memory; and 
 one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
 obtain, from the one or more environmental sensors, environmental data that corresponds to a physical environment; 
 determine environmental characteristics of the physical environment based at least in part on the environmental data; 
 in response to obtaining a request to display a user interface (UI) element, determine a prominence-display value for the UI element; 
 in response to determining that the prominence-display value for the UI element exceeds a prominence threshold:
 modify one or more visual characteristics of the UI element based on the environmental characteristics of the physical environment, and 
 display, via the display, the UI element as a first overlay on the physical environment, wherein the UI element includes the one or more modified visual characteristics while displayed as the first overlay; and 
 
 in response to determining that the prominence-display value for the UI element does not exceed the prominence threshold:
 forego modifying the one or more visual characteristics of the UI element; and 
 display, via the display, the UI element as a second overlay on the physical environment, wherein the second overlay is different from the first overlay, and wherein the UI element does not include the one or more modified visual characteristics while displayed as the second overlay. 
 
 
 
     
     
       17. The device of  claim 16 , wherein the one or more modified visual characteristics includes modifying at least one of a brightness, a contrast, a hue, a saturation, rotation coordinates, translational coordinates, and a size value of the UI element. 
     
     
       18. The device of  claim 16 , wherein the environmental characteristics of the physical environment corresponds to at least one of lighting characteristics, objects within the physical environment, and a background color. 
     
     
       19. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with one or more environmental sensors and a display, cause the device to:
 obtain, from the one or more environmental sensors, environmental data that corresponds to a physical environment; 
 determine environmental characteristics of the physical environment based at least in part on the environmental data; 
 in response to obtaining a request to display a user interface (UI) element, determine a prominence-display value for the UI element; 
 in response to determining that the prominence-display value for the UI element exceeds a prominence threshold:
 modify one or more visual characteristics of the UI element based on the environmental characteristics of the physical environment, and 
 display, via the display, the UI element as a first overlay on the physical environment, wherein the UI element includes the one or more modified visual characteristics while displayed as the first overlay; and 
 
 in response to determining that the prominence-display value for the UI element does not exceed the prominence threshold:
 forego modifying the one or more visual characteristics of the UI element; and 
 display, via the display, the UI element as a second overlay on the physical environment, wherein the second overlay is different from the first overlay, and wherein the UI element does not include the one or more modified visual characteristics while displayed as the second overlay. 
 
 
     
     
       20. The non-transitory memory of  claim 19 , wherein the one or more modified visual characteristics includes modifying at least one of a brightness, a contrast, a hue, a saturation, rotation coordinates, translational coordinates, and a size value of the UI element. 
     
     
       21. The non-transitory memory of  claim 19 , wherein the environmental characteristics of the physical environment corresponds to at least one of lighting characteristics, objects within the physical environment, and a background color.