Operating visual user interface controls with ink commands

Abstract

A facility enabling a user to operate visual user interface controls with ink commands is described. The facility causes to be displayed a control operable with mouse input, and receives an ink command directed to the displayed control. In response, the facility operates the displayed control in accordance with the received ink command.

1 . A processor-based device, comprising:
 a processor; and   a memory having contents that cause the processor to, in response to execution of an operating system by the processor:
 cause to be displayed a control operable with mouse and/or keyboard input; 
 receive ink input proximate to the displayed control; 
 interpret the received ink input as an ink command; and 
 operate the displayed control in accordance with the interpreted ink command. 
   
     
     
         2 . The device of  claim 1  wherein the displayed control is a one-dimensional value-setting control for setting an index value, and wherein the received ink input comprises a handwritten number, and wherein the interpreted ink command specifies an index value that is based on the handwritten number. 
     
     
         3 . The device of  claim 2  wherein the displayed control is a one-dimensional value-setting control for setting a media playback index time, a page number, or an audio volume level. 
     
     
         4 . The device of  claim 1  wherein the displayed control is for setting a quantity, and has
 a handle and a track, the track having a first end and a second end, the handle being repositionable to different points on the track corresponding to different values of the quantity, 
 wherein the received ink input comprises ink that crosses the control's track a particular percentage of the distance from the first end of the track to the second end of the track, 
 and wherein the interpreted ink command specifies a value for the quantity that is based upon the percentage. 
 
     
     
         5 . The device of  claim 4  wherein the displayed control is a slider, scrollbar, or ruler. 
     
     
         6 . The device of  claim 1  wherein the displayed control is a button bearing a visual symbol,
 and wherein the received ink input comprises an approximation of the visual symbol borne by the button, 
 and wherein operating the displayed control in accordance with the interpreted ink command comprises activating the button. 
 
     
     
         7 . The device of  claim 1  one wherein the displayed control is one or more electronic document reader pages,
 and wherein the received ink input comprises a selected number of hand-drawn carets pointing in a selected direction, 
 and wherein operating the displayed control in accordance with the interpreted ink command comprises performing the selected number of page flips in the selected direction. 
 
     
     
         8 . The device of  claim 1  wherein the displayed control is a date picker control for setting a date, the control presenting a plurality of dates arranged in rows each corresponding to a week,
 and wherein the received ink input is a containing shape that contains a selected date presented by the control, 
 and wherein operating the displayed control in accordance with the interpreted ink command comprises setting the selected date. 
 
     
     
         9 . The device of  claim 1  wherein the displayed control is a date picker control for setting a date, the control that present either (1) a plurality of dates arranged in rows each corresponding to a week, and/or (2) two or more controls for independently setting different components of a date,
 and wherein the received ink input is text indicating a date, 
 and wherein operating the displayed control in accordance with the interpreted ink command comprises setting the indicated date. 
 
     
     
         10 . The device of  claim 1  wherein the displayed control is an enumerated textual string picker control for setting one of a plurality of the enumerated textual strings each comprised of a sequence of characters,
 and wherein the received ink input is a handwritten sequence of characters matching the beginning of the character sequence of exactly one of the enumerated textual strings, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises setting the enumerated textual string the beginning of whose sequence matches the handwritten sequence of characters. 
 
     
     
         11 . The device of  claim 1  wherein the displayed control is a container control for selecting items in a container, the container control containing a visual representation of each of at least a portion of the items in the container,
 and wherein the received ink input is an enclosing shape surrounding one or more of the item visual representations, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises selecting the items whose visual representations are surrounded by the enclosing shape. 
 
     
     
         12 . The device of  claim 11  wherein the displayed container control shows items in the container in a grid, list, or tree. 
     
     
         13 . The device of  claim 1  wherein the displayed control is a container control for selecting items in a container, the container control containing a visual representation of each of at least a portion of the items in the container, each of the items in the container having a textual string associated with it,
 and wherein the received ink input is a handwritten textual string comprising a sequence of characters, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises subsetting the items for which the container control contains visual representations to only those items whose associated textual strings match the handwritten textual string. 
 
     
     
         14 . The device of  claim 1  wherein the displayed control is a container control containing a visual representation of each of at least a portion of the items in a source container, at least one of the item visual representations indicating that the represented item is selected,
 and wherein the received ink input is an arrow beginning near one of the item visual representations that indicates that the represented item is selected, and ending near a visual indication of a target container, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises transferring the selected items from the source container to the target container. 
 
     
     
         15 . The device of  claim 1  wherein the displayed control is a container control containing a visual representation of each of at least a portion of the items in a source container, at least one of the item visual representations indicating that the represented item is selected,
 and wherein the received ink input is an arrow beginning near one of the item visual representations that indicates that the represented item is selected, and ending near a visual indication of an operation that can be performed against items, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises subjecting the selected items to the indicated operation. 
 
     
     
         16 . The device of  claim 1  wherein the displayed control is a container control containing a visual representation of each of at least a portion of the items in the container,
 and wherein the received ink input comprises a shape drawn over at least a portion of the representation of the selected item, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises deleting the selected item from the container. 
 
     
     
         17 . The device of  claim 16  wherein the shape comprised by the received ink input is a horizontal segment, a horizontal segment with a loop at one end, or a scribble. 
     
     
         18 . The device of  claim 1  wherein the displayed control is a container control for selecting items in a container, the container control containing a visual representation of each of at least a portion of the items in the container,
 and wherein the received ink input comprises a check mark, a star shape, or an asterisk drawn near the representation of each of one or more distinguished items, 
 and wherein operating the displayed control in accordance with the interpreted ink input comprises selecting the distinguished items. 
 
     
     
         19 . A computer-readable medium having contents configured to cause a computing system to:
 cause to be displayed a control operable with mouse input;   receive an ink command directed to the displayed control; and   operate the displayed control in accordance with the received ink command.   
     
     
         20 . A method in a computing system, the method comprising:
 receiving ink input at a particular location relative to a display region;   based on the received ink input's shape and location, interpreting the received ink input as an ink command; and   performing the interpreted ink command.