User interface for context labeling of multimedia items

Abstract

In certain embodiments, a neural network may be trained to associated context information with multimedia items. In some embodiments, context predictions for multimedia items may be obtained via a neural network. A first multimedia item and a first task related to a first context prediction for the first multimedia item may be presented on a user interface. A user response to the first task may be obtained via the user interface. Based on the user response to the first task, prediction feedback related to the first context prediction or the first multimedia item may be provided to the neural network to cause the neural network to be updated based on the prediction feedback.

1 .- 20 . (canceled) 
     
     
         21 . A system for content labeling of multimedia items comprising:
 a computer system comprising one or more processors programmed with computer program instructions that, when executed, cause the computer system to perform operations comprising:
 determining a set of multimedia items based on multimedia items and corresponding labels predicted by an instance of a neural network; 
 causing a first graphical representation of the multimedia items to be presented as a first array of multimedia items on a user interface, the first array of multimedia items arranged for a user to complete a first task; 
 causing a second graphical representation of the multimedia items to be presented as a second array of multimedia items on the user interface, the second array of multimedia items arranged for a user to complete a second task, wherein the first array of multimedia items and the second array of multimedia items are presented at a same time on the user interface; 
 obtaining, via the user interface, at least one user indication for at least one of the first array of multimedia items and the second array of multimedia items, wherein the user indication is related to at least one of where to place a threshold and whether to change a label; and 
 providing, to the neural network, the at least one user indication to cause the neural network to be updated based on the at least one user indication. 
   
     
     
         22 . The system of  claim 21  further comprising providing, to at least one other neural network, the at least one user indication to cause the at least one other neural network to be updated. 
     
     
         23 . The system of  claim 21 , wherein at least one of the first and the second array of multimedia items is arranged for display in a list, grid, stack, or objects arranged in 3-D space. 
     
     
         24 . The system of  claim 21 , wherein at least one of the first and the second array of multimedia items are rendered for display in the user interface at sufficient size so content associated with each multimedia item may be distinguished, but small enough so the user can view multiple multimedia items simultaneously. 
     
     
         25 . The system of  claim 21 , further comprising:
 obtaining, via the user interface, at least one user change of a label of a multimedia item of at least one of the first array and the second array of multimedia items; and   providing, to the neural network, the at least one change of the label to cause the neural network to be updated.   
     
     
         26 . The system of  claim 21 , further comprising, after determining the set of multimedia items based on multimedia items and corresponding labels predicted by an instance of the neural network:
 converting multimedia items having a time dimension into a visual representation while preserving a property enabling such multimedia items to appear visually similar to multimedia items not having a time dimension.   
     
     
         27 . A method for content labeling of multimedia items comprising:
 determining, by a computer system, a set of multimedia items based on multimedia items and corresponding labels predicted by an instance of a neural network;   causing, by the computer system, a first graphical representation of the multimedia items to be presented as a first array of multimedia items on a user interface, the first array of multimedia items arranged for a user to complete a first task;   causing, by the computer system, a second graphical representation of the multimedia items to be presented as a second array of multimedia items on the user interface, the second array of multimedia items arranged for a user to complete a second task, wherein the first array of multimedia items and the second array of multimedia items are presented at a same time on the user interface;   obtaining, by the computer system via the user interface, at least one user indication for at least one of the first array of multimedia items and the second array of multimedia items, wherein the user indication is related to at least one of where to place a threshold and whether to change a label; and   providing, by the computer system to the neural network, the at least one user indication to cause the neural network to be updated based on the at least one user indication.   
     
     
         28 . The method of  claim 27  further comprising providing, by the computer system to at least one other neural network, the at least one user indication to cause the at least one other neural network to be updated. 
     
     
         29 . The method of  claim 27 , wherein at least one of the first and the second array of multimedia items is arranged for display in a list, grid, stack, or objects arranged in 3-D space. 
     
     
         30 . The method of  claim 27 , wherein at least one of the first and the second array of multimedia items are rendered for display in the user interface at sufficient size so content associated with each multimedia item may be distinguished, but small enough so the user can view multiple multimedia items simultaneously. 
     
     
         31 . The method of  claim 27 , further comprising, after determining the set of multimedia items based on multimedia items and corresponding labels predicted by an instance of the neural network:
 converting, by the computer system, multimedia items having a time dimension into a visual representation while preserving a property enabling such multimedia items to appear visually similar to multimedia items not having a time dimension.   
     
     
         32 . The method of  claim 27 , further comprising:
 obtaining, by the computer system via the user interface, at least one user change of a label of a multimedia item of at least one of the first array and the second array of multimedia items; and   providing, by the computer system to the neural network, the at least one change of the label to cause the neural network to be updated.   
     
     
         33 . A non-transitory computer-readable media comprising instructions that, when executed by at least one processor, causes operations comprising:
 determining a set of multimedia items based on multimedia items and corresponding labels predicted by an instance of a neural network;   causing a first graphical representation of the multimedia items to be presented as a first array of multimedia items on a user interface, the first array of multimedia items arranged for a user to complete a first task;   causing a second graphical representation of the multimedia items to be presented as a second array of multimedia items on the user interface, the second array of multimedia items arranged for a user to complete a second task, wherein the first array of multimedia items and the second array of multimedia items are presented at a same time on the user interface;   obtaining, via the user interface, at least one user indication for at least one of the first array of multimedia items and the second array of multimedia items, wherein the user indication is related to at least one of where to place a threshold and whether to change a label; and   providing, to the neural network, the at least one user indication to cause the neural network to be updated based on the at least one user indication.   
     
     
         34 . The non-transitory computer-readable media of  claim 33  comprising further instructions that, when executed by the at least one processor, causes providing, to at least one other neural network, the at least one user indication to cause the at least one other neural network to be updated. 
     
     
         35 . The non-transitory computer-readable media of  claim 33  wherein the instructions for causing the first graphical representation of the multimedia items to be presented as a first array of multimedia items on the user interface and for causing the second graphical representation of the multimedia items to be presented as a second array of multimedia items on the user interface causes an array of rendered multimedia items arranged as at least one of a list, grid, stack, or objects arranged in 3-D space. 
     
     
         36 . The non-transitory computer-readable media of  claim 33  wherein the instructions for causing the first graphical representation of the multimedia items to be presented as a first array of multimedia items on the user interface and for causing the second graphical representation of the multimedia items to be presented as a second array of multimedia items on the user interface causes rendering for display in the user interface at sufficient size so content associated with each multimedia item may be distinguished, but small enough so the user can view multiple multimedia items simultaneously. 
     
     
         37 . The non-transitory computer-readable media of  claim 33  comprising further instructions that, when executed by the at least one processor, causes operations comprising:
 obtaining, via the user interface, at least one user change of a label of a multimedia item of at least one of the first array and the second array of multimedia items; and 
 providing, to the neural network, the at least one change of the label to cause the neural network to be updated. 
 
     
     
         38 . The non-transitory computer-readable media of claim  13  wherein after the instructions for determining the set of multimedia items based on multimedia items and corresponding labels predicted by an instance of the neural network, further instructions that, when executed by the at least one processor, causes operations comprising:
 converting multimedia items having a time dimension into a visual representation while preserving a property enabling such multimedia items to appear visually similar to multimedia items not having a time dimension.