Apparatus and methods for providing a persistent companion device

Abstract

A development platform for developing a skill for a persistent companion device (PCD) includes an asset development library having an application programming interface (API) configured to enable a developer to at least one of find, create, edit and access one or more content assets utilizable for creating a skill, an expression tool suite having one or more APIs via which are received one or more expressions associated with the skill as specified by the developer wherein the skill is executable by the PCD in response to at least one defined input, a behavior editor for specifying one or more behavioral sequences of the PCD for the skill and a skill deployment facility having an API for deploying the skill to an execution engine of the PCD.

What is claimed is: 
     
         1 . A development platform for developing a skill for a persistent companion device (PCD) comprising:
 an asset development library having an application programming interface (API) configured to enable a developer to at least one of find, create, edit and access one or more content assets utilizable for creating a skill that is executable by the PCD;   an expression tool suite having one or more APIs via which receive one or more expressions associated with the skill as specified by the developer wherein the skill is executable by the PCD in response to at least one defined input;   a behavior editor for specifying one or more behavioral sequences of the PCD for the skill; and   a skill deployment facility having an API for deploying the skill to an execution engine for executing the skill.   
     
     
         2 . The development platform of  claim 1  wherein at least one of the APIs of the asset development library and the expression tool suite comprises a Javascript API. 
     
     
         3 . The development platform of  claim 1  wherein the behavioral editor utilizes a graphical user interface to display the one or more behavioral sequences. 
     
     
         4 . The development platform of  claim 3 , wherein at least one behavioral sequence is displayed by graphically representing a plurality of sequential physical positions of the PCD for the sequence. 
     
     
         5 . The development platform of  claim 3 , wherein at least one behavioral sequence is displayed by representing a multi-modal expression of the PCD from a plurality of modes selected from the group consisting of plurality of sequential physical positions of the PCD, a sequence of sounds for the PCD, a sequence of graphics displayed by the PCD, a sequence of lighting effects displayed by the PCD and a sequence of speech by the PCD. 
     
     
         6 . The development platform of  claim 3 , wherein at least one behavioral sequence comprises at least one of a control signal to another device, a software application, and an input from at least one of another device, a software application for behaviors and an asset development. 
     
     
         7 . The development platform of  claim 3  wherein the one or more behavioral sequences are specified using a directed graph. 
     
     
         8 . The development platform of  claim 7  wherein the directed graph is a recursive directed graph. 
     
     
         9 . The development platform of  claim 1  wherein the expression tool suite enables specification of at least one of a mood, a mode and a persona configured to modulate the expression of a given skill by a PCD based on at least one of a mood, a mode, a user attribute model and a persona. 
     
     
         10 . The development platform of  claim 1  further comprising a sensory input handling module for handling one or more sensory inputs that can be captured by the PCD. 
     
     
         11 . The development platform of  claim 10 , wherein the sensory input module of the PCD has a defined local perceptual space in which the PCD is adapted to detect and track sensory input associated with the environment in which the PCD is deployed. 
     
     
         12 . The development platform of  claim 1  further comprising a simulation module for at least one of executing code developed for the skill and simulating the local perceptual space of the PCD, wherein the simulation module and the PCD can execute the same code. 
     
     
         13 . The development platform of  claim 12 , wherein the PCD and the simulation module can act as input sources or output targets for each other. 
     
     
         14 . The development platform of  claim 1 , wherein the platform is adapted for delivery of a skill to a PCD that includes at least one interface for receiving and executing the skill. 
     
     
         15 . The development platform of  claim 1 , wherein the at least one defined input comprises at least one of a digital input and a physical input. 
     
     
         16 . The development platform of  claim 1 , wherein the execution engine is deployed on the PCD. 
     
     
         17 . The development platform of  claim 1 , wherein the execution engine is deployed as part of the development platform. 
     
     
         18 . The development platform of  claim 1 , wherein the execution engine is deployed as part of a simulator for the behavior of the PCD. 
     
     
         19 . The development platform of  claim 1  wherein the asset development library comprises one or more library assets. 
     
     
         20 . The development platform of  claim 1  wherein the expression tool suite comprises a timeline editor configured to temporally coordinate multi-model expressions of the PCD. 
     
     
         21 . The development platform of  claim 20  wherein the multi-model expressions are selected from the group consisting of speech, sound effects, sound filters, physical movements, screen graphics, and LED lighting effects. 
     
     
         22 . The development platform of  claim 1  comprising a real-time debugger/editor to support live coding for WYSIWYG viewing of at least one of resulting code and other asset changes on at least one of the simulator output and the PCD. 
     
     
         23 . A platform for enabling development of a skill using a software development kit (SDK) comprising:
 a logic level module configured to map received inputs to coded responses; and   a perceptual level module comprising:
 a vision function module configured to detect one or more vision function events and to inform the logic level module of the one or more detected vision function events; and 
 a speech/sound recognition and understanding module configured to detect defined sounds and to inform the logic level module of the detected speech/sounds; and 
   an expression engine configured to generate one or more animations expressive of defined emotional/persona states and to transmit the one or more animations to the logic level module.   
     
     
         24 . The platform of  claim 23 , further comprising a multi-modal data collection and machine-learning module for enabling automated or human-guided improvement of at least one of a skill and a component of a skill. 
     
     
         25 . The platform of  claim 23 , further comprising a persona module inclusive of character-based interactivity with emotive, social and task-based attributes for creating one of a stored and learned digital representation for a persona for the PCD. 
     
     
         26 . The platform of  claim 23 , further comprising a user attribute module for at least one of learning and storing attributes of a user of the PCD. 
     
     
         27 . The platform of  claim 23 , wherein the animations consist of effects selected from the group consisting of sequences of physical positions of the PCD, sounds for the PCD, graphics displayed by the PCD, lighting effects displayed by the PCD and speech by the PCD. 
     
     
         28 . The platform of  claim 23  wherein the expression engine receives an input selected from the group consisting of pre-recorded audio, speech, text, touch, textually formatted output utterances, animations, textures, data from other device and applications. 
     
     
         29 . The platform of  claim 23  wherein the speech/sound recognition and understanding module receives an input generated via a process selected from the group consisting of phrase spotting, non-phrase spotting, non-speech sound recognition, automatic speech recognition, the application of natural language rules, tools for data collection and training of new recognition and understanding models or improvement of existing ones, the identification of the user, and the direction of the source of sound. 
     
     
         30 . The platform of  claim 23  wherein communication between the logic level module and the perceptual level module operates to translate perceived and internal events into expressed skills. 
     
     
         31 . The platform of  claim 23 , wherein communication between the logic level module and the perceptual module operates to at least one of provide an input and accept an input from an interface with at least one of another device, a web service, a mobile application, a web application, a cloud computing resource, a cloud storage resource, a social media interface and a video conferencing interface. 
     
     
         32 . The platform of  claim 23 , wherein communication between the logic level module and the perceptual level module operates to modulate at least one of the persona and the behavior of the PCD. 
     
     
         33 . The platform of  claim 32  wherein the communication operates with contextual information comprising a user attribute model. 
     
     
         34 . The platform of  claim 23  wherein pre-recorded audio and textually formatted output utterances are processed by a text-to-speech and audio splicing/filtering engine to create a speech component of an animation. 
     
     
         35 . The platform of  claim 34 , wherein the expression engine generates a sound effect associated with the speech component of the animation. 
     
     
         36 . The platform of  claim 23  wherein the vision function module is configured to perform functions selected from the group consisting of picture taking, video recording, face and facial feature tracking, person tracking, object recognition, location of the PCD, emotion recognition, gesture recognition, gesture tracking and facial recognition. 
     
     
         37 . The platform of  claim 23  wherein the type of speech input to be accepted by the PCD is defined by a user via a user interface. 
     
     
         38 . The platform of  claim 23 , further comprising a publication module for publishing a skill to at least one of an online store and a cloud-based skill repository where a skill can be at least one of discovered, purchased and downloaded to a PCD of a user. 
     
     
         39 . The platform of  claim 23  wherein the one or more vision function events comprise at least one of a social visual feature and an emotive visual feature. 
     
     
         40 . The platform of  claim 38  wherein the emotive visual feature comprises at least one of a depth feature, a color feature, an illumination feature, and an infra-red (IR) feature. 
     
     
         41 . The platform of  claim 23  wherein the defined sounds comprise at least one of a social feature and an emotive sound based feature. 
     
     
         42 . The platform of  claim 23 , wherein communication between the logic level module and the perceptual level module operates to modulate a user attribute model. 
     
     
         43 . The platform of  claim 42  wherein communication operates with contextual information comprising the persona model. 
     
     
         43 . The platform of  claim 38 , wherein the skill is downloaded from the cloud. 
     
     
         44 . The platform of  claim 23  wherein inputs comprise reusable multi-modal input-output modules for behavior tree development, library asset development, and skill development.