Mobile brain-based device for use in a real world environment

Abstract

A mobile brain-based device BBD includes a mobile base equipped with sensors and effectors (Neurally Organized Mobile Adaptive Device or NOMAD), which is guided by a simulated nervous system that is an analogue of cortical and sub-cortical areas of the brain required for visual processing, decision-making, reward, and motor responses. The brain-based device BBD learns to discriminate among multiple objects with shared visual features, and associated “target” objects with innately preferred auditory cues. The brain-based device BBD is moveable, in a rich real-world environment involving continual changes in the size and location of visual stimuli due to self-generated or autonomous, movement, and shows that reentrant connectivity and dynamic synchronization provide an effective mechanism for binding the features of visual objects so as to reorganize object features such as color, shape and motion while distinguishing distinct objects in the environment.

1 . A brain-based device (BBD) adapted to visually discriminate between objects in a visual scene within an environment, comprising:
 an adaptive device moveable within the environment and having a sensor configured to obtain visual information; and   a simulated nervous system operable on one or more microprocessors, the simulated nervous system modeling neuro-anatomy of cortical regions of a human brain to visually discriminate between objects in the visual scene;   wherein the simulated nervous system includes
 a first neural area corresponding to a ventral cortical pathway forming a visual system responsive to visual information obtained from the sensor to produce visual stimuli, 
 a second neural area corresponding to an ascending neuromodulatory system responsive to a salient event while moving within the environment to produce value stimuli, and 
 a third neural area corresponding to a superior colliculus area responsive to the visual stimuli and the value stimuli to control movement of the adaptive device; 
   wherein the first, second, and third neural areas are connected via reentrant connectivity;   wherein connectivity between the first neural area and the second neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the first neural area to the third neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the second neural area to the third neural area comprises excitatory synaptic plastic connections;   wherein each of the first, second, and third neural areas includes a plurality of simulated neuronal units connected via reentrant connectivity; and   wherein the simulated nervous system is configured to discriminate between objects via interaction of local processes including activities within each of the first, second, and third neural areas, and global processes including functional neural circuits formed throughout the simulated nervous system during operation of the simulated nervous system.   
     
     
         2 . The BBD of  claim 1 , wherein the neuronal units in each of the first, second, and third neural areas have relative neuronal activity whose timing is represented by a firing rate variable and the relative timing of which is represented by a phase variable, in which similar firing phases of neuronal units in the areas reflect synchronous activity. 
     
     
         3 . The BBD of  claim 1 , wherein said value stimuli modify a strength of the synaptic plastic connections between the first, second and third neural areas to provide for the adaptive behavior of the brain-based device in the environment. 
     
     
         4 . The BBD of  claim 1 , wherein the first neural area corresponds to a vertical cortical pathway having neural areas V 1 , V 2 , V 4  and IT coupled in a pathway V 1 →V 2 →V 4 →IT. 
     
     
         5 . The BBD of  claim 1 , wherein each of the first, second and third neural areas includes neuronal units, in which the neuronal units have excitatory synaptic connections amongst themselves, and each of the excitatory synaptic connections are voltage-dependent. 
     
     
         6 . The BBD of  claim 5 , wherein the first, second and third neural areas have reentrant excitatory connections between the areas, and all reentrant excitatory connections are voltage-dependent. 
     
     
         7 . The BBD of  claim 1 , wherein
 the sensor is a first sensor; and   the adaptive device has a second sensor configured to obtain auditory information; and   the simulated nervous system further includes a fourth neural area forming an auditory system responsive to auditory information obtained from the second sensor for producing auditory stimuli; and   the third neural area is responsive to the auditory stimuli to control movement of the adaptive device.   
     
     
         8 . The BBD of  claim 1 , wherein the simulated nervous system is remote from the adaptive device and configured to wirelessly communicate with the adaptive device. 
     
     
         9 . A brain-based device (BBD) adapted to visually discriminate between objects in a visual scene within an environment, comprising:
 an adaptive device moveable within the environment and having a first sensor configured to obtain visual information and a second sensor configured to obtain auditory information; and   a simulated nervous system operable on one or more microprocessors, the simulated nervous system modeling neuro-anatomy of cortical regions of a human brain to visually discriminate between objects in the visual scene;   wherein the simulated nervous system includes
 a first neural area corresponding to a ventral cortical pathway forming a visual system responsive to visual information obtained from the sensor to produce visual stimuli, 
 a second neural area corresponding to an ascending neuromodulatory system responsive to a salient event while moving within the environment to produce value stimuli, 
 a third neural area forming an auditory system responsive to auditory information obtained from the second sensor for producing auditory stimuli, and 
 a fourth neural area corresponding to a superior colliculus area responsive to the visual stimuli, the auditory stimuli, and the value stimuli to control movement of the adaptive device; 
   wherein the first, second, third, and fourth neural areas are connected via reentrant connectivity;   wherein connectivity between the first neural area and the second neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the first neural area to the fourth neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the second neural area to the fourth neural area comprises excitatory synaptic plastic connections;   wherein each of the first, second, and third neural areas includes a plurality of simulated neuronal units connected via reentrant connectivity; and   wherein the simulated nervous system is configured to discriminate between objects via interaction of local processes including activities within each of the first, second, and third neural areas, and global processes including functional neural circuits formed throughout the simulated nervous system during operation of the simulated nervous system.   
     
     
         10 . The BBD of  claim 9 , wherein the neuronal units in each of the first, second, third, and fourth neural areas have relative neuronal activity whose timing is represented by a firing rate variable and the relative timing of which is represented by a phase variable, in which similar firing phases of neuronal units in the areas reflect synchronous activity. 
     
     
         11 . The BBD of  claim 9 , wherein said value stimuli modify a strength of the synaptic plastic connections between the first, second and third neural areas to provide for the adaptive behavior of the brain-based device in the environment. 
     
     
         12 . The BBD of  claim 9 , wherein the first neural area corresponds to a vertical cortical pathway having neural areas V 1 , V 2 , V 4  and IT coupled in a pathway V 1 →V 2 →V 4 →IT. 
     
     
         13 . The BBD of  claim 9 , wherein each of the first, second, third and fourth neural areas includes neuronal units, in which the neuronal units have excitatory synaptic connections amongst themselves, and each of the excitatory synaptic connections are voltage-dependent. 
     
     
         14 . The BBD of  claim 13 , wherein the first, second, third, and fourth neural areas have reentrant excitatory connections between the areas, and all reentrant excitatory connections are voltage-dependent. 
     
     
         15 . The BBD of  claim 9 , wherein the simulated nervous system is remote from the adaptive device and configured to wirelessly communicate with the adaptive device. 
     
     
         16 . A brain-based device (BBD) adapted to discriminate between objects in an environment, comprising:
 an adaptive device moveable within the environment and having a sensor configured to obtain visual information; and   a simulated nervous system operable on one or more microprocessors to discriminate between objects in the environment;   wherein the simulated nervous system includes
 a first neural area responsive to visual information obtained from the sensor to produce visual stimuli, 
 a second neural area responsive to a salient event while moving within the environment to produce value stimuli, and 
 a third neural area responsive to the visual stimuli and the value stimuli to control movement of the adaptive device; 
   wherein the first, second, and third neural areas are connected via reentrant connectivity;   wherein connectivity between the first neural area and the second neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the first neural area to the third neural area comprises value-dependent synaptic plastic connections;   wherein connectivity from the second neural area to the third neural area comprises excitatory synaptic plastic connections; and   wherein each of the first, second, and third neural areas includes a plurality of simulated neuronal units connected via reentrant connectivity.   
     
     
         17 . The BBD of  claim 16 , wherein the neuronal units in each of the first, second, and third neural areas have relative neuronal activity whose timing is represented by a firing rate variable and the relative timing of which is represented by a phase variable, in which similar firing phases of neuronal units in the areas reflect synchronous activity. 
     
     
         18 . The BBD of  claim 16 , wherein said value stimuli modify a strength of the synaptic plastic connections between the first, second and third neural areas to provide for the adaptive behavior of the brain-based device in the environment. 
     
     
         19 . The BBD of  claim 16 , wherein
 the sensor is a first sensor; and   the adaptive device has a second sensor configured to obtain auditory information; and   the simulated nervous system further includes a fourth neural area forming an auditory system responsive to auditory information obtained from the second sensor for producing auditory stimuli; and   the third neural area is responsive to the auditory stimuli to control movement of the adaptive device.   
     
     
         20 . The BBD of  claim 16 , wherein the simulated nervous system is remote from the adaptive device and configured to wirelessly communicate with the adaptive device.