Systems and methods for safe and reliable autonomous vehicles

Abstract

Autonomous driving is one of the world's most challenging computational problems. Very large amounts of data from cameras, RADARs, LIDARs, and HD-Maps must be processed to generate commands to control the car safely and comfortably in real-time. This challenging task requires a dedicated supercomputer that is energy-efficient and low-power, complex high-performance software, and breakthroughs in deep learning AI algorithms. To meet this task, the present technology provides advanced systems and methods that facilitate autonomous driving functionality, including a platform for autonomous driving Levels 3, 4, and/or 5. In preferred embodiments, the technology provides an end-to-end platform with a flexible architecture, including an architecture for autonomous vehicles that leverages computer vision and known ADAS techniques, providing diversity and redundancy, and meeting functional safety standards. The technology provides for a faster, more reliable, safer, energy-efficient and space-efficient System-on-a-Chip, which may be integrated into a flexible, expandable platform that enables a wide-range of autonomous vehicles, including cars, taxis, trucks, and buses, as well as watercraft and aircraft.

1 . A system-on-a-chip for an autonomous vehicle including:
 at least one central processing unit (CPU) cluster or CPU complex supporting virtualization, wherein the CPU cluster or CPU complex includes multiple CPU cores and associated caches,   at least one graphics processing unit (GPU) providing multi-core parallel processing,   an embedded hardware accelerator cluster, and   at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive optical sensor data,   wherein the at least one CPU cluster or complex, the at least one GPU providing multi-core parallel processing, and the at least embedded hardware accelerator cluster comprising the system-on-a-chip, interoperate to process at least the received optical sensor data to perform autonomous driving.   
     
     
         2 . The system-on-a-chip of  claim 1  wherein the system-on-a-chip is configured to enable the autonomous vehicle controller to be substantially compliant with level 5 full autonomous driving as defined by SAE specification J3016. 
     
     
         3 . The system-on-a-chip of  claim 1  wherein the system-on-a-chip is configured to enable the autonomous vehicle controller to be substantially compliant with integrity level “D” defined by ISO Standard 26262. 
     
     
         4 . A system-on-a-chip for use in an autonomous vehicle including:
 at least one central processing unit (CPU),   at least one programmable graphics processing unit (GPU) providing parallel processing and configured to use a tensor instruction set including mixed-precision processing cores partitioned into multiple processing blocks,   at least one programmable vision accelerator and/or at least one deep learning accelerator, and   at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing program code that when executed by system-on-the-chip, configures the system-on-a-chip to operate as an autonomous vehicle controller configured to receive optical sensor data,   wherein the at least one CPU, the at least one GPU, and the at least one programmable vision accelerator and/or the at least one deep learning accelerator, interoperate to process the optical sensor data and a trajectory estimation and/or route plan to provide autonomous driving control of an automobile.   
     
     
         5 . The system-on-a-chip of  claim 4  wherein the at least one CPU, the at least one GPU, and the at least one programmable vision accelerator and/or the at least one deep learning accelerator are structured and interconnected to be substantially compliant with integrity level “D” defined by Standard 26262 of the International Organization for Standardization. 
     
     
         6 . The system-on-a-chip of  claim 4  wherein the system-on-a-chip includes at least one memory device connected to the system-on-a-chip, the memory device storing instructions that when executed by the CPU and/or the GPU provides autonomous vehicle control that is substantially compliant with level 5 full autonomous driving as defined by SAE specification J3016. 
     
     
         7 . The system-on-a-chip of  claim 4  wherein the at least one GPU providing parallel processing is programmable and the deep learning accelerator comprises a tensor processing unit configured to execute the neural networks based on a tensor instruction set. 
     
     
         8 . The system-on-a-chip of  claim 4  wherein the at least one GPU is power-optimized for performance in automotive embedded use applications. 
     
     
         9 . The system-on-a-chip of  claim 4  wherein the at least one GPU is fabricated on a FinFET (Fin field effect transistor) high-performance manufacturing process. 
     
     
         10 . A system-on-a-chip for an autonomous vehicle including:
 at least one central processing unit (CPU),   at least one graphics processing unit (GPU) providing parallel processing,   a cache available to both the at least one CPU and the at least one GPU,   an embedded hardware accelerator cluster comprising at least one hardware-based accelerator configured to accelerate neural networks and/or accelerate programmable vision; and   at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing code that when executed by the system-on-the-chip, configures the system-on-a-chip to operate as an autonomous vehicle controller configured to receive sensor data,   wherein the at least one CPU, the at least one GPU providing parallel processing, and the at least one hardware-based accelerator interoperate to process the received sensor data to perform autonomous driving.   
     
     
         11 . A system-on-a-chip for an autonomous vehicle including:
 at least one central processing unit (CPU),   at least one graphics processing unit (GPU) providing parallel processing cores,   an embedded hardware accelerator cluster comprising at least one hardware-based accelerator, and   at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing program instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive sensor data,   wherein the at least one CPU, the at least one GPU providing parallel processing cores, and the at least one hardware-based accelerator interoperate to process at least the received sensor data to perform autonomous driving of an automobile.   
     
     
         12 . The system-on-a-chip of  claim 11  wherein the at least hardware-based accelerator includes one or more tensor processing units. 
     
     
         13 . The system-on-a-chip of  claim 12  wherein the one or more tensor processing units are configured for supporting INT8/INT16/FP16 data type for both features and weights. 
     
     
         14 . The system-on-a-chip of  claim 11  wherein the at least one accelerator is configured to accelerate computer vision algorithms for autonomous driving. 
     
     
         15 . The system-on-a-chip of  claim 11  wherein the at least one GPU is power-optimized for performance in automotive embedded use applications. 
     
     
         16 . The system-on-a-chip of  claim 11  wherein the at least one GPU is fabricated on a FinFET (Fin field effect transistor) high-performance manufacturing process. 
     
     
         17 . A system-on-a-chip for an autonomous vehicle including:
 at least one central processing unit (CPU) supporting virtualization,   at least one graphics processing unit (GPU) providing parallel processing,   a cache memory available to both the at least one CPU and the at least one GPU,   at least one accelerator, and   at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive LIDAR sensor data,   wherein the at least one CPU, the at least one GPU providing parallel processing, and the at least one accelerator interoperate to process the LIDAR sensor data and a trajectory estimation and/or route planning to provide autonomous driving.   
     
     
         18 . The system-on-a-chip of  claim 17  wherein the accelerator is configured to accelerate computer vision algorithms for autonomous driving. 
     
     
         19 . The system-on-a-chip of  claim 17  wherein the accelerator is configured for deep neural network acceleration. 
     
     
         20 . The system-on-a-chip of  claim 17  wherein the system-on-a-chip is configured to comprise at least a part of an autonomous vehicle controller.