Mobileye, a leader in advanced driver-assistance systems (ADAS), is at the forefront of the autonomous driving revolution. They have developed technologies such as REM crowdsourced mapping, True-Redundancy sensing, and Responsibility-Sensitive Safety (RSS) to pave the way for self-driving vehicles. With over 125 million vehicles worldwide using Mobileye technology, they became an independent company in 2022, separate from Intel.
To address the challenge of creating affordable, feature-rich high-definition (HD) maps, Mobileye’s Road Experience Management (REM) group built a robust microservices ecosystem on Amazon Web Services (AWS). They utilized Apache Spark workloads on Amazon Elastic Kubernetes Service (EKS) clusters to accelerate their HD map creation process.
Mobileye showcased four key reasons for choosing Kubernetes for data workloads, including scalability, orchestration, dynamic configuration, and standardization. By using Amazon EKS, they achieved agility and efficiency necessary for building and running machine learning workloads on Apache Spark.
In their journey with Amazon EKS, Mobileye developed a cloud-spark architecture, utilizing Kubernetes for creating and managing Spark applications. They also navigated challenges such as networking with VPC CNI, CoreDNS autoscaling, and Spark local storage optimization.
Optimizing inter-AZ network traffic, using Karpenter autoscaling for Spark on Kubernetes, and enhancing observability and logging in Spark were crucial steps taken by Mobileye. They also modified the Spark operator and utilized multiple EKS clusters for enhanced performance.
By adopting Amazon EKS and Karpenter, Mobileye effectively managed their data workloads, scaled to run over 400,000 virtual CPUs, and reduced developer overhead by 50% using AWS Managed Services. These technologies have propelled Mobileye to the forefront of innovation in autonomous driving, showcasing the transformative potential of Amazon EKS and Karpenter in the automotive industry.
Mobileye’s experience in building the platform on Amazon EKS is further explored in the AWS Podcast, discussing the challenges faced and the innovative solutions employed to overcome them. Their strategic use of Kubernetes, Spark, and EKS exemplifies how cutting-edge technology can drive advancements in autonomous driving and mobility solutions.
Article Source
https://aws.amazon.com/blogs/containers/mobileye-revolutionizing-hd-map-creation-for-autonomous-vehicles-with-spark-on-amazon-eks/