About the Role
Spark is our main batch big data analytics query engine at Uber. It accounts for the majority of our data compute resources, powers most of our major ingestion, ETL, data science, and machine learning workloads. Spark at Uber is used by over 2.5K weekly active users, who run 3M+ applications every week.
As part of the Spark team, you will design, implement, optimize, and manage large scale distributed computing infrastructure that will impact multiple business use cases at Uber scale. You will also have the opportunity to collaborate with the open source community for Spark and related domains.
What the Candidate Will Need / Bonus Points
---- What the Candidate Will Do ----
Want more jobs like this?
Get jobs in Bangalore, India delivered to your inbox every week.
- Work with Uber data science and engineering teams to improve Spark applications and use-cases at Uber.
- Leverage knowledge of spark internals to dramatically help improve customer's Spark job performance.
- Design and implement distributed algorithms for large scale batch data processing.
- Work with multiple partner teams within and outside of Uber and build cross-functional solutions in a collaborative work environment.
- Work on Spark internals so as to continue improving Spark's robustness and performance for Uber's unique use cases.
- Work with the community to upstream Uber's contributions to open source and also keep our internal fork up to date
---- Basic Qualifications ----
- Bachelor's degree in Computer Science or related field.
- 5+ years of experience building large scale distributed software systems.
- Solid understanding of Java for backend / systems software development.
---- Preferred Qualifications ----
- MS / PhD in Computer Science or related field.
- Experience managing production systems with a strong availability SLA.
- Experience working with Apache Spark or similar analytics technologies.
- Experience working with large scale distributed systems, HDFS / Yarn.
- Experience working with SQL Compiler, SQL Plan / Runtime Optimization.
We welcome people from all backgrounds who seek the opportunity to help build a future where everyone and everything can move independently. If you have the curiosity, passion, and collaborative spirit, work with us, and let's move the world forward, together.
Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.
*Accommodations may be available based on religious and/or medical conditions, or as required by applicable law. To request an accommodation, please reach out to accommodations@uber.com.