Meta's AI Training and Inference Infrastructure is growing exponentially to support ever increasing uses cases of AI. We need to build and evolve our network infrastructure that connects myriads of training accelerators like GPUs together. In addition, we need to ensure that the network is running smoothly and meets stringent performance, availability and reliability requirements of RDMA workloads that expects a loss-less fabric interconnect. To improve performance of these systems we constantly look for opportunities across the stack: network fabric and host networking, communication libraries and scheduling infrastructure.
AI/HPC Systems Production Engineer Responsibilities:
- Responsible for the overall reliability of the communication system, including monitoring, troubleshooting and proactive identification of production issues.
- Develop, extend and maintain CI/CD, testing pipelines for host components of training stack infrastructure, e.g. collective communication libraries (NCCL, RCCL), RDMA host stack dependencies.
- Active member of a multi-disciplinary team to develop solutions for large scale training systems. Work with performance engineers to ensure safe and robust rollout of new features.
Want more jobs like this?
Get Data and Analytics jobs in London, United Kingdom delivered to your inbox every week.
- BS/MS/PhD in relevant fields (EE, CS), with 4+ years work experience.
- Python, C/C++ coding skills
- Knowledge of Linux and foundational networking principles
- Experience working with up-to-date AI training workload packaging, CI/CD and distribution processes, containerization principles.
- Understanding of RDMA network stack principles and pain points on InfiniBand and RoCE Networks. Experience in development of systems and applications utilizing RDMA technologies. Experience with using communication libraries, such as MPI, NVIDIA Collective Communication Library (NCCL).
- Experience with GPU accelerator development frameworks, for example CUDA, OpenCL
- Experience in developing and troubleshooting system level software
Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today-beyond the constraints of screens, the limits of distance, and even the rules of physics.
Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.