We are seeking a Senior Data Infrastructure Engineer, responsible for supporting a data processing infrastructure cluster.
The selected individual will have a pivotal role in sustaining, improving, and resolving problems from the AWS level down to specific data pipeline arrangements. Responsibilities will include deploying updates, establishing new instances, and overseeing continuous deployment pipelines to guarantee efficient functioning.

#LI-DNI

Responsibilities

• Address incidents and diagnose problems from AWS to data pipeline config
• Resolve identified problems or escalate them to the data team with detailed reasoning
• Apply enhancements within the infrastructure to boost efficiency and dependability
• Configure new instances according to project specifications
• Oversee and improve continuous deployment (CD) pipelines
• Cooperate with diverse engineering teams to incorporate their contributions into the infrastructure
• Guarantee the security of the system and adherence to relevant regulations
• Track system performance and suggest enhancement measures
• Record infrastructure modifications and refresh system documentation as necessary

• At least 3 years of relevant experience in a data infrastructure role
• Proficiency in AWS and Kubernetes (K8S)
• Understanding of Terraform for infrastructure as code (IaC) applications
• Expertise in either Airflow or Spark for data pipeline execution
• Qualifications in managing CI/CD tools like Jenkins
• Capability to operate autonomously and within a collaborative environment
• Exceptional problem-solving abilities and meticulous attention to detail

• Background in Flink and Databricks
• Proficiency in Spinnaker for CD management
• Ability to quickly adapt and master new technologies