We are looking for a seasoned Senior Data Platform Engineer to join EPAM, a company at the forefront of shaping digital solutions for Fortune 1000 businesses.
As a critical part of our team, the ideal candidate will bring deep expertise in Databricks and experience across multi-cloud environments. This role will focus on building, optimizing, and maintaining innovative, high-performance data platforms that enable powerful data insights and drive meaningful business transformations.
#LI-DNI
Responsibilities
- Architect and deploy robust data platforms using Databricks, focusing on optimal performance and security
- Create solutions that are cloud-agnostic across AWS, Azure, and GCP to ensure system flexibility and resilience
- Design and implement comprehensive data pipelines involving data lakes, warehouses, and streaming technologies
- Utilize Databricks SQL, Delta Lake, MLflow, and Spark for data interaction and performance enhancements
- Collaborate with various teams to implement and maintain workflows based on Databricks best practices
- Develop CI/CD pipelines tailored for data platform deployment and testing
- Set up and manage frameworks for monitoring, logging, and alerting to ensure infrastructure health
- Optimize compute and storage resources to balance cost-efficiency and performance
- Troubleshoot Databricks and Spark performance issues
- Mentor team members on effective cluster management and resource allocation in Databricks environments
- Maintain compliance and security standards throughout platform operations
- Drive adoption of advanced Databricks capabilities like Photon and Graviton instances
- Regularly refine architectures to align with evolving business and technology requirements
Want more jobs like this?
Get jobs in Madrid, Spain delivered to your inbox every week.
- Extensive experience in Databricks, Apache Spark, and distributed data processing systems
- Strong programming skills in Python, Scala, SQL
- Proficiency in data engineering services on AWS (S3, IAM, Lambda), Azure, GCP
- Expertise in data architecture and ETL workflows, focusing on data lakes and lakehouses
- Hands-on experience with Terraform, CloudFormation, and CI/CD tools
- Familiarity with monitoring and observability frameworks in large-scale data environments
- Strong analytical and problem-solving capabilities
- Excellent communication and collaboration skills
- Ability to self-manage in a dynamic work environment
- Certifications in Databricks, AWS, Azure, GCP
- Knowledge of Kubernetes and containerized deployments for data pipelines
- Experience with real-time data streaming frameworks and governance tools
- Private health insurance
- EPAM Employees Stock Purchase Plan
- 100% paid sick leave
- Referral Program
- Professional certification
- Language courses
- Why Join EPAM
- WORK AND LIFE BALANCE. Enjoy more of your personal time with flexible work options, 24 working days of annual leave and paid time off for numerous public holidays.
- CONTINUOUS LEARNING CULTURE. Craft your personal Career Development Plan to align with your learning objectives. Take advantage of internal training, mentorship, sponsored certifications and LinkedIn courses.
- CLEAR AND DIFFERENT CAREER PATHS. Grow in engineering or managerial direction to become a People Manager, in-depth technical specialist, Solution Architect, or Project/Delivery Manager.
- STRONG PROFESSIONAL COMMUNITY. Join a global EPAM community of highly skilled experts and connect with them to solve challenges, exchange ideas, share expertise and make friends.