Job Description Summary
Sr. Data Engineer (Databricks)
Job Description
We are the makers of possible
BD is one of the largest global medical technology companies in the world. Advancing the world of health™ is our Purpose, and it's no small feat. It takes the imagination and passion of all of us-from design and engineering to the manufacturing and marketing of our billions of MedTech products per year-to look at the impossible and find transformative solutions that turn dreams into possibilities.
We believe that the human element, across our global teams, is what allows us to continually evolve. Join us and discover an environment in which you'll be supported to learn, grow and become your best self. Become a maker of possible with us.
Want more jobs like this?
Get jobs in Mahwah, NJ delivered to your inbox every week.
As an Azure Databricks Developer, you will be responsible for designing, implementing, and optimizing data solutions using Azure Databricks, Azure Data Factory, LogicApp etc. You will collaborate closely with multi-functional teams to ensure the efficient management, processing, and analysis of large-scale datasets. Your expertise will play a crucial role in enhancing our data-driven decision-making processes. Additionally, you'll take the lead in guiding other developers to deliver high-quality projects end to end.
Responsibilities:
- Design, develop, and implement scalable data pipelines using Azure Databricks to ingest, transform, and store structured and unstructured data.
- Analyze, optimize, and tune existing data pipelines for performance, reliability, and efficiency.
- Collaborate with data scientists, analysts, functional consultant, and other collaborators to understand business requirements and translate them into technical solutions.
- Provide expertise and guidance in selecting appropriate Azure services and tools to build robust data solutions.
- Implement security measures and standard processes to ensure data privacy and compliance with regulatory standards.
- Solve issues related to data processing, performance, and system failures.
- Stay updated with the latest Azure Databricks features, tools, and best practices and provide recommendations for improvements.
Requirements:
- Bachelor's degree in computer science or engineering, or equivalent work experience within Data Engineering.
- Proven experience working with Azure Databricks in large-scale data environments, 6-8 years of experience with Databricks or similar technologies.
- Must be well versed with Databricks Lakehouse, Unity Catalog concept and its implementation in enterprise environments.
- Must have strong understanding of Data warehousing and various governance and security standards around Databricks.
- Prior experience handling ERP data like SAP, JD Edwards and high-level understanding of different functional modules is required.
- In-depth knowledge of data engineering concepts, ETL processes, and data architecture principles. Strong understanding of cloud computing principles, particularly within the Azure ecosystem.
- Proficiency in programming languages such as Python /PySpark and query languages like SQL.
- Experience with Databricks Unity Catalog, Databricks Workflow, Delta Sharing, Auto Loader, Delta Live Table, Overwatch, Databricks Volume etc.
- Good to have knowledge of RESTful API and its implementation.
- Experience with data modeling, data warehousing, and working with both structured and unstructured data.
- Excellent problem-solving skills and the ability to work in a collaborative team environment.
- Strong communication skills to convey complex technical concepts to non-technical partners.
Preferred Qualifications:
- Master's Degree or higher
- 8+ years of rigorous experience developing in Databricks, PySpark, SQL (any flavor)
- Databricks Data Engineer Associate certification will be preferred.
- Experience with other Azure services such as Azure SQL Database, Azure Data Lake Storage, LogicApp, Azure Data Factory etc.
- Knowledge of machine learning and GenAI concepts is an added advantage.
For certain roles at BD, employment is contingent upon the Company's receipt of sufficient proof that you are fully vaccinated against COVID-19. In some locations, testing for COVID-19 may be available and/or required. Consistent with BD's Workplace Accommodations Policy, requests for accommodation will be considered pursuant to applicable law.
Why Join Us?
A career at BD means being part of a team that values your opinions and contributions and that encourages you to bring your authentic self to work. It's also a place where we help each other be great, we do what's right, we hold each other accountable, and learn and improve every day.
To find purpose in the possibilities, we need people who can see the bigger picture, who understand the human story that underpins everything we do. We welcome people with the imagination and drive to help us reinvent the future of health. At BD, you'll discover a culture in which you can learn, grow, and thrive. And find satisfaction in doing your part to make the world a better place.
To learn more about BD visit https://bd.com/careers
Becton, Dickinson and Company is an Equal Opportunity/Affirmative Action Employer. We do not unlawfully discriminate on the basis of race, color, religion, age, sex, creed, national origin, ancestry, citizenship status, marital or domestic or civil union status, familial status, affectional or sexual orientation, gender identity or expression, genetics, disability, military eligibility or veteran status, or any other protected status.
#earlycareer
Primary Work Location
USA NJ - Franklin Lakes
Additional Locations
Work Shift