Introduction
At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk.
Your Role and Responsibilities
- We are seeking an experienced data engineer to join our team. As a data engineer, you will be responsible for designing, implementing, and maintaining data structures and pipelines for efficient data processing and analysis. Your role will involve working with various databases, including DB2, Cloudant, PostgreSQL, and REDIS. You will collaborate with cross-functional teams to understand data requirements, optimise data flow, and ensure data integrity. Your expertise in data modelling, ETL processes, and database technologies will be instrumental in building robust and scalable data solutions.
- Responsibilities:
- Design and implement data models and database structures to meet business requirements.
- Develop and maintain ETL processes to extract, transform, and load data from various sources.
- Optimise data pipelines and workflows for efficient data processing and analysis.
- Collaborate with data scientists and analysts to define data requirements and deliver actionable insights
- Ensure data quality, integrity, and security throughout the data lifecycle.
Want more jobs like this?
Get Data and Analytics jobs in Bangalore, India delivered to your inbox every week.
Monitor and troubleshoot data pipelines, resolving issues and performance bottlenecks.
Stay updated with emerging trends and technologies in data engineering and database management.
Mentor junior data engineers and provide technical guidance and support.
Design and implement data models for DB2, Cloudant, PostgreSQL, and REDIS databases.
Implement and automate data validation and testing processes.
Explore and evaluate new tools and technologies for data engineering and database management.
Required Technical and Professional Expertise
- Proven experience as a data engineer or similar role
- Strong expertise in designing data structures and database modelling
- Experience with DB2 and Cloudant databases is required.
- Knowledge of PostgreSQL and REDIS databases is a plus.
- Proficiency in ETL processes, Pentaho/DataStage, and tools such as Apache Spark, Apache Kafka, or Talend
- Familiarity with data warehousing concepts and technologies
- Strong SQL skills for data manipulation and querying
- Understanding of data governance and data management principles
- Knowledge of data security and compliance standards
- Experience with data integration and data pipeline optimisation
- Familiarity with cloud-based data platforms such as AWS or Azure
- Experience with data visualisation tools and techniques is a plus.
Preferred Technical and Professional Expertise
- strong analytical and problem-solving skills
- Excellent communication and collaboration abilities
- Ability to work in a fast-paced and dynamic environment
- Continuous learning mindset to stay updated with industry trends and advancements