Senior Data Engineer

Job Description:

The Senior Data Engineer will design, develop, document, and implement end-to-end data solutions applying best engineering practices, testing, deployment, and providing production support. The incumbent will demonstrate technical skills in tools like Python, Snowflake, data analysis/modeling, ETL/ELT processes, and API development and maintenance to support diverse business customers. An ideal candidate is passionate about analyzing data needs, aspiring to improve data quality, and contributing innovative solutions to our data platforms which includes cloud and/or on-premise infrastructure technologies.

The Expertise we're looking for

• 5+ years of relevant experience in business intelligence, analytics or data process improvement
• Bachelor's degree (e.g., Computer Science, Engineering, Statistics) / Master's degree preferred
• Immediately apply Industry-leading analytics approaches and tools to transform data into insights
• Deliver high quality, timely, cost effective and maintainable software solutions in an agile environment with diverse a diverse tech stack to meet functional and non-functional business requirements
• Understand and develop highly scalable distributed systems in public Cloud
• Ability to develop and drive best engineering practices and data strategies as per the needs of the domain
• Partner with data practitioners and other stakeholders to address complex data challenges and troubleshoot data-related issues effectively.
• Build and maintain robust ETL pipelines to integrate data from multiple sources into OLAP data stores, ensuring data integrity and consistency.
• Bring an innovative spirit in search of efficiencies, process improvement opportunities, technical improvements, and other ways to add value to the organization with focus on operations improvement
• Pproduction support in a 24x7 on-call rotation

• Relational database management expertise (Oracle, Snowflake, PostgreSQL)
• Extensive hands on experience with ETL/ELT tools (Informatica, Nifi, Snaplogic)
• Expertise in Data Analysis, Data Profiling and Data Modeling skills
• Knowledge of Data Warehousing methodologies and concepts
• Extensive experience with ANSI SQL, database stored procedures, and performance tuning
• Proficiency in Snowflake DB using Javascript, Python, and snowSQL; Snowflake features (Snow pipe, Data Sharing, etc.) and stay updated with the latest Snowflake features and best practices.
• Proficiency with python for data movement/transformation including development of classes and object oriented code
• Complex batch cycle orchestration (tools like Control-M, Autosys or Crontab)
• Implement best practices in data security, role-based access control, and data masking to maintain compliance and data governance standards
• Knowledge of Cloud platforms and Services (AWS - IAM, EC2, S3, Lambda, RDS)
• Knowledge of data streaming tools like Kafka, Kinesis
• Design experience of scalable data models, optimized for data ingestion and analytics requirements including SCD
• Developing and automating deployment with GIT, Jenkins, and CICD Processes
• Experience with REST APIs and in-memory technologies

• Knowledge of non-RDBMS databases like Graph, No-SQL, Timeseries databases
• Working knowledge of analytics and BI front-end tools like Power BI, Tableau, and Thoughtspot
• Financial Services experience