Overall 8 to 10 Years' experience in Data Integration/ Data Warehousing projects.
Should have strong working experience in Databricks
Should have working experience in Azure Data Factory and ETL/ Data Warehousing.
Strong understanding on Data Warehousing, Data Model, Data Validation, Data Migration, Azure Cloud Platform.
Must be very Strong in SQL & SQL performance tuning. Preferably in MS SQL Server, if not then at least in Oracle/ Teradata.
Working experience in Python is desirable.
Should be able to design and implementing data ingestion pipelines from multiple sources using Azure Databricks. Develop scalable and re-usable frameworks for ingesting of data sets. Integrating the end to end data pipelines - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times. Ability to work with event based / streaming technologies to ingest and process data. Ability to work with other members of the project team to support delivery of additional project components (API interfaces, Search). Ability to evaluate the performance and applicability of multiple tools against customer requirements
Want more jobs like this?
Get Data and Analytics jobs in Bangalore, India delivered to your inbox every week.
Work with other developers, designers, and architects to ensure data applications meet requirement and performance, data security, and analytics goals.
Anticipate, identify, track, and resolve issues and risks affecting delivery.
Responsibilities:
Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization.
Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services.
Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data.
Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts, and demonstrating it to stakeholders.
DataBricks - Data Engineering