Description and Requirements
The Platform architect leads and implements best-in-class data management strategies and practices in standing up and managing the enterprise data platform (Data Lakehouse). They will be responsible for designing and building new capabilities to integrate into the platform - from researching new technologies/services to building POCs, training and mentoring others. Additionally they will collaborate with the data product teams to ensure the platform enables value.
Key Responsibilities:
- Architect and Design the Data Lakehouse: Lead the design and implementation of a scalable and secure Data Lakehouse on AWS, including data storage and compute layers.
- Storage Solutions: Design and implement storage solutions using AWS services like S3, Iceberg,
- Integrate relevant metadata from platform with data catalog and/or metadata management solutions.
- Compute Resources: Architect and optimize compute resources using AWS services like Glue, EMR, and Lambda for ETL processes, and possibly Redshift or Athena for query execution.
- Develop POCs, POVs and pilots to test architecture, capabilities etc. and collaborate with collaborate with data engineers to ensure seamless integration and ingestion of data from various sources into the Lakehouse.
- Security and Compliance: Implement best practices for data security, including encryption, IAM roles, and compliance with relevant data protection regulations.
- Performance Optimization: Continuously monitor and optimize the performance of the data lakehouse, including storage costs and compute efficiency.
- Collaboration: Work closely with data engineers, data scientists, and business stakeholders to ensure the platform meets their needs for data products.
- Documentation and Training: Provide thorough documentation and training to the internal team on the architecture and use of the Data Lakehouse.
Want more jobs like this?
Get jobs that are Remote delivered to your inbox every week.
Minimum Experience:
- Extensive hands-on experience with AWS services related to data storage and compute (e.g., S3, Glue, EMR, Redshift, Athena, Lambda).
- Proven experience in designing and implementing Data Lakehouses or Data Lakes on AWS using Lake Formation
- Ability to design scalable, reliable, and secure architectures that meet business needs.
- Strong analytical and problem-solving skills with attention to detail.
- Collaboration and Communication: Excellent communication skills and the ability to work effectively in a collaborative environment.
- Ability to mentor junior staff.
- 10+ years of experience designing and/or developing data management platforms, at least 5 in the cloud. Experience as hands on data engineer is a must.
Preferred Experience:
- AWS Big Data Certification (or equivalent)
- Familiarity with Apache Iceberg or similar table formats for handling large-scale data in a lakehouse environment.
- Experience in enabling monitoring using system observability tools like Splunk, Prometheus (with Grafana)
- Experience in enabling compute through containers and container orchestration using Kubernetes.
- Security Best Practices: Knowledge of data security and compliance best practices on AWS.
- Experience or knowledge of RBAC, TBAC access control on AWS
- Experience harvesting and leveraging metadata to drive technical processes and implementation.
- Experience with modern data catalogs.
Hiring Range:
- Greater New York City Area (NY, NJ, CT residents): $162,600 - $248,795
- All Other Locations (within approved locations): $139,000 - $212,670
As a candidate for this position, your salary and related elements of compensation will be contingent upon your work experience, education, licenses and certifications, and any other factors Healthfirst deems pertinent to the hiring decision.
In addition to your salary, Healthfirst offers employees a full range of benefits such as, medical, dental and vision coverage, incentive and recognition programs, life insurance, and 401k contributions (all benefits are subject to eligibility requirements). Healthfirst believes in providing a competitive compensation and benefits package wherever its employees work and live.
The hiring range is defined as the lowest and highest salaries that Healthfirst in "good faith" would pay to a new hire, or for a job promotion, or transfer into this role.
WE ARE AN EQUAL OPPORTUNITY EMPLOYER. Applicants and employees are considered for positions and are evaluated without regard to mental or physical disability, race, color, religion, gender, gender identity, sexual orientation, national origin, age, genetic information, military or veteran status, marital status, mental or physical disability or any other protected Federal, State/Province or Local status unrelated to the performance of the work involved.