Provides recommendations and insight on data management, governance procedures, and intricacies applicable to the acquisition, maintenance, validation, and utilization of data
Build and support Lake house model enabling publishers and consumers
Partner with architecture team to rapidly design and build highly efficient, secure, fault tolerant and resilient compute capabilities in Cloud environment
Embrace a culture of experimentation and constantly strive for improvement and learning
Designs and delivers trusted data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way
Defines database back-up, recovery, and archiving strategy
Generates advanced data models for one or more teams using firmwide tooling, linear algebra, statistics, and geometrical algorithms
Approves data analysis tools and processes
Creates functional and technical documentation supporting best practices
Advises junior engineers and technologists
Evaluates and reports on access control processes to determine effectiveness of data asset security
Adds to team culture of diversity, equity, inclusion, and respect
Required qualifications, capabilities, and skills
10+ years of data engineering experience, 5+ years experience leading teams
5+ years of solid AWS experience, Glue/EMR/Lake Formation/API Gateway/EKS/ECS/EC2
10+ Years of programming experience with Python or Java(Spring, Hibernate)
Very strong experience building/monitoring containerized applications
Experience with data platforms like snowflake or Databricks
Experience leading multi-platform, multi petabyte ecosystem
Experience designing fine grain access control across the platforms
Working experience with MPP systems like Teradata
Working experience with ETL tools like Ab Initio, Informatica
Hands on experience with logging and monitoring frameworks(Otel, Splunk, datadog, Prometheus)
Hands on experience with unit testing, integration testing