Job responsibilities
- Solves the companies most challenging cloud data platform problems by building innovative technical solutions around Data Lake Tools.
- Designs, implements, and maintains a managed AWS Databricks platform, and providing engineering and operational support.
- Adds features to and refactors existing code in Infra as Code - terraform modules, API development with Python and/or other tech products.
- Performs platform design, set-up and configuration, workspace administration, resource monitoring, providing engineering support to data engineering teams, Data Science/ML, and Application/integration teams.
- Collaborates closely with the Team Leads, other Platform/Architecture teams and Data Engineers/Scientists/Architects/Modelers/Analysts to design technical solutions for AWS Databricks platform.
- Drives decisions that influence the product design, application functionality, and technical operations and processes
- Serves as a function-wide subject matter expert in one or more areas of focus
- Influences peers and project decision-makers to consider the use and application of leading-edge technologies
- Adds to the team culture of diversity, equity, inclusion, and respect
Required qualifications, capabilities, and skills
- Formal training or certification on software engineering concepts and 5+ years applied experience
- Programming experience in one or more of the following Java or Python
- Hands-on experience in AWS Cloud Services Provisioning and Usage: RDS, Dynamo DB , Lambdas, IAM Role, EMR, EKS, Security Groups
- Hands-on experience in Terraform Enterprise, CI/CD pipelines using Jenkins and Groovy
- Good understanding of Networking Rules / Firewalls / DNS / IPv4 / Oauth / SAML / VPCs in AWS and On-Prem Systems, Big Data Compute Engines (Spark)
- Cyber security fundamentals and working knowledge
- Experience in designing, developing, or maintaining production-grade cloud solutions in Cloud ecosystems such as Amazon Web Services
- Hands-on practical experience delivering system design, application development, testing, and operational stability
- Ability to tackle design and functionality problems independently with little to no oversight
- Experience in Computer Science, Computer Engineering, Mathematics, or a related technical field
Preferred qualifications, capabilities, and skills
- Exposure to AWS & Databricks Platform administration
- Experience with Agile development processes, as needed (SCRUM/KANBAN) using JIRA.
- Experience in managing product release lifecycle at enterprise level.