Architect, design, and implement scalable, reliable, and highly available data infrastructure solutions to support workflow orchestration across the company.
Develop and maintain automation tools, streamlining deployment and management of the workflow orchestration platform.
Collaborate with infrastructure engineers, data engineers, and other users of the data warehouse to ensure efficient integration and smooth operation of our platform with other systems and services.
Conduct performance testing and help optimize our orchestration platform to meet the evolving needs of internal users.
Manage and prioritize multiple projects and tasks, communicating progress and status with key stakeholders.
Participate in technical discussions, code reviews, and pair programming sessions to foster knowledge-sharing and team growth.
Provide input on best practices to improve scalability, reliability, and usability for the data platform.
Your Expertise:
5+ years of experience in software engineering, with a focus on data infrastructure or supporting data engineers and/or data scientists.
Strong programming skills in one or more of the following languages: Java and Python.
Experience building data applications that create and implement higher-level abstractions on top of the lower-level data infrastructure to simplify complex data operations, improve scalability, and enhance overall system performance.
Experience with data modeling, database design and various SQL dialects.
Experience with containerization technologies such as Docker and Kubernetes.
Extensive background in workflow orchestration solutions such as Apache Airflow, Luigi, Azkaban, Oozie, Prefect or Kubeflow.
Utilize experience in Data Engineering to optimize processes and bring forth new ideas and technologies that enhance our data capabilities.
Excellent communication skills and the ability to collaborate with cross-functional teams to build empathetic tools and platforms.
Excellent analytical and problem solving skills.
BS/MS/PhD in Computer Science, a related field, or equivalent work experience preferred.