Job Responsibilities
- Design, develop, and manage ETL jobs, data marts, and event collection and processing tools.
- Build data pipelines and tooling to support stakeholders across the project.
- Create secure and high-quality production code and maintain algorithms that run synchronously with appropriate systems.
- Write and maintain documentation of technical architecture.
- Participate in regular code reviews to maintain best code quality and adhere to best practices.
- Identify areas for quick wins to improve the experience of end users.
- Stay updated with the latest trends and technologies in data engineering.
- Work effectively in a team environment and contribute to team goals.
- Add to a team culture of diversity, equity, inclusion, and respect.
Required Qualifications, Capabilities, and Skills
- Formal training or certification on data engineering concepts and 3+ years applied experience
- Proficiency in one or more programming languages such as Python, Java, or Scala.
- Ability to design and implement scalable data pipelines for batch and real-time data processing.
- Experience with big data technologies such as Spark, Hadoop, Hive, EMR, etc.
- Experience working with modern data warehouse platforms like Amazon Redshift, Google BigQuery, or Snowflake.
- Experience with cloud platforms such as AWS, GCP, or Microsoft Azure.
- Experience in developing, debugging, and maintaining code in a large corporate environment.
- Overall knowledge of the Software Development Life Cycle.
- Solid understanding of agile methodologies such as CI/CD, Application Resiliency, and Security.
Preferred Qualifications, Capabilities, and Skills
- Knowledge of modern data lake table formats, i.e., Iceberg, Hudi, etc.
- Experience with Kubernetes for container orchestration, including deploying, scaling, and managing containerized applications.
- Certifications in relevant technologies or platforms, such as AWS Certified Big Data – Specialty, Google Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate, can be advantageous.
- Relevant industry experience, preferably in a data engineering role.