Responsibilities:

· Build and manage Kafka based streaming data pipelines

· Build and manage Airflow based ETLs

· Monitor and improve performance of the pipelines

· Develop storage layer (relational / non-relational) for hosting data for analytical queries

· Work on fast pace projects to deliver innovative solutions for our business users

Requirements:

· Bachelor’s degree in computer science/software engineering or related fields with 3+ years of experience in data engineering

· 3+ years of experience working with Python, NodeJS

· 2+ years of experience working with Kafka based pipeline development

· Strong knowledge of SQL is required

· 1+ year of hands-on experience with Apache Airflow

· Ability to debug production issues using tools like Grafana, Splunk or server logs

Nice to have

· Experience with distributed computing using Hadoop ecosystem, Spark, Presto.

· Knowledge of Java, Scala

· Knowledge of Kafka connect ecosystem

· Experience with Docker, Kubernetes