Share
Key job responsibilities
- Design, build, and operate highly scalable, fault-tolerant data processing systems using modern AWS services like Redshift, S3, Glue, EMR, Kinesis, and Lambda, and orchestration systems using Airflow
- Leverage your expertise in Python, Scala, or other modern programming languages to develop custom data processing frameworks and automation tools
- Collaborate closely with data scientists, analysts, and product managers to understand business requirements and translate them into technical solutions
- Continuously optimize data pipeline performance, reliability, and cost-efficiency using best practices in CI/CD and infrastructure-as-code
- Mentor junior engineers and share your knowledge of data engineering best practices
- Stay up-to-date with the latest big data technologies and architectural patterns, and drive the adoption of innovative solutions
- 1+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala)
- Experience with one or more scripting language (e.g., Python, KornShell)
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience with any ETL tool like, Informatica, ODI, SSIS, BODI, Datastage, etc.
These jobs might be a good fit