Finding the best job has never been easier
Share
As a Data Engineer, you will be responsible for building a reliable and efficient data platform to support Analytics and intuitive data products. This will enable our stakeholders to easily leverage data in an efficient and self-service manner. You will work on diverse, industry-leading data technologies such as Spark, Iceberg, Kafka, and various other AWS technologies to build a scalable and robust data platform supporting Analytics and Experimentation. Additionally, you will own the creation of end-to-end data pipelines that consolidate large data sets across disparate systems and combine them into easily consumable data sets, driving actionable insights for your customers.Key job responsibilities
- Design, develop, and maintain our PubTech data platform's processing and analytics needs.- Implement data quality processes, monitoring, and alerting mechanisms.
- Collaborate with data scientists, analysts, and product teams to understand their data requirements and translate them into efficient and reliable data pipelines.- Mentor and guide junior data engineers, fostering their professional growth and development.
- 3+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
- Knowledge of distributed systems as it pertains to data storage and computing
- Do you have experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets?
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence
These jobs might be a good fit