המקום בו המומחים והחברות הטובות ביותר נפגשים
As a Sr Data Engineer, you will help define, design, implement and test the data process and infrastructure to productionalize and monitor this use case.You will be working with a large and complex dataset. You should have deep expertise in the design, creation, management, and business use of extremely large datasets. You should be expert at designing, implementing, and maintaining stable, scalable, low cost data infrastructure.Key job responsibilities
Deliver large data solutions in difficult or ambiguous data areas (new or existing systems).
Provide significant knowledge/experience with data design approaches and industry technologies.
Evaluate end-to-end data designs for strengths and weaknesses (data quality, scalability, latency, security, performance, data integrity, etc.).
Anticipate data access patterns and remove bottlenecks. Ask the right questions and drive the right technical solution(s).
Split development work into parallel tasks that can be performed by them and others and reassembled successfully.
Influence team technical strategy. Understand that not all problems are new (or require new data solutions). Make appropriate architectural trade-offs. Show good judgment when making technical trade-offs between short-term technology needs and long-term business needsDrive data engineering best practices and set standards.
Understand system limitations, scaling factors, boundary conditions, and/or the reasons for architectural decisions.
- 5+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with SQL
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
- Experience mentoring team members on best practices
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience operating large data warehouses
- Experience with Redshift and SQL
- Previous experience working with AWS SageMaker
- Experience dealing with manufacturing data at scale
משרות נוספות שיכולות לעניין אותך