You will join the Workflow Management Developer Experience team in Big Data scope to support Migration from Hadoop / Oozie (Spark, Perl, etc) legacy setup to a modern platform (AWS / S3 / Snowflake / Airflow). The Workflow Management Developer Experience (WFME) team is an addition to the existing Workflow Management (WFM) to support the customer with new functionality and processes built on top of the platform. Existing team is building the platform capability, while the WFME team will be responsible for supporting customers with migration from the legacy scope, building processes, new operators and integrations required to make WFM service truly customer centric.
What you will be Doing:To support the migration we need to analyze the existing workflows and build tools for their migration. This involved reading code to understand nuances and translate them into technical requirements and implementations for the new platform, with a strong focus on Migration Tooling, SDKs and building operators. The scope is huge, around 4000 workflows. The ideal candidate should have some good examples where they would address a similar problem in the past and enjoy using the “Divide and Conquer” principle while doing this kind of work. They should be able to articulate their thinking to a wide range of partners such as Product, Engineering management, team members and users.
B.responsibleMentor and coach team members
Provide technical leadership to the team
Process unstructured input in order to transform it into a technical solution
Analyze Platform user experience (our users are Data Engineers) in order to improve their performance
Proactively identify work to be done and break it down into clear manageable tasks, so more junior members could pick them up
Reviewing work of other team members
Doing dogfooding of the platform
Develop necessary operators and integrations
Drive requirements for the migration tooling
Analyzing oozie jobs, finding in scheduling and compute usage
Proactively contribute to the workflow migration project
Implement user friendly Tooling to support oozie migration to Airflow taking into account the nuances of the legacy structures and the modern tech stack
Own end-to-end data and data applications by defining, monitoring and adjusting relevant SLIs and SLOs
Handle, mitigate and learn from incidents in a manner that improves the overall system health
Continuously evolve your craft by keeping up to date with the latest developments in data engineering and related technologies, introducing them to the community and promoting their application in areas where they can generate impact
Push for improvements, scaling and extending data engineering tooling and infrastructure, collaborating with central teams
Thrive on change and gets things done
Demand a high standard of excellence in their craft
Embrace the opportunity to improve
Understand success starts with accountability and ownership
Care more about being successful and reaching goals together than individually
Curious, experiments and continuously learns
Humble, open, friendly and remember that diversity gives us strength
Develop vital SDKs, operators and integrations
Drive requirements for the migration tooling
Analysing oozie jobs, finding in scheduling and compute usage
Proactively contribute to the migration strategy
Implement user friendly Tooling to support oozie migration to Airflow taking into account the nuances of the legacy structures and the modern tech stack
Train and support users with the migration stack:
Python
Spark
DBT or SQL
git CI / CD
AWS
Airflow
8+ years in software or data engineering, preferably with a focus on infrastructure or data engineering platforms.
SRE experience is a plus
Working in Platform or Developer Experience team is a plus
Experience working with Kubernetes
Experience deploying data processing solutions on cloud providers is a plus
Experience with big data tools (eg Spark) and ML frameworks (eg TensorFlow) commonly used for ML is a plus
Experience working with workflow management tools such as Apache Airflow is a plus
A deep understanding of software development in a team, and a track record of developing and shipping software
Strong hands-on technical skills in coding and system design.
You are required to live within a commutable distance from your assigned office location
Booking.com’s Total Rewards Philosophy is not only about compensation but also about benefits. Our Total Rewards are striven to make it easier for you to experience all that life has to offer on your terms, so you can focus on what really matters. We offer competitive compensation as well as thoughtful, valuable, and even fun benefits which include:
Medical, life, and disability insurance*
Annual paid time off and generous paid leave scheme including: parent, grandparent, bereavement, sick and care leave
Industry leading product discounts for yourself, friends, and family, including automatic Genius Level 3 status and quarterly Booking.com wallet credit
Free access to online learning platforms, mentorship programs, and a complimentary Headspace membership
Collaborative, friendly and diverse culture
Referral Program
For this role will have a salary range of: $207,600 - $228,000
Additional Annual or Quarterly bonus potential (role dependent)
*Please note that while our philosophy is the same in every location, benefits may differ by office/country.
Should you require accommodation to meet the essential functions of this job, please let us know.
Pre-Employment Screening
If your application is successful, your personal data may be used for a pre-employment screening check by a third party as permitted by applicable law. Depending on the vacancy and applicable law, a pre-employment screening may include employment history, education and other information (such as media information) that may be necessary for determining your qualifications and suitability for the position.
