Job responsibilities
- Deliver end-to-end data pipeline solutions on cloud infrastructure leveraging the latest technologies and best industry practices.
- Use domain modeling techniques to build best-in-class business products.
- Structure software for easy understanding, testing, and evolution.
- Build solutions that avoid single points of failure using scalable architectural patterns.
- Develop secure code to protect our customers and ourselves from malicious actors.
- Promptly investigate and fix issues, ensuring they do not resurface.
- Ensure releases happen with zero downtime for end-users.
- Optimize data writing and reading for our needs.
- Monitor performance, identifying and solving problems effectively and ensure systems are reliable and easy to operate.
- Continuously update technologies and patterns.
- Support products through their entire lifecycle, including production and incident management.
Required qualifications, capabilities and skills
- Formal training or certification in data engineering concepts and applied experience.
- Recent hands-on professional experience as a data engineer.
- Experience coding in a recent version of Python.
- Experience designing and implementing effective tests (unit, component, integration, end-to-end, performance, etc.).
- Excellent written and verbal communication skills in English.
- Experience with cloud technologies and distributed systems.
- Experience with data transformation frameworks and data pipeline orchestration tools.
- Experience managing large volumes of data and optimizing data processing.
- Understanding of event-based architecture, data streaming, and messaging frameworks.
- Ability to coach team members on coding practices, design principles, and implementation patterns.
- Stakeholder management and effective prioritization across multiple work streams.
Preferred qualifications, capabilities and skills
- Experience in a highly regulated environment/industry.
- Experience with AWS cloud technologies.
- Experience with data governance frameworks.
- Understanding of incremental data processing and versioning.
- Understanding of RESTful APIs and web technologies.