As a
Job responsibilities
- Contribute to end-to-end data pipeline solutions on cloud infrastructure leveraging the latest technologies and the best industry practices
- Use domain modelling techniques to allow us to build best in class business products.
- Structure software so that it is easy to understand, test and evolve.
- Build solutions that avoid single points of failure, using scalable architectural patterns.
- Develop secure code so that our customers and ourselves are protected from malicious actors.
- Promptly investigate and fix issues and ensure they do not resurface in the future.
- Make sure our releases happen with zero downtime for our end-users.
- See that our data is written and read in a way that's optimized for our needs.
- Keep an eye on performance, making sure we use the right approach to identify and solve problems.
- Ensure our systems are reliable and easy to operate.
- Keep us up to date by continuously updating our technologies and patterns.
- Support the products you've built through their entire lifecycle, including in production and during incident management
Required qualifications, capabilities & skills
- Formal training or certification on data engineering concepts and applied experience
- Recent hands-on professional experience as a data engineer
- Experience in coding in a recent version of the Python programming language
- Experience in designing and implementing effective tests (unit, component, integration, end-to-end, performance, etc.)
- Excellent written and verbal communication skills in English
- Understanding of cloud technologies and distributed systems
- Understanding of data transformation frameworks and data pipeline orchestration tools
- Understanding of how to manage and optimize queries for large volumes of data within different kinds of data stores (including relational data stores)
- Experience with ML products/services and data would be best
- Experience with either of the two is good
- Experience with Python is fine too and willingness to learn
Preferred qualifications, capabilities & skills
- Experience in working in a highly regulated environment / industry
- Understanding of AWS cloud technologies
- Understanding of data governance frameworks
- Understanding of event-base architecture, data streaming and messaging frameworks