Core responsibilities: Design, own and maintain scalable and robust test automation frameworks, create services and tools for testing and monitoring of data processing infrastructure and of data monitoring reports. Provide feedback and reporting on data pipelines and inform partnering teams of issues or progress. Test the functionality and accuracy of data pipelines and data monitoring tools by leveraging Python and Spark. Understand complex software and data processing systems and be able to break the systems down into smaller components in order to deep dive into the implementation to ensure robust and reliable data processing produces the highest quality of data.