Design and implement scalable data architectures and robust data pipelines to process high-volume IoT sensor data, ensuring reliable data capture and processing for AI/ML workloads, including training data preparation and feature engineering.
Develop and optimize RAG (Retrieval Augmented Generation) systems and data integration solutions that enhance the retrieval and use of combined industrial IoT and enterprise data sources for AI model training and inference.
DataOps:
Implement DataOps practices for the continuous integration and delivery of data pipelines that support AI solutions, including automated testing frameworks for data quality and AI model performance monitoring.
Create self-service data assets and automated documentation systems to enhance data accessibility, lineage, and AI model provenance for data scientists and machine learning engineers.
Collaborate with ML engineers and data scientists to optimize data workflows for model training and deployment, fostering continuous improvement in data engineering practices while adapting to the dynamic AI landscape.
Key Skills and Qualifications
Bachelor’s degree from an accredited institution in a technical discipline such as science, technology, engineering or mathematics
Data engineering experience with strong understanding of CDC, ELT/ETL workflows, streaming replication, and data quality frameworks. Hands-on experience with PySpark/Scala, and experience with cloud platforms (Azure/GCP/Databricks), particularly in implementing AI/ML data workflows
Possesses a strong understanding of data modelling for analytical and AI workloads, along with expertise in implementing RAG architectures, working with LLM-powered applications, and building data pipelines for MLOps practices and AI model deployment.
Skilled in real-time data processing frameworks such as Apache Spark Streaming, experienced with time-series databases and IoT data modelling patterns, while also knowledgeable in data quality implementation for AI training data and graph databases for AI applications.
Familiar with containerization (Docker) and orchestration (Kubernetes) for AI workloads, with a strong background in data security and governance practices, and experienced in working with distributed teams using Agile and Scrum methodologies
Our Offer
A culture that fosters inclusion, diversity, and innovation in an international work environment
Market specific training and ongoing personal development.
Experienced leaders to support your professional development