Required qualifications, capabilities, and skills
- Formal training or certification on Data engineering concepts and 3+ years applied experience
- Strong programming skills in Python, with experience in developing and maintaining production-level code.
- Proficiency in working with large datasets and data preprocessing.
- Solid understanding of AI/ML algorithms and techniques, including deep learning, time series forecasting and natural language processing.
- Experience with cloud platforms, such as AWS for deploying and scaling AI/ML models.
- Experience with ETL tools such as Airflow.
Preferred qualifications, capabilities, and skills
- Experience in backend development, including databases (SQL/NoSQL/Graph), programming languages (Python/Java/Node.js), web frameworks, APIs, and microservices and possess front-end development skills, including HTML, CSS, and JavaScript
- Knowledge of SRE practices. Experience working with AWS EKS, ECS, RDS and DynamoDB.
- Knowledge of large language models (LLMs) and accompanying toolsets the LLM ecosystem (e.g. Langchain, Vector databases, opensource Hugging Face Models.
- Exposure to cloud automation technologies such as Terraform