Your role and responsibilities
Build, test, and maintain the data pipelines required for effective data ingestion and integration into the AWS datalake.
Key Responsibilities:
- Develop and deploy data ingestion processes for structured and unstructured data.
- Implement best practices for data quality and error handling.
- Work with the Data Architect to optimize performance and scalability.
- Ensure compliance with governance and security protocols.
Required education
Associate's Degree/College Diploma
Preferred education
Bachelor's Degree
Required technical and professional expertise
- 5-8 years of hands-on experience in data engineering.
- Proficiency in AWS data services (AWS Glue, AWS S3, and optionally AWS Lake Formation).
- Strong programming skills in languages such as Python or Java.
- Experience in ETL/ELT processes and data quality assurance.
Software Experience:
- AWS Glue, AWS S3
- Familiarity with AWS Lake Formation is a plus
- Knowledge of data engineering from an analytics, ML, data science use case and data need fulfillment is necessary.
- Experience is building reusable templates and assets on AWS is essential.
