MSD Lead Data Engineer 

India, Maharashtra 

627723666

Job Description

Key Responsibilities:

• Lead the design, development, and implementation of data pipelines and ETL workflows using Matillion and Databricks.
• Architect and implement data solutions in AWS, including but not limited to Amazon S3, AWS Glue, Amazon Redshift, and Amazon RDS.
• Manage and optimize workflow orchestration using Apache Airflow to ensure reliable data processing and job scheduling.
• Collaborate with cross-functional teams to gather requirements and translate them into scalable data architecture and process designs.
• Mentor and guide junior data engineers, providing technical leadership and fostering a culture of continuous improvement and innovation.
• Ensure data quality and integrity by implementing best practices in data governance and validation.
• Monitor performance, troubleshoot issues, and optimize data systems for efficiency and scalability.
• Stay abreast of industry trends and emerging technologies to ensure continuous improvement of the data engineering practices.

Qualifications:

• Bachelor’s degree in Computer Science or related Engineering, or a related field.
• 10+ years of experience in data engineering or related roles, with a proven track record of leading technical teams.
• Extensive experience with AWS services, particularly in data storage, data processing, and database management.
• Hands-on experience with Matillion for ETL processes and data integration.
• Proficient in using Databricks for data analysis and processing with Apache Spark.
• Strong knowledge of Apache Airflow for workflow orchestration and scheduling.
• Experience with SQL and data modeling principles; familiarity with NoSQL databases is a plus.
• Excellent problem-solving skills and ability to work in a fast-paced, agile environment.
• Strong communication skills to effectively interact with technical and non-technical stakeholders.

Preferred Skills:

• Experience with Python, Pyspark, and SQL for data processing and scripting.
• Experience in AWS services,Databricks,Airflow & Matillion
• Familiarity with data visualization tools such as Power BI.
• Knowledge of machine learning practices and frameworks.

Current Contingent Workers apply

Preferred Skills:

*A job posting is effective until 11:59:59PM on the dayBEFOREthe listed job posting end date. Please ensure you apply to a job posting no later than the dayBEFOREthe job posting end date.