EY Senior Consultant -Data Engineer - PySpark Databricks Specialist Mumbai 

India, Maharashtra, Mumbai 

301529181

The opportunity : Senior Consultant -Data Engineer - PySpark and Databricks Specialist - Mumbai

CNS - Risk - Internal Audit :

Financial Services Risk - helps clients in the financial sector to identify, manage and monitor the risks in their business. It is done through a transformative and agile approach including process and procedures, risk, data modeling and analytics, reporting to stakeholders and third parties, business requirements (for software selection and IT implementation), analysis of assets and liabilities, and capital and liquidity management.

Your key responsibilities

Key Responsibilities:

• Develop and maintain scalable data pipelines within Databricks and ensure the seamless flow of data across our data ecosystem.
• Utilize PySpark to process large datasets efficiently and perform complex data transformations.
• Collaborate with data scientists and analysts to support data modeling, data mining, and machine learning efforts.
• Optimize data retrieval and develop ETL processes to improve data quality and efficiency.
• Implement best practices for data security, compliance, and governance within the Databricks environment.
• Troubleshoot and resolve any issues related to data performance and integrity.
• Stay current with emerging technologies and industry trends to enhance our data capabilities.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Minimum of 3 years of experience as a Data Engineer with a focus on PySpark and Databricks.
• Proficient in SQL, data modeling, and schema design.
• Experience with cloud services (AWS, Azure, GCP) and understanding of cloud architecture.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.

Preferred Skills:

• Experience with additional big data tools (Hadoop, Kafka, etc.).
• Familiarity with data visualization tools and techniques.
• Knowledge of machine learning frameworks and libraries.

What we look for

People with the ability to work in a collaborative manner to provide services across multiple client departments while following the commercial and legal requirements. You will need a practical approach to solving issues and complex problems with the ability to deliver insightful and practical solutions. We look for people who are agile, curious, mindful and able to sustain postivie energy, while being adaptable and creative in their approach.

What we offer

If you can confidently demonstrate that you meet the criteria above, please contact us as soon as possible.