Microsoft Data Engineer II 

India, Telangana, Hyderabad 

922360848

As part of the Decision Sciences Engineering team in Azure, we are at the heart of Microsoft’s cloud data strategy.  Our mission is to deliver the data for our customers & partners by combining the best of Data science and Engineering culture of delivering platforms and services on Azure Data platforms. Our Culture : Growth mindset – we practice a learn it all vs a know-it-all attitude. Wins are cheered, failures are celebrated and viewed as learning opportunities.  Diverse and Inclusive – every voice matters and is appreciated, where you can express yourself. Leadership oriented – each of us are leaders who strive to create clarity, generate energy, and deliver success. We are looking for a Data Engineer, who can demonstrate the following traits:  • A team player who is self-starter, customer obsessed, highly inquisitive and biased towards action. • Comfortable with cold start problems, thrive in ambiguity with an eye for detail. • Delivers results through innovation and persistence.  • Demonstrate depth in areas of Data Engineering, especially depth in Spark, • Demonstrates a product mindset and looks to deliver re-usable components that can be deployed as-is or into existing pipelines. • Have a flair for Artificial Intelligence/Machine Learning technology.

Minimum Qualifications

• Bachelor's Degree in Computer Science, Data Science, Mathematics or equivalent field OR Master's Degree in Computer Science, Data Science, Mathematics or equivalent • A successful history of architecting, building, and maintaining end-to-end, high-throughput data lake systems and their supporting services. • Minimum of 8+ years of hands-on development experience designing and building data lakes/datawarehouses • 7+ years of experience developing Big Data applications in the cloud using technologies such as Kusto, Databricks, Spark, Azure Data Factory, Azure ML, Azure Synapse or equivalent. • 8+ years of hands-on coding experience on either Scala, Spark SQL, pyspark, good to have optimized Query writing, Performance Tuning, Troubleshooting & development experience. • 8+ years of hands-on data pipelines and ETL/ELT experience • 5+ years of experience in building OLAP systems like Azure Analysis Services, Power BI etc. • Strong understanding of Data modeling including Dimensional and relational modeling.

Preferred Qualifications

• Proficient working knowledge of cloud-computing environments in the data stack - Azure preferred.  • Utilizing a variety of data stores, including data lakes/warehouses, relational database systems, in-memory caches, and searchable document databases. • Developing for continuous integration and automated deployments.

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Understanding of modern data architecture patterns and concepts like Data mesh, Lambda Architecture, Delta lake etc. Understanding existing systems and lay out the path for a more modern and highly scalable data platform. • In depth understanding of Spark internals and passion for data engineering at scale. Identify, design, and implement process improvements to optimize our data delivery, and re-designing infrastructure and pipelines to achieve greater scalability. • Design, develop, and maintain data pipelines and back-end services for real-time decisioning, reporting, optimization, data collection, and related functions. • Ensure that all features embody engineering best practices, efficient, scalable architecture, elegant design, and high-quality implementation. • Collaborate with stakeholders, including Product, Executive and Domain teams to support them on data-related issues and data infrastructure needs. • Seek and incorporate continuous feedback to improve the efficiency of the business outcomes. • A team player who is self-starter, customer obsessed, highly inquisitive and biased towards action while engaging everyone in team. • Comfortable with cold start problems, thrive in ambiguity with an eye for detail. • Delivers results through innovation and persistence.  • A person with an eye for details and ‘can-do’ attitude. • Provide mentorship and guidance to other team members.