Microsoft Senior Software Engineer Big Data Systems 

United States, New York, New York 

756399462

is an online advertising platform, where advertisers bid to display brief ads, service offers, product listings and videos to web users. Our data-enabled technology platform, encompassing Invest, Monetize, and Curate, optimize return on investment for both buyers and sellers, while maintaining a commitment to an open marketplace and empowering the open web globally.

As a
Senior Software Engineer, Big Data Systems,you will operate and support our diverse Big Data Platform. This platform consists of large Hadoop, HBase, and Kafka clusters in anall Linuxenvironment. The platform currently ingests 300TB of new data and runs 20,000 ETL jobs every day across 14 Hadoop, HBase, and 6 Kafka Clusters.

This growing team consists of curious, passionate, talented technologists who enjoy working on complex, large scale data stores. Our team members thrive in a learning and teaching environment. Each team member is encouraged to explore solutions and efficiencies to support, optimize, and maintain our systems. We are enthusiastic about automation and optimization.

Your primary responsibilities will involve supporting and enhancing data stores for extremely intricate, high-availability, low-latency, business-critical real-time systems. This will entail utilizing contemporary open-source tools and architectural approaches, working closely with other highly skilled engineering teams, honing your problem-solving abilities in complex scenarios, and becoming proficient in managing large-scale data stores.

We anticipate exponential growth in data and transactions on our platform and so we are seeking an engineer who is excited about this opportunity and would love to:

• Work on large-scale, distributed systems
• Learn new technologies, tools and applications

Your primary responsibilities will involve coding, support, upgrades, troubleshooting, performance tuning, maintenance and automation. This will entail utilizing contemporary open-source tools and architectural approaches, working closely with other highly skilled engineering teams, honing your problem-solving abilities in complex scenarios and becoming proficient in managing large-scale data stores.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Qualifications:

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience
• 4+ years of relevant experience in implementing, troubleshooting, and supporting distributed systems such as Hadoop.
• 4+ years of relevant experience in scripting/writing/modifying code for monitoring/deployment/automation in one of the following (or comparable): Python, Shell, C#, Java.
• 4+ years of relevant experience in implementing, troubleshooting, and supporting the Unix/Linux operating system with concrete knowledge of system administration/internals.

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications

• 5+ years of relevant experience with any of the following technologies: Hadoop-HDFS, Yarn-MapReduce, HBase, Kafka, Trino, Cassandra/ScyllaDB,IRONdb, Druid, QFS.
• 5+ years of relevant experience with any of the following technologies: Puppet or equivalent configuration management tool.
• 5+ years of relevant experience with any of the following container technologies: Kubernetes or Docker.
• Experience with Cloud Computing such as Azure, AWS or Google Cloud Platform.
• Experience with Nagios or similar monitoring tools.
• Experience with data collection/graphing tools like Graphite and Grafana.
• Understanding of code versioning tools such as git.

Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Microsoft will accept applications for the role until July 13, 2024.

• Support a complex Data Pipeline Platform by monitoring,maintaining, provisioning, and upgrading Hadoop, HBase, Kafka, Trino, Cassandra/ScyllaDB,IRONdb, Druid, QFS, Metrics, and ETL systems using proprietary automation tools.
• Develop new tools to automate routine day-to-day tasks, such as security patching, software upgrades, and hardware allocation.Utilize automated system monitoring tools to verify the integrity and availability of all hardware, server resources, and critical processes.
• Code scripts to automate andoptimizeoperational processes.
• Troubleshoot and analyze hardware or software failures and provide solutions for recovery.Identifyand resolve faults, inconsistencies, and systemic issues.
• Collaborate with engineering team partners to resolve complex system performance issues.
• Prepare documentation for standard operating procedures.
• Effective with time management and ability to manage priority tasks and requests.
• Provide production support andparticipatein on call rotation.
• Embody our