Data Scientist L5 - Customer Service Experimentation jobs at Netflix

In this role, you will:

• Design and develop systems and reusable frameworks for the full cycle of ML/AI R&D in the multimedia domain, including data processing, efficient distributed model training with GPUs, and deploying models into creator workflows and production systems.

• Collaborate cross-functionally with research scientists, product managers, and creative and business partners to define and prioritize system requirements.

• Work closely with ML/CV/Graphics researchers and Studio teams to productionize algorithms and models, and efficiently run human-in-the-loop inference tasks.

• Participate in model development and propose scalable designs.

• Champion and facilitate software engineering best practices within the team.

About you:

• Proven experience in software engineering within a production environment, including proficiency in object-oriented programming (Python, Java, or C++), CI/CD, etc.

• Familiarity with ML, CV, GenAI, and/or graphics pipelines, and experience with distributed data processing systems.

• Proficient in cloud infrastructure, including S3, Docker containers.

• Basic understanding of (multi) GPU training and inference for debugging and performance assessment, and CUDA runtime.

• Passionate about transforming ideas into products and enhancing user experience.

• Excellent communication and interpersonal skills, with a strong ability to navigate ambiguity.

Bonus experience:

• Experience building end-to-end multimedia systems and algorithms.

• Deep experience with AWS and understanding of FSx.

• Proficiency with deep learning frameworks such as PyTorch and TensorFlow.

• Experience in the content creation domain, such as visual effects or animation.

• Familiarity with generative models and tools, such as diffusion-based models and ComfyUI.

• Knowledge of graphics and VFX tools and/or game engines, such as Unreal, Houdini, or Nuke.

$150,000 - $750,000

Key Responsibilities

• Audience Analysis & Presentation:Utilize proprietary and syndicated tools to analyze and present Netflix's audience data, effectively communicating insights to the marketplace and advertiser clients.

• Support for Advertising Sales:Assist the Advertising Sales team with research and analytics, including audience data, cultural insights, campaign metrics, and campaign measurement solutions.

• Industry & Trend Analysis:Conduct research and analysis on industries, companies, business models, and marketing strategies across key advertiser verticals such as CPG, Retail, Tech, and Telecom to generate market-facing insights on current and emerging trends.

• Stakeholder Communication:Maintain effective communication with stakeholders by setting clear timelines and next steps, ensuring timely delivery of project objectives.

• Campaign Insights & Optimization:Integrate campaign measurement results to develop compelling narratives, present insights to clients and agencies, and provide guidance on how to shift their investments to Netflix.

• Causal Testing & Research Design:Design and execute advanced causal tests for Brand Effectiveness and Incremental Lift, along with quantitative and qualitative research on audiences, industries, and content.

• In-Depth Analysis & Insight Generation:Conduct thorough analyses of measurement outputs to deliver comprehensive and actionable insights.

• Learning Agendas & Playbooks Development:Contribute to the creation of advertiser and category-specific learning agendas and playbooks, incorporating innovative approaches.

• Case Study Development:Develop case studies based on ongoing and completed analyses to highlight successful strategies and outcomes.

Experience & Skills

• 4-6 years at a Streaming, TV, digital media company, or media / digital agency.

• Understanding of quantitative and qualitative methodologies within the advertising landscape.

• Experience in research methods and data analysis - 1st party data, customer research, qualitative and quantitative studies.

• Strong understanding of methodology, data, and systems from syndicated research providers (e.g. Nielsen Ibope, HR Media, GWI, TGI, comScore).

• Understanding of local video audience measurement systems.

• Understanding of the competitive streaming and TV marketplace.

• Intermediate understanding of digital media ad sales and marketplaces including Direct IO & Programmatic advertising.

• Self-starter and self-motivated with strong communication skills and expertise in managing larger-scale, complex projects.

• Expertise in advanced measurement methodologies is desirable (Reach & Frequency, Incremental lift, Brand lift, Matched-Market testing, Multi-Cell testing, A/B testing, Marketing Mix Models and bespoke campaign effectiveness studies).

• Comfort in analyzing large and disparate data sets to generate strategic and actionable insights.

• Proficiency with Microsoft Excel and PowerPoint & Data visualization tools (Tableau is desirable).

• Familiarity with Google Docs is desirable.

• Familiarity with concepts of database design and SQL is desirable.

• Able to communicate effectively in English

• Design and build scalable infrastructure solutions to enable reliable and secure access to source code, build artifacts and continuous integration.

• Develop software solutions to integrate with existing infrastructure to enhance and extend existing capabilities.

• Evolve and improve logging, monitoring, tracing, alerting and dashboarding tools to enable improved insight into our infrastructure.

• Collaborate with various other Platform teams to ensure their systems can integrate with the development infrastructure.

• Automate the deployment and upgrading of third-party solutions in order to keep them up-to-date seamlessly.

• Support customers with a variety of source control, build and environment needs.

• Support cross-organization critical change campaigns across a wide variety of source code repositories.

• Engage with vendors on license management and support issues and deliver feedback on product improvements to increase Netflix developer productivity.

• Participate in product reviews and team meetings, providing technical insight. Help scope, estimate, and prioritize between conflicting needs.

• Evaluate new technologies and approaches to streamline and improve our rapid application development tooling and conventions.

• Develop and execute disaster recovery planning and preparedness.

• Work with our security team on long-term security initiatives and promptly respond to security incidents.

• Regularly learn new systems and tools as the Netflix platform and ecosystem evolve.

• Participate in our on-call rotation and contribute to incident reviews.
  

• You are self-driven and highly motivated to deliver top-tier solutions with minimal guidance. You also recognize when you're wrong, learning from your mistakes and moving past them.

• You look beyond short-term fixes in favor of long-term solutions, make wise decisions despite ambiguity, and use data to inform your intuition.

• You enjoy collaborating and pairing with engineers to understand what they want, figure out what they really need, and finally, what we should build.

• You strive to embrace best practices and are always searching for opportunities to continually improve.
  

• You have experience operating third-party software at scale on a major cloud platform such as AWS, GCP, or Azure.

• You have experience with programming languages like Java, Python or Golang.

• You are comfortable scripting solutions using languages like Python or Bash.

• You have experience operating source control management systems in a cloud environment, like GitHub Enterprise Server (GHES) or Atlassian Bitbucket.

• You have experience operating an artifact management system like JFrog Artifactory in a cloud environment like AWS.

• You have experience with monitoring and alerting systems and how to integrate them with existing solutions.

• You are experienced with relational and non-relational data stores.

Our culture is unique, and we live by our values, so it’s worth learning more about

Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top-of-market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $100,000- $720,000.

| RESPONSIBILITIES

• Act as a high-responsibility service owner for a broad range of enterprise applications

• Relentlessly pursue improvements, with a keen eye for how engineering solutions reduce toil and eliminate manual tasks

• Design, implement, and maintain scalable and reliable infrastructure to support our services

• Develop, test, and deploy API integrations with other platforms and web services

• Implement and improve monitoring and alerting systems to proactively identify and address potential issues

• Partner closely with our Product Management and Netflix Technology Experience teams to identify support-enabling tools (i.e. self-service tools to help our Support teams scale)

• Administer, manage, and configure a variety of internal tools and SaaS applications such as Google Workspace, Slack, and Zendesk

• Partner with existing internal customer teams to assess opportunities for improvement, workflow efficiencies, and engineering solutions

| WE VALUE

• Strong problem-solving skills and the ability to make data-driven decisions

• Passion for engineering principles and developing tools and applications for impact with high quality

• Ability to thrive in a dynamic environment where needs shift and clearly articulate priorities and critical outcomes to your team

• Demonstrated sound judgment by taking a strategic approach to solving technical problems with a holistic view of the business impact

• Collaboration and continuous improvement are fundamental to growing the team’s impact over time

• A desire to learn and readiness to mentor others both within and outside of the team

| SKILLS AND EXPERIENCE

• 5+ years of experience as a Systems Engineer or in a similar role

• Strong scripting and programming skills (Python, Go, Java or JavaScript/)

• Expertise administering and developing solutions for enterprise software platforms like Slack & Google Workspace

• Experience with cloud platforms like AWS, microservices architecture

• Experience with low-code/no-code development platforms like Workato and Okta Workflows

• Experience with incident management and response

• Experience with employee onboarding and offboarding workflows

• Excellent communication & collaboration skills and a continuous improvement mindset

• Proven ability to cultivate relationships through influence

• Proven ability to assess complex issues and implement effective solutions

Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $100,000 - $700,000.

Job is open for no less than 7 days and will be removed when the position is filled.

• Partner closely with multi-disciplinary team members - Product Design, Content Design, Product Management, Engineering, Research, Operations, and others – to take the business to the next level;

• Work across all stages of the design process (user research, ideation, concept development, validation, workflow diagramming, prototyping, final visual design, and implementation support);

• Deeply understand users and their needs, including generative user research and exploration;

• Create beautiful, emotionally compelling, pixel-perfect mockups and prototypes of the end-to-end user experience;

• Conduct usability testing and iterate on designs with users;

• Partner with PMs to craft systematic solutions that connect the dots between user needs, data, business goals, and business strategy;

• Partner with engineers to build impactful products and features within the constraints of our technology and timeline;

• Facilitate collaborative work sessions with all levels of company stakeholders;

• Advance the discipline of product design within the organization and with partners and clients.

• 5+ years of experience in enterprise software product design;

• Track record of designing and shipping enterprise experiences that have scaled to thousands of usersand had successful outcomes for global users and the business; Ideally in the areas of GenAI or Creative Tooling

• Ability to think at a high level about product strategy and articulate a vision;

• Ability to navigate ambiguity, and drive clarity that brings the user, technology, and business together into an actionable roadmap;

• Experience connecting the dots across the organization with other related initiatives across the company while driving collaboration;

• Exceptional communication skills, in writing and verbally, whether in small group discussions, large-scale presentations, or with executives;

• Extraordinary thoughtfulness and craft in interaction design, visual design, and prototyping;

• Humble, collaborative approach to partnering with PMs, engineers, and business stakeholders to find the best solution together;

• Mastery of Figma’s advanced features such as Autolayout, Variables, DevMode, Responsive Prototypes, etc;

• Experience collaborating with machine learning and data science teams to influence algo design from a user-centered lens

• Experience working with off-shore development teams and a point of view on how best to collaborate with them.

• At Netflix, all roles are expected to exemplify the

• An outstanding portfolio, showcasing your strengths in visual design craft, interaction design logic, and the rationale behind your design decisions;

• Effective English verbal/written communication and presentation skills, with the ability to successfully articulate design concepts to your team and senior stakeholders;

• Strong ability to think technically and collaborate seamlessly with engineers and data scientists as it pertains to understanding the benefits, possibilities, and constraints of common coding languages and machine learning platforms;

• Effective time management skills to balance project schedule with fast-paced deadlines;

• Strong conceptual thinking that balances user needs, business goals, and feasibility;

Pay Transparency

Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top-of-market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $120,000 - $515,000.

See more detail about our Benefits

Job is open for no less than 7 days and will be removed when the position is filled.

| RESPONSIBILITIES

• Act as a high-responsibility service owner for a broad range of enterprise applications

• Relentlessly pursue improvements, with a keen eye for how engineering solutions reduce toil and eliminate manual tasks

• Design, implement, and maintain scalable and reliable infrastructure to support our services

• Develop, test, and deploy API integrations with other platforms and web services

• Implement and improve monitoring and alerting systems to proactively identify and address potential issues

• Partner closely with our Product Management and Netflix Technology Experience teams to identify support-enabling tools (i.e. self-service tools to help our Support teams scale)

• Administer, manage, and configure a variety of internal tools and SaaS applications such as Google Workspace, Slack, and Zendesk

• Partner with existing internal customer teams to assess opportunities for improvement, workflow efficiencies, and engineering solutions

| WE VALUE

• Strong problem-solving skills and the ability to make data-driven decisions

• Passion for engineering principles and developing tools and applications for impact with high quality

• Ability to thrive in a dynamic environment where needs shift and clearly articulate priorities and critical outcomes to your team

• Demonstrated sound judgment by taking a strategic approach to solving technical problems with a holistic view of the business impact

• Collaboration and continuous improvement are fundamental to growing the team’s impact over time

• A desire to learn and readiness to mentor others both within and outside of the team

| SKILLS AND EXPERIENCE

• 5+ years of experience as a Systems Engineer or in a similar role

• Strong scripting and programming skills (Python, Go, Java or JavaScript/)

• Expertise administering and developing solutions for enterprise software platforms like Slack & Google Workspace

• Experience with cloud platforms like AWS, microservices architecture

• Experience with low-code/no-code development platforms like Workato and Okta Workflows

• Experience with incident management and response

• Experience with employee onboarding and offboarding workflows

• Excellent communication & collaboration skills and a continuous improvement mindset

• Proven ability to cultivate relationships through influence

• Proven ability to assess complex issues and implement effective solutions

Our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $100,000 - $700,000.

Job is open for no less than 7 days and will be removed when the position is filled.

• Common Languages and Tools: Java & other JVM languages, Gradle, Nebula, Spring Boot, JUnit, GraphQL, Kafka, PostgreSQL.

• Implement and manage build solutions that enhance the efficiency and reliability of software delivery.

• Develop and maintain backwards-compatible tools for dependency management and analysis.

• Integrate internal and vendor-provided build and test infrastructure into engineering workflows, focusing on reliability and ease of use.

• Design and develop tools and infrastructure to automatically detect, quarantine, and reproduce flaky tests.

• Create and maintain tools for analysis of distributed tracing tools for test runs.

• Develop and integrate software solutions that provide high-quality synthetic test data generated from captured production traffic and API schema registries.

• Correlate test coverage data with code changes, runtime execution, and trace data for comprehensive reporting.

• Maintain a strong focus on scalability, usability, and reliability in platform design to support a growing cohort of engineers.

• Be willing and able to showcase our team’s offerings to internal audiences

• Consult with other teams about how to best collaborate, integrate, and/or set up solutions for their needs

• Stay informed about industry trends and emerging technologies to ensure Netflix remains at the forefront of build, dependency management, and testing infrastructure.

• A skilled software engineer with experience in developer platform or productivity teams.

• A meticulous software designer who researches and documents technical tradeoffs clearly and concisely.

• A self-motivated and organized individual who can independently drive engineering-wide solutions.

• A proactive communicator who engages effectively with technical and non-technical stakeholders.

• An advocate for strong build, dependency management, and testing practices, with familiarity in popular build tools, test frameworks, code coverage tools, continuous integration systems, and post-deployment verification methods (a healthy contempt for flaky tests is a plus).

• You have shipped and maintained Java code in production.

• You have worked on various technology stacks and are familiar with a variety of ways that software could be designed for different optimizations.

• You have designed and implemented build and dependency management solutions.

• You have built (and tested) custom Gradle plugins.

• You have assembled JVM Spring Boot applications using Gradle.

• You have experience with Nebula, Gradle, Maven, etc, for dependency management.

• You have experience with using and explaining Develocity dashboards.

• You are comfortable working with Zipkin or similar tools in the tracing space.

• You have implemented advanced log, metric, or error stacktrace analysis.

Job is open for no less than 7 days and will be removed when the position is filled.