

Share
What you’ll be doing:
Optimize distributed computing infrastructure by analyzing cost and right-sizing for latency and performance.
Identify benchmarks and establish metrics for tracking on dashboards and alerting.
Improve and sustain a strong, expandable, dependable, live data processing service.
Develop reusable framework deployments for data ingestion, processing, and analysis.
Build data systems and pipelines, ensuring that data sources, ingestion components, transformation functions, and destinations are well understood for implementation.
Prepare data for prescriptive and predictive modeling by ensuring the data is complete, cleansed, and has necessary rules in place.
Train data engineers, data scientists, and production engineers on adopting data processing workflows.
What we need to see:
Master's or Bachelor's degree in Computer Science, Electrical Engineering, or CE, or equivalent experience.
Minimum of 5 years of work experience in similar domain.
Built required infrastructure for optimal extraction, transformation, and loading of data from various sources using AWS, Azure, SQL, or other technologies, and knowledgeable in Databricks, Splunk, or Snowflake solutions.
Strong coding skills, including the ability to write readable, testable, maintainable, and extensible code (primarily Python).
Strong Kubernetes experience on-premise and/or CSP, developing containerized microservices.
Hands-on experience in performancetuning/troubleshootingspark applications.
Familiarity with metrics collection, health monitoring, and observability tools.
Strong experience in data cleaning, aggregation, transformation, and extraction.
Experience in active ML production pipelines is a plus (MLflow, Kubeflow).
Ways to stand out from the crowd:
Strong ability to drive continuous improvement of systems and processes.
Prior data processing at scale on NVIDIA GPUs.
Proven ability to work in a fast-paced environment where strong organizational skills are essential.
Expertise in optimizing Spark applications and micro services for enhanced performance and scalability.
You will also be eligible for equity and .
These jobs might be a good fit