Architect and implement: Design and develop scalable, cost-effective, and secure distributed architectures and solutions, focusing on data ingestion and processing, utilizing appropriate GCP services and technologies
Develop data pipelines: Design, implement, and maintain data pipelines that efficiently collect, process, and store large volumes of data from various sources, ensuring data quality and integrity
Monitor, analyze, and optimize data pipelines to improve performance, reduce costs, and ensure reliability and scalability
Ensure that data solutions comply with relevant data security and privacy regulations, and implement best practices for securing data at rest and in transit
Mentor junior members of the team, sharing your experience, helping with any blockers
Required qualifications, capabilities, and skills
Formal training or certification on software engineering concepts and applied experience
Programming languages: Actively writing code & proficiency in one or more languages commonly used in data engineering , such as Python, R, Java, or Scala. Strong problem-solving and critical-thinking skills, with the ability to break down complex problems and develop innovative solutions
Data engineering skills: Experience in designing, building, developing, and optimizing data pipelines, as well as experience with big data processing tools like Apache Spark, Hadoop, and Dataflow
Experience in designing & operating Operational Datastore/Data Lake/Data Warehouse platforms at scale with high-availability
Data integration: Familiarity with data integration tools and techniques, including ETL (Extract, Transform, Load) processes and real-time data streaming
Cloud platforms expertise: Good understanding of GCP/AWS data engineering services, architectures, and best practices, with hands-on experience in designing and implementing scalable and cost-effective solutions.
Data storage and databases: Knowledge of various data storage options (e.g., relational databases, NoSQL, data lakes) and hands-on experience with managing and optimizing databases such as PostgreSQL, MySQL, BigQuery, and Redshift
Leadership and communication: Ability to lead a team of engineers, collaborate effectively with cross-functional teams, and communicate complex technical concepts to both technical and non-technical stakeholders.
A desire to teach others and share knowledge. We aren’t looking for hero developers, more for team players. We want you to coach other team members on coding practices, design principles, and implementation patterns
Comfortable in uncharted waters. We are building something new. Things change quickly. We need you to learn technologies and patterns quickly