We are seeking an experienced Confluent Kafka Administrator to manage and maintain our Kafka infrastructure. The ideal candidate will have a minimum of 5-8 years of experience with Confluent Kafka and a strong background in data streaming and real-time data processing.
Your key responsibilities
- Kafka Cluster Management: Install, configure, and manage Kafka clusters, including Confluent Kafka components.
- Monitoring and Performance Tuning : Monitor Kafka clusters for performance, reliability, and capacity planning. Optimize Kafka performance and troubleshoot issues.
- Security: I mplement and manage security measures for Kafka clusters, including encryption, authentication, and authorization.
- Data Integration: Work with development teams to integrate Kafka with various data sources and sinks.
- Upgrades and Patching : Plan and execute upgrades and patches for Kafka clusters to ensure they are up-to-date with the latest features and security fixes.
- Disaster Recovery : Develop and maintain disaster recovery plans for Kafka clusters.
- Documentation : Create and maintain documentation for Kafka configurations, processes, and procedures.
- Support: Provide support for Kafka-related issues and work closely with other teams to resolve any problems.
- Topic Management: Manage Kafka topics, including adding and removing topic configurations, setting cleanup policies, and configuring retention times.
- Support 24/7 shift coverage for BAU production support. Ready to work in Shift and flexible to work on weekends.
Skills and attributes for success
- Experience: Minimum of 5 years of experience with Confluent Kafka.
- Technical Skills: Strong knowledge of Kafka architecture, Kraft mode (Kafka Raft), Kafka Connect, Kafka Streams, and KSQL.
- Scripting and Automation: Proficiency in scripting languages (e.g., Python, Bash) and automation tools (e.g., Ansible, Terraform).
- Monitoring Tools: Experience with monitoring tools like Prometheus, Grafana, Dynatrace, and Splunk ITSI.
- Security: Knowledge of security best practices for Kafka, including SSL/TLS, Kerberos, and RBAC.
- Problem-Solving: Strong analytical and problem-solving skills.
- Communication: Excellent communication and collaboration skills.
To qualify for the role, you must have
- Certifications: Confluent Certified Administrator or Developer.
- Cloud Experience: Experience with cloud platforms (e.g., AWS) and managing Kafka in cloud environments.
- DevOps : Familiarity with DevOps practices and tools.
- Database Knowledge : Experience with databases such as PostgreSQL, MySQL, or MongoDB.
- Big Data Technologies : Familiarity with big data technologies like Hadoop, Spark, or Flink.
- Containerization and Orchestration : Experience with Docker and Kubernetes.
- CI/CD Pipelines: Knowledge of continuous integration and continuous deployment (CI/CD) pipelines.
- Agile Methodologies: Experience working in Agile development environments.
- Soft Skills: Strong leadership, project management, and mentoring skills.
What we offer
- Continuous learning : You’ll develop the mindset and skills to navigate whatever comes next.
- Success as defined by you : We’ll provide the tools and flexibility, so you can make a meaningful impact, your way.
- Transformative leadership: We’ll give you the insights, coaching and confidence to be the leader the world needs.
- Diverse and inclusive culture: You’ll be embraced for who you are and empowered to use your voice to help others find theirs.
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.