Big Data Engineer ConsultantWe are looking for a Big Data Engineer to create solutions suitable for the job in a real-time, highly data-driven organization and make technical innovation happen. With a broad range of projects to develop data processing applications, you will have the opportunity to use cutting-edge technology across all stages of the lifecycle to support our projects related to NLP, data science models and marketing promotions.
What are our goals in Marketing?
- Drive perception
- Drive demand
- Grow the buyer base
- Build the best content
Key responsibilities:
You will work with other engineers and data scientists to solve some of the hardest business problems. You will learn what it takes to build, deploy & scale ML/AI models in real-world (it’s a hard problem, we assure you). You will build analytical data sets on which model’s will be built. This would entail two key tasks – feature engineering on large datasets and optimization of our pipelines to drive scalability. If you are already good at it, we will make you better.
You will:
- Feature engineering: Stitching together and aggregation of multiple large data sets working with the data scientist to a desired format. You will need to optimize both queries and architecture to support big data sets. SQL is usually the preferred language, but some NoSQL or Spark jobs are necessary.
- Implement complex Data pipelining / ETL process: Once the initial dataset are prepared, it is normally run through a further pipeline to prepare it for modelling. Here you will create features that make ML/AI algorithms work (e.g., translating text into category variables). Python/R is usually the preferred language.
- Develop and maintain documentation.
- Producing ad hoc datasets and reports.
Essential Requirements:
- 8 to 10 years of related experience in a professional role with a Bachelor’s degree in Business, Computer Science, Engineering,; or 6 to 8 years with a Master’s degree in Business, Computer Science, Engineering,
- Experience as a technical leader embedded with Data engineering teams operating in Agile methodologies such as Lean Software Development, Extreme Programming or equivalent.
- Self-directed and ability to build products that drive tangible user outcomes in a fast-paced environment.
- Strong and demonstrable external client-facing skills specifically in the areas of communication, issue, and escalation management, with demonstrated ability to drive cross-functional decision making for highly technical solutions.
Desirable requirements:
- Experience with data processing software (such as Hadoop, Spark, Pig, Hive), along with data processing algorithms (MapReduce, Flume).
- Exposure to machine learning or machine learning pipelines is a plus.
Salary starting from: 4 400 EUR gross monthly
1st May 2024