Senior Data Engineer (AWS)

At Grape Up, we transform businesses by unlocking the potential of AI and data through innovative software solutions.

We partner with industry leaders, from the automotive and finance industry, to build sophisticated Data & Analytics platforms that transform how organizations manage and leverage their data assets. Our solutions provide comprehensive capabilities spanning data storage, management, advanced analytics, machine learning, and AI, enabling enterprises to accelerate innovation and make data-driven decisions.

Responsibilities 

• Implement a scalable architecture capable of handling the high volume of simulation data

• Build a flexible data preprocessing pipelines that are extensible and that can be integrated into customer’s existing platform

• Define KPIs to measure the improved reusability and automation of the new pipelines and test their performance in an end-to-end setting with model training.

• Develop and implement processes and best practices for data management and governance

• Optimize and enhance system setup and improve data structures following industry best practices

• Collaborate effectively with data engineering team members while partnering closely with analytics and data science teams to meet user needs

Requirements 

• PhD or master’s degree in computer science, Data Science, AI, or related field

• 5+ years of professional experience in Data Engineering and Big Data

• Proven experience in implementing and deploying solutions in AWS using AWS stack (Redshift, Kinesis, Athena)

• Proven experience with AWS Data Processing (Glue, EMR)

• Experience with Data Pipelines Orchestration (Airflow, Prefect)

• Experience with Big Data & Distributed Processing (Hadoop, Spark)

• Proven experience in designing and implementing data governance and data management frameworks

• Expert level knowledge of Python

• Automotive or IoT data processing experience

• Strong problem-solving skills and independent work ethic

• Fluency in English, both written and spoken

Nice To Have

• Experience with Infrastructure as Code (Terraform, CloudFormation)

• Experience with Kubernetes

• Experience with alternative data platforms (Databricks, Snowflake)

• Experience with Machine learning and MLOps