Data Engineer with Pyspark & AWS Glue

We are looking for a Data Engineer to build with our client data platforms in AWS, leveraging Python and Spark. Our customer is a global financial markets infrastructure and data provider. 

You will design, implement and optimize ETL workflows using Python and Spark, contributing to our robust data lakehouse architecture on AWS.

Your main responsibilities for this position will be:

• Designing, building, and maintaining robust, scalable and efficient ETL pipelines using Python and Spark.
• Developing workflows leveraging AWS services such as EMR Serverless, Glue, Glue Data Catalog, Lambda and S3.
• Implementing data quality frameworks to ensure reliable data processing.
• Optimizing existing workflows and driving migration to a modern data lakehouse architecture, integrating Apache Iceberg.
• Monitoring system performance and ensure data reliability through proactive optimizations.

This offer will be a perfect match for you if you have:

• At least 5 years of practice in related roles.
• Expertise in Python and Spark, with proven experience in designing and implementing data workflows.
• Hands-on experience with AWS EMR Serverless, Glue, Glue Data Catalog, Lambda, S3, and EMR.
• Strong understanding of data quality frameworks, governance practices and scalable architectures.

It is worth joining us because of:

• Flexibility - working hours are flexible just like the work mode, you can work remotely or in a hybrid model from one of our modern offices in Warszawa.
• Great atmosphere - we value a friendly, informal atmosphere, and direct contact with everyone in the company.
• Outstanding People - we understand that great teams are about personalities, not just skills. Therefore our team accommodates a fantastic blend of individuals and management that removes roadblocks.
• Modern technologies - we use proven technologies that are currently up-to-date. Even if you have not used all of them, you can make up for it with us!
• Unlimited possibilities - you’ll get the opportunity to develop your qualifications thanks to sponsorship for the industry's meetups and conferences as well as working on challenging international projects with the latest technologies. 
• Private medical care and Multisport - we care about your health and wellbeing so you’ll get access to private medical care for you and your family, and partial funding for a sports card.