Senior Data Engineer

Senior Data Engineer @ Limango

Remote/Wroclaw

B2B contract – daily rate 900-1250pln

• Managing data infrastructure (Terraform AWS / Databricks) and ETL code (PySpark, Delta Live Tables, SparkSQL)
• Development of Central Lakehouse Data Platform based on AWS / Databricks ecosystem
• Implementing monitoring, data quality tests, unit testing for code, and automated alerts into the system
• Refactoring legacy AWS solutions (AWS Glue, Redshift) into monitored and CI/CD deployed Lakehouse ecosystem with data expectations / data quality testing
• Active support for building medallion lakehouse architecture for ML and analytics data and introducing data-mesh concepts
• Active support for business analytics and ML teams working on the data platform

You will need

• Terraform, PySpark experience is a must at least on the basic lvl
• degree in Information Systems, Computer Science, Mathematics/Physics, Engineering, or a related field
• several years of experience building data engineering solutions (ideally in E-commerce)
• very good knowledge of Python programming (building applications, automated testing, packaging and deployment) 
• real experience with PySpark data processing and good understanding of Spark processing engine architecture
• Advanced SQL processing skills
• experience with data lakes architecture and data modelling paradigms
• experience with Terraform infrastructure-as-code and CI/CD deployments
• experience with building and deploying data engineering ETL applications – CI/CD pipelines in git environments 
• Fluency in English – we work in an international environment
• Team player attitude and willingness to learn modern technologies / frameworks

Nice-to-have

• Familiarity with AWS Cloud architecture and Databricks platform will be very welcomed
• Experience with AWS Glue and Redshift warehousing
• Experience with CloudFormation templates
• Experience with AWS Kinesis and Spark streaming processing
• Good knowledge of Scala coding, functional programming and application development  
• Experience with MLOps environments (e.g., MLFlow)  

Benefits:

• Access to private healthcare, ensuring the well-being of you and your loved ones.
• Language classes in English and German, tailored to your skill level.
• Remote work opportunities and flexible working hours to accommodate your schedule.
• Co-finance of Multi-sport card
• Office located in the vibrant center of Wrocław, with nearby amenities including cinemas, fitness clubs, and a variety of lunch options.

After hours we often organize interesting outings or meetings in our office or near by 😉