Data Engineer

At Madiff, we connect top IT talent with cutting-edge companies through remote Agile teams and nearshore outsourcing. With operations in Poland, Portugal, France, and the UK, we specialize in AI, DevOps, and Cybersecurity, supporting industries like Telecom, Banking, and Hi-Tech.

We are currently looking for an experienced Senior Data Engineer to join a cross-functional team responsible for designing and implementing large-scale, cloud-native data platforms. The project is delivered in cooperation with an enterprise client from the consulting and professional services industry.

You will take part in designing and building robust data lakes and streaming pipelines using Azure cloud services and Databricks. Your work will directly support product and business owners in building next-generation data and AI platforms.

Main responsibilities:

• Design and develop scalable ETL/ELT data pipelines in cloud environments
• Build and optimize data processing workflows using Spark and PySpark
• Configure and manage Azure services including Databricks, ADLS, ADF, and Azure Functions
• Apply strong data modeling principles and performance tuning techniques
• Ensure data quality, versioning, lineage, and security compliance
• Collaborate with stakeholders and cross-functional teams in an Agile setup

Requirements:

• Minimum 4 years of experience building data solutions on cloud platforms (preferably Azure)
• Deep knowledge of Azure Databricks, Spark, PySpark, and Delta Lake
• Strong expertise in Azure Data Factory and Azure Functions for automation
• Advanced SQL skills with experience in performance tuning and partitioning strategies
• Solid Python skills, especially in data handling libraries (pandas, numpy)
• Practical experience with data versioning, lineage, and quality frameworks

Nice to have:

• Experience working in Agile teams for enterprise-grade projects
• Familiarity with Git-based development workflows
• Understanding of data formats such as Parquet, Avro, and JSON

What we offer:

• 100% remote work with occasional visits to the client’s office
• Long-term cooperation in an international, multicultural team
• Participation in high-impact projects involving cloud, data, and AI technologies
• Private medical care