Lead Data Engineer – Python

Dear Consultant,

We are seeking an experienced Lead Data Engineer with strong expertise in Python and modern data engineering tools to design, build, and optimize data pipelines and infrastructure. This role requires a mix of technical excellence, leadership skills, and the ability to collaborate with cross-functional teams including data scientists, analysts, and business stakeholders. Send CV to (marcillina.tiejen@dcvtechnologies.co.uk) if you are interested.

Location:r Remote Poland

]Experience Level: Lead (8+ years)

Contract Type: [Permanent / B2B Contract]

Key Responsibilities

• Design, build, and maintain robust data pipelines for real-time and batch processing.

• Collaborate with data scientists and analysts to understand data needs and deliver efficient solutions.

• Implement data models to support analytical and operational use cases.

• Ensure data governance, quality, and security best practices are followed.

• Leverage Azure cloud services (ADLS, Data Factory, Databricks, Functions, Service Bus) to develop scalable solutions.

• Work with big data technologies including Apache Hadoop and Apache Spark.

• Optimize database performance and manage Azure SQL Server.

• Use version control tools (Git) for collaborative development.

• Troubleshoot and resolve data-related issues quickly and effectively.

Required Skills

• 8+ years of experience in data engineering roles.

• Proficiency in Python (PySpark); additional skills in Java, SQL, or Scala beneficial.

• Experience with Azure data services, including ADLS, Data Factory, Databricks, Functions, and Service Bus.

• Strong understanding of big data frameworks (Hadoop, Spark).

• Solid knowledge of data modeling, ETL processes, and data warehousing.

• Good understanding of machine learning basics to support integration with data science.

• Excellent communication skills in English, both written and verbal.

• Strong analytical and problem-solving capabilities.