Lead Data Engineer

Join a global tourism leader as a Lead Data Architect to spearhead an extensive cloud data modernization program. The primary focus of this long-term mission is a complex migration to Databricks Unity Catalog, involving the transition of all data layers (landing, raw, prepared, reporting, services) from Hive Metastore. You will be responsible for re-architecting the Lakehouse structure, standardizing governance, and modernizing ETL/ELT pipelines in a high-stakes, professional environment.

Szczegóły wspólpracy

• Start Date: ASAP

• Duration: Long-term.

• Working Hours: 9:00 – 17:00 CET/CEST.

• English Level: Minimum B2+.

Responsibilities

• Architectural Leadership: Designing and developing data architecture within Azure and Databricks environments, serving as the technical lead for the project team.

• Unity Catalog Migration: Leading the migration of infrastructure, pipelines (DLT, Python/SQL jobs, Synapse/ADF), and metadata frameworks to Unity Catalog.

• Lakehouse Design: Implementing Medallion Architecture and Metadata-Driven ETL standards across the organization.

• Optimization & Standards: Fine-tuning data processing workflows (Python, SQL, Spark) and implementing robust logging, monitoring, and data quality frameworks.

• DevOps Mastery: Creating and implementing CI/CD processes and branching strategies using Azure DevOps and Git.

• Stakeholder Collaboration: Partnering with business and technical stakeholders to deliver top-tier solutions and mentoring junior team members.

Requirements

• Cloud Infrastructure: Advanced knowledge of Microsoft Azure (networking, authorization, data environment design).

• Databricks Expert: Deep proficiency in Unity Catalog, Delta Live Tables (DLT), Workflows, and workspace administration.

• Synapse Stack: Strong experience with Azure Synapse (Serverless and Pipelines).

• Development: Excellent Python skills and experience with Apache Spark (workflow design and code optimization).

• Data Modeling: Advanced SQL (query optimization, SQL Server) and experience designing scalable data warehouses.

• Modern Practices: Proven track record with Medallion Architecture, Metadata-Driven ETL, and Lakehouse design.

• Leadership: Previous experience in a Lead Engineer role, including technical decision-making and mentoring.

Nice-to-Have

• Familiarity with Power BI for reporting and analytics integration.

• Strong documentation skills and a passion for knowledge sharing.