Senior Data Engineer (Azure Databricks)

Location: 1–3 days per week from the Client’s office in Wroclaw

Tasks:

• Design, develop, and optimize scalable data warehouses and ETL/ELT pipelines in Azure Databricks using PySpark.

• Build reliable and efficient data pipelines (batch and streaming) supporting analytical and reporting environments.

• Implement data solutions within the Azure ecosystem, ensuring adherence to cloud best practices.

• Integrate various structured and unstructured data sources and formats such as XML, JSON, CSV, Parquet, and flat files, using APIs, message queues, streaming platforms, and database replication.

• Participate in BI architecture design based on Kimball and Inmon methodologies, including data modeling for analytics.

• Ensure data quality, consistency, performance, and security (validation, monitoring, performance tuning).

• Collaborate within Agile teams; use Azure DevOps for CI/CD, version control, and code reviews while actively contributing to knowledge sharing.

• (Nice to have) Support data visualization and reporting using Power BI (datasets, model optimization, publication).

Requirements:

• Strong theoretical background in Business Intelligence and deep understanding of data warehousing concepts (Kimball, Inmon).

• Fluency in SQL and hands-on experience with SQL databases such as MS SQL Server and other RDBMS.

• Extensive senior-level experience with Azure Databricks, including development, optimization, and deployment of data pipelines and ETL processes using PySpark.

• Proven experience with Azure Cloud services and the ability to design and implement data solutions within the Azure ecosystem.

• Hands-on experience with multiple data formats (XML, JSON, CSV, Parquet, flat files) and both structured and unstructured data sources.

• Strong knowledge of ingestion techniques: APIs, message queues, streaming platforms, and database replication.

• Experience with Azure DevOps (CI/CD pipelines, version control) and solid understanding of Agile methodologies.

• Demonstrates a strong growth mindset, continuous learning approach, adaptability to new technologies, and willingness to share knowledge.

• (Nice to have) Experience with front-end BI tools, preferably Power BI.

Offer:

• Medicover healthcare package

• Multisport card

• Access to an e-learning platform

• Group life insurance