Senior Azure Data Engineer with Databricks

General Information:

Industry: IT

Location: remote

Rate: up to 176 PLN/h

Form of cooperation: B2B

Summary: The primary objective of this role is to ensure the efficient and effective management of data pipelines using Databricks and the Azure Analytics stack. This position emphasizes the need for a deep understanding of modern features, strong software engineering skills, and a collaborative approach to enhance data solutions within the organization.

Responsibilities:

• Curate structured, semi-structured, and unstructured data by creating efficient, cost-effective, and scalable pipelines leveraging Databricks and Azure Analytics stack.

• Create and maintain robust data pipeline architecture, ensuring data quality, reliability, and scalability.

• Assemble and manage large, complex data sets to meet functional and non-functional business requirements.

• Identify, design, and implement process improvements, including automation of manual processes and optimization of data delivery.

• Work closely with Product Owners, Scrum Masters, and Enterprise Data Management leadership to ensure timely and high-quality delivery of data solutions.

• Align with Data Engineering chapter standards and processes.

• Mentor and coach team members to achieve engineering excellence.

• Collaborate with architects to design robust, scalable, and cost-effective data solutions.

Key Requirements:

• 6 years in a Data Engineer role, with a degree in Computer Science, Statistics, Informatics, Information Systems, or a related field.

• Strong software engineering capabilities, including proficiency in writing unit tests (Python) and using Git.

• Experience with Databricks and knowledge of recent features of the tool.

• Proficiency in Microsoft Azure analytics stack and real-time analytics tools.

• Advanced SQL skills and familiarity with various databases.

• Understanding of data modeling techniques for analytics.

• Ability to perform root cause analysis on data and processes.

• Experience with data security operations, particularly in Azure.

• Capability to troubleshoot and solve complex data-related issues.

• Awareness of data management frameworks (e.g., DAMA) and experience with data quality tools like Collibra or Alation.

• Experience in API development and strategies for real-time data delivery.

• Good verbal and written communication skills in English.