Senior Data Engineer with Python

Senior Data Engineer (Azure / Databricks / Python)

Location: Remote

Type: Full-time

Level: Senior

Duration: 4 months

Start: ASAP

About the role

We are looking for a Senior Data Engineer to design and implement scalable data pipelines supporting advanced, customizable data processing workflows.

You will be responsible for building end-to-end solutions that ingest data from streaming sources, process it in distributed analytics environments, and expose curated datasets through well-defined interfaces. The role also involves integrating external machine learning workloads into an existing cloud-native architecture.

This position is well-suited for engineers who enjoy working at the intersection of data engineering, cloud platforms, and applied machine learning.

Responsibilities

• Design and implement scalable ETL/ELT data pipelines for streaming and batch workloads

• Ingest data from cloud-based event streaming platforms into distributed processing environments

• Build and maintain data processing workflows using Azure-native services

• Expose processed and curated data according to architectural specifications

• Integrate external machine learning or analytical workloads into existing data pipelines

• Ensure reliability, performance, and scalability of data processing solutions

Core Skills

• Extensive hands-on experience with Azure Databricks

• Strong experience with Azure Event Hubs and Azure Functions

• Proven expertise in designing and operating robust, scalable ETL/ELT pipelines

• Strong Python programming skills for data processing and orchestration

• Hands-on experience integrating LLM APIs into production systems

Highly Desirable

• Experience integrating machine learning or custom analytics models into production data pipelines

• Familiarity with streaming and time-series data processing

• Knowledge of data formats commonly used in large-scale monitoring or analytics systems

• Experience designing data solutions for high-volume, high-velocity data

What we offer

• Work on complex, large-scale data processing systems

• Real impact on architecture and data platform design

• Collaboration with experienced engineers and architects

• Competitive compensation