Data Engineer/ Tech Lead (Streaming Focus)

About the Role

We are seeking a Data Engineer / Tech Lead with a strong background in real-time data streaming to lead the design and delivery of cutting-edge streaming solutions. This role combines hands-on engineering with technical leadership – you will both architect scalable data streaming systems and guide a team to successful implementation.

Key Responsibilities

• Act as technical lead for projects with a strong streaming/data engineering focus.

• Design end-to-end architectures for data streaming use cases, ensuring scalability, reliability, and performance.

• Implement Change Data Capture (CDC) and other real-time extraction mechanisms.

• Drive adoption and best practices in Confluent Kafka and Spark Structured Streaming.

• Lead the development of streaming pipelines using Python, SQL, Bash, Node.js, and Linux/Unix Shell.

• Oversee code quality, versioning, release management, and CI/CD processes.

• Guide the team in applying DevOps and cloud-native tools: GitHub Actions, Terraform, Kubernetes, etc.

• Collaborate with business stakeholders, data scientists, and platform teams to align streaming solutions with organizational goals.

• Mentor junior/mid-level engineers, fostering growth and technical excellence within the team.

Essential Skills & Experience

• Proven experience in data engineering with a strong focus on streaming.

• Strong expertise in Confluent Kafka and Spark Structured Streaming.

• Hands-on experience with CDC and real-time data extraction.

• Solid programming background: Python, SQL, Bash, Node.js, Linux/Unix Shell.

• Strong knowledge of data engineering lifecycle: versioning, release management, deployment, agile delivery.

• Experience with cloud-native & DevOps tools: GitHub Actions, Terraform, Kubernetes.

• Previous experience in a technical leadership role (designing solutions, guiding teams, mentoring).

Nice to Have

• Multi-cloud exposure (AWS, Azure, GCP).

• Knowledge of wider big data & analytics platforms (Databricks, Snowflake, Delta Lake).

• Experience in stakeholder management and solution consulting.