Senior Python Engineer

Summary

The Senior Python Engineer role is focused on building a modern, scalable Data Platform using Microsoft Azure and Fabric. The primary objective is to design reusable libraries, define standards, lead a team, and ensure long-term support for a high-impact data platform.

Main Responsibilities:

• Design and build reusable Python + SQL libraries/frameworks for data ingestion and transformation.

• Strong hands-on experience with Microsoft Azure cloud services.

• Architect event-driven ingestion pipelines using Kafka and Azure Functions.

• Implement robust data transformation and loading frameworks (batch and near real-time).

• Apply strong data warehousing concepts (fact/dimension modeling, SCDs, incremental loads, partitioning, indexing).

• Develop scalable data loading strategies into warehouse/lake environments.

• Build and manage APIs for data exposure and integration.

• Lead CI/CD implementation using GitHub (Actions), including automated testing, versioning, and deployments.

• Ensure production readiness: monitoring, logging, alerting, and operational support.

• Guide architecture decisions focused on performance, scalability, and cost optimization on Azure.

• Mentor team members and lead code reviews and best practices adoption.

Key Requirements:

• 10+ years of experience in Python.

• Strong hands-on expertise in Python (advanced) and SQL (advanced).

• Understanding of Data Warehousing concepts.

• Experience with Kafka in production environments.

• Hands-on experience with Azure services (Azure Functions, ADLS, Storage, App Services, etc.).

• Good understanding of Azure architecture and cost analysis/optimization.

• Experience building and deploying APIs and applications in Azure.

• Strong experience with GitHub and CI/CD pipelines.

• Experience leading small teams or owning modules end-to-end.

Nice to Have:

• Experience with DuckDB (preferred but NOT mandatory).

• Experience with modern lakehouse technologies.

• Containerization, Infrastructure as Code (Terraform/Bicep).

• Observability tools (App Insights, monitoring frameworks).

• Exposure to AI/ML data preparation pipelines.

Other Details:

• Location: Pune Hybrid model

• Team Structure: Collaborative work with Data Engineering and AI teams.

• Reporting Line: Reports to the Head of Data Engineering.

• Project Context: Focus on building a modern enterprise data platform.