Design and implement scalable data processing infrastructure and ETL pipelines on Azure, ensuring efficient and reliable data flow to support AI/ML models, analytics, reporting, and other data-driven applications.
Establish and enforce robust data governance practices, ensuring the security, integrity, and availability of data across the platform.
Continuously enhance your data engineering expertise, staying ahead of industry trends, and proactively sharing insights and best practices with the team to foster a culture of learning and innovation.
What kind of experience should you have?
Strong hands-on experience with Azure cloud services, including Spark/Databricks, and a deep understanding of cloud-based data architectures.
Minimum 5 years of experience as a Data Engineer
Proficiency in Python and SQL, with a proven track record of building and optimizing complex ETL/ELT processes in Data Lakehouse environments.
Experience with data mesh architecture is highly desirable, demonstrating your ability to work within decentralized data environments.
A passion for continuous learning, coupled with the ability to communicate complex technical concepts to non-technical stakeholders.