Data Engineer

Introduction & Summary:

We are seeking an experienced Data Engineer to design and implement data pipelines that efficiently ingest, transform, and serve data for various data products. The ideal candidate must possess strong proficiency in Databricks with at least 3 years of experience, alongside a robust understanding of data modeling, quality assurance, and optimization techniques to enhance data workflows.

Main Responsibilities:

• Design and implement data pipelines for data products.

• Build and maintain data models that support analytics and reporting.

• Ensure data quality through validation, testing, and monitoring.

• Manage and optimize data workflows for performance and cost efficiency.

• Collaborate with cross-functional teams to convert business needs into technical solutions.

• Contribute to CI/CD processes for version control and automated testing.

• Maintain documentation of data pipelines and business logic.

• Monitor production pipelines and troubleshoot issues for stable operation.

• Implement governance, security, and access control standards.

• Continuously improve existing data pipelines based on feedback.

• Stay updated on the latest tools and best practices in data engineering.

Key Requirements:

• Proven experience with Databricks (minimum 3 years).

• Strong knowledge of data modeling techniques.

• Experience in data quality assurance and monitoring.

• Ability to manage and optimize data workflows.

• Excellent collaboration skills with cross-functional teams.

• Familiarity with CI/CD processes and automated testing.

• Proficiency in documenting technical processes and business logic.

• Experience in implementing data governance and security standards.

Nice to Have:

• Knowledge of additional data pipeline tools and frameworks.

• Experience in data visualization tools.