DataOps Engineer

Responsibilities:

• Design, develop, and implement data pipelines using Apache Airflow and other data orchestration tools.
• Monitor and optimize ELT data pipelines to ensure performance, reliability, and scalability.
• Troubleshoot and resolve issues related to data pipelines, ensuring data integrity and quality.
• Utilize dbt for data transformation, optimise its usage and operations, define standards and best practices.
• Implement and maintain data governance practices, including data lineage and metadata management.
• Work closely with the DevOps team to ensure seamless integration and deployment of data pipelines using GitLab.
• Stay up-to-date with industry trends and best practices in data engineering and data orchestration.

Requirements:

• Proven experience as a DataOps Engineer or similar role.
• Strong proficiency in data orchestration tools, particularly Apache Airflow.
• Solid understanding of data engineering concepts and best practices.
• Proficiency in Git, Python and SQL.
• Experience with data transformation tools (especially dbt).
• Experience with AWS cloud platform (i.a. Redshift, MWAA, Glue, Lambda, S3).
• Familiarity with data quality and data observability tools.
• Familiarity with DevOps tools (e.g. GitLab) and concepts (i.a. CI/CD, containerization).
• Strong problem-solving and troubleshooting skills.
• Excellent communication and collaboration abilities.