Lead Data Engineer with Google Cloud (GCP)

Work model: remote

Assignment type: B2B

Start: ASAP

Project length: 6 months + possible extension

Project language: English

Industry: Banking

The project is focused on decommissioning a legacy CRM system and successfully migrating to a new platform by the end of the year. The main function of the Lead Data Engineer is to support the development of robust data pipelines and ensure smooth data migration and integration.

Main Responsibilities

• Be a self-starter and proactive, experienced Technical Lead.

• Exhibit excellent communication, interpersonal, and decision-making skills.

• Participate and influence in regular planning and status meetings.

• Provide input to the development process through involvement in Sprint reviews and retrospectives.

• Contribute to system architecture and design.

• Utilize good experience with SQL and NoSQL databases.

• Engage in software design, Python & GCP development, and automated testing of new and existing components in an Agile, DevOps, and dynamic environment.

• Promote development standards, conduct code reviews, and facilitate knowledge sharing.

• Build data pipelines using technologies such as BigQuery, DataProc, Python, Composer, and Spark.

• Demonstrate knowledge and experience of the GCP ecosystem and data management frameworks.

• Have CI/CD experience.

• Possess knowledge of Software Development Life Cycle (SDLC) and methodologies like DevOps, Agile, Scrum, Waterfall, and Iterative process.

• Show good knowledge of Data Warehouse concepts.

• Implement tools and processes while handling performance, scale, availability, accuracy, and monitoring.

Key Requirements

• Experience in a Technical Lead position.

• Strong background in SQL and NoSQL databases.

• Proficiency in Python and GCP development.

• Experience with CI/CD processes.

• Solid understanding of Agile and DevOps methodologies.

• Good knowledge of Data Warehouse concepts.

Nice to Have

• Knowledge and experience of Hadoop.

• Familiarity with PySpark, Scala, Hive, and Java.