Lead Data Engineer - AWS/Snowflake (Big Data & Clinical)

The Non-CRF Data Provisioning initiative focuses on ingesting non-CRF data into the organization and performing conformance checks against predefined standards. The scope of data includes not only laboratory data but also non-traditional types such as images, video, and other digital biomarkers. The product supports data collection during clinical study conduct, data cleaning, transformation into standard formats, and integration with other clinical datasets, ensuring readiness for analysis.

Your responsibilities

• Development Oversight: Lead, coach, and mentor the development/technical team (primarily external resources), ensuring best practices and coding standards are consistently applied. Coordinate technical resources and communicate relevant changes in infrastructure, systems, or processes.

• Technical Solution Design & Review: Design solutions or review designs proposed by the external team, ensuring scalability and maintainability.

• Technical Support: Provide guidance and support for technical issues, particularly during study setup, troubleshooting data issues, and working with distributed teams across multiple time zones.

• Code Review & Quality Assurance: Conduct regular code reviews, enforce coding standards, and ensure compliance with GxP requirements. Maintain high-quality technical documentation and user stories in Jira.

• System & Domain Knowledge: Understand the current system landscape, contribute to the future technical roadmap, and collaborate with Product and Solution Architects to define platform evolution.

• Collaboration: Partner with Product, Project, and Validation teams to ensure fit-for-purpose testing, validation, and continuous improvement of technical processes. May also take on System Owner responsibilities if required.

Our requirements

• Strong expertise in digital device data, big data ingestion, processing, and storage.

• Ability to analyze complex data and interpret technical information effectively.

• Proficiency in: AWS Python Snowflake Airflow Amazon EMR Spark (e.g., PySpark)

• Familiarity with: Tableau Terraform CI/CD principles and tools

• Experience with validated projects and clinical trial data (preferably DHT data).

• Strong communication, leadership, and mentoring skills.

• Ability to work with distributed, cross-functional teams and navigate complex organizational environments.