Job Description:
We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will be a Snowflake expert with extensive experience in data pipeline programming, cloud technologies, and DevOps practices. The role involves working on cutting-edge data solutions that provide actionable insights and support data exploration with Business Intelligence and Self-service tools.
Key Responsibilities:
- Utilize Snowflake for efficient data storage and retrieval.
- Implement and manage ETL/ELT processes, ensuring data quality and real-time streaming capabilities.
- Collaborate with cross-functional teams to support data architecture and data modeling projects.
- Database Design: SQL developers design the structure of databases, including creating tables, defining relationships, and optimizing data storage for efficiency.
- Analyzing data using SQL queries, filtering and transforming data.
- Creating data-driven reports and dashboards.
- Creating visualizations using tools like Tableau, Power BI, or Python libraries (e.g., Matplotlib, Seaborn).
- Analyzing trends, patterns, and anomalies and providing actionable insights to the business.
- Design, develop, and maintain data pipelines using Python or R.
- Apply DevOps practices using Git, GitLab, Docker, Jenkins, and Terraform.
- Ensure seamless integration and maintenance of various data storage solutions (filesystem, relational, MPP, NoSQL).
- Perform performance analysis, troubleshooting, and remediation in Unix/Linux environments.
- Provide technical onboarding and support to platform users, offering consultancy sessions and resolving support requests.
Required Skills:
- Solid experience with Snowflake (not a beginner).
- Proven experience as a Data Engineer, working with Snowflake, ServiceNow or Talend.
- Proficiency in SQL and experience with Snowflake's SQL syntax and capabilities.
- Experience in Database development, good knowledge of relational databases
- Good command of data storage solutions (Snowflake, Oracle, PostgreSQL, MySQL, etc,)
- Experience with data modeling, data warehousing, and ETL processes.
- Familiarity with data integration tools (e.g., Talend, Informatica, ServiceNow) and cloud platforms (e.g., AWS, Azure, GCP).
- Experience using a software development life cycle (SDLC) methodology
- Proficiency in scripting languages such as Python,Java or R
- experience in data pipelines maintenance and data architecture concepts. various kinds of data (structured, unstructured, metrics, logs, etc.)
- knowledge of DevOps practices and tools (Git, GitLab, Docker, Jenkins, Terraform).
- Proficiency in Unix/Linux environments.
- Familiarity with data serialization languages such as JSON, XML, YAML.
Preferred Skills:
- Training in data mesh/data products concepts.
- Experience with Talend and other related platforms.
- Knowledge of pharma data formats (SDTM).
- Ability to develop Metrics and Visualizations (tools like Tableau, Power BI, or Python libraries like Matplotlib and Seaborn)
- Experience with observability/authorization tools like Monte Carlo/Immuta.
- Familiarity with ServiceNow, Google Workspace, and other relevant IT tools is highly preferred.
- Experience with cloud technologies emphasizing data pipelines (Airflow, DBT, Glue, Dataflow).