Data Engineer (Scala & Python Focus)

7N is an agent for high-end IT professionals. For over 30 years of operation we have proven that a clear and transparent financial model, collaboration exclusively with experts in their respective fields, and taking good care of them, comprise the best possible IT consulting model. We act as an individual agent for our consultants and promote their competences to our clients by offering them a wide range of projects in which they may participate. We add wage transparency, career development support and professional stability. Our main goal is a long-term collaboration; therefore the majority of our staff have been with us for many years. 

About the project

We are looking for two experienced Data Engineers to join a team in a global company form Life Science industry. The team is specializing in building a Data Lake and will be migrating the solution to Python. This is an exciting new project where the focus will be on implementation, not maintenance.

The role offers the possibility of 100% remote work.

Tasks

• Design and develop ETL solutions to process large amounts of medical data efficiently
• Create and implement automated tests to ensure high data quality
• Continuously improve the data platform, reducing technical debt
• Support the introduction of new tools and lead the migration to Azure Cloud
• Provide guidance and support to less experienced team members
• Contribute to the definition of standards and promote best practices across the team
• Work closely with cross-functional teams to ensure that the data platform meets business needs
• Collaborate with product teams to understand data requirements and ensure data quality and integrity
• Troubleshoot and resolve issues related to data processing and access

Requirements

• 6+ years of experience in data engineering, with a focus on ETL design and implementation
• 3+ years of experience with batch processing
• 2+ years of experience in data analysis, profiling, integration, and reporting
• Strong expertise in Scala and SQL, with at least basic knowledge of Python (4+ years)
• Solid experience with Apache Spark, Kafka, Flink, and HDFS
• Extensive knowledge in data engineering principles, databases, and data modeling
• Experience with Azure cloud and Databricks is a plus
• Excellent problem-solving skills and attention to detail
• A collaborative mindset and experience working in a team-oriented environment
• Proficiency in English at the C1 level

Nice to have:

• Extensive knowledge of database design and modeling
• Commercial experience with Databricks and Azure cloud
• Mastery of Scala and Python, with the ability to develop high-performance data pipelines
• Strong analytical skills and troubleshooting abilities

We offer

• Financial transparency: the salary is: 26 880 – 30 240 PLN (160 – 180 PLN/h) + VAT
• Clear wage model. 7N Consultants know exactly what’s 7N fee for its agent role on the project.   
• Professional development support. We subsidize trainings, technical certificates, conference participation, and foreign language classes. Our Career Development Program offers consultants advisory support in career planning. Additionally, 7N Consultants have the opportunity to participate in interpersonal skills training through the 7N Secret Code. Learn more about it here: 7N: 7N – The Secret Code   
• 7N Inspiration Team’s support and an opportunity to join their ranks. They are a group of 7N Consultants who consistently and actively share their professional knowledge and expertise.  
• Cooperation with real experts – the average professional experience of a 7N consultant is 10 years. Check out one of our discussion panels here: Does a Scrum Master need a background in programming (in Polish) 
• Comprehensive and personal project support from one of a 7N Agent. Priority regarding project continuity and quality. See what one 7N Consultant said about the role of 7N Agents and project opportunities.  
• Numerous (10-12 each year) high-end events – both online and offline. See the 7N KickOff 2024 clip here: https://www.youtube.com/watch?v=8MjSywOKZnc
• Healthcare, Benefit Multisport and life insurance subsidies regardless of cooperation model. Access to Mindgram, a psychological support and professional development platform.  
• Professional and quality-oriented recruitment process led by the most experienced recruiters in the industry. Read more about it in the 7N candidate recruitment satisfaction survey (in Polish).