Data Engineer

For our client, a leading US financial services company, we are ooking for a Senior Data Engineer / Datalake Expert.

The Data Team is responsible for designing, building, deploying, and maintaining a web-scale, cloud-native data platform.

The company works in tribes and squads that focus on specific products and projects – and depending on their strengths and interests, team members have the opportunity to move between them, joining an agile team that fosters collaboration and innovation. 

Location: 100% remote work possible; however, candidates who can work hybrid from the Warsaw office are preferred.

Responsibilities

• Executes standard software solutions, design, development, and technical troubleshooting
• Writes secure and high-quality code using the syntax of at least one programming language with limited guidance 
• Designs, develops, codes, and troubleshoots with consideration of upstream and downstream systems and technical implications 
• Applies knowledge of tools within the Software Development Life Cycle toolchain to improve the value realized by automation 
• Applies technical troubleshooting to break down solutions and solve technical problems of basic complexity  
• Gathers, analyzes, and draws conclusions from large, diverse data sets to identify problems and contribute to decision-making in service of secure, stable application development 
• Learns and applies system processes, methodologies, and skills for the development of secure, stable code and systems 
• Adds to team culture of diversity, equity, inclusion, and respect 

Requirements

• At least 5+ years of recent hands-on professional experience (actively coding) working as a data engineer (back-end software engineer considered) 
• Understanding of distributed systems and cloud technologies (perfectly AWS)
• Understanding of data streaming and scalable data processing frameworks (Kafka, Spark Structured Streaming, Flink, Beam etc.)
• Experience with SQL (any dialect) and Data tools
• Experience in the all stages of software development lifecycle (requirements, design, architecture, development, testing, deployment, release and support) 
• Experience with large scale datasets , data lake and data warehouse technologies on at least TB scale (ideally PB scale of datasets) with at least one of {BigQuery, Redshift, Snowflake} 
• Experience in Infrastructure as Code (ideally Terraform) for Cloud based data infrastructure
• Excellent knowledge of Python  
• Excellent written and verbal communication skills in English 

Preferred qualifications, capabilities and skills  

• Experience with a scheduling system (Airflow, Azkaban, etc.) 
• Understanding of (distributed and non-distributed) data structures, caching concepts, CAP theorem 
• Understanding of security frameworks / standards and privacy 
• Desired – experience in automating deployment, releases and testing in continuous integration, continuous delivery pipelines 
• A solid approach to writing unit level tests using mocking frameworks, as well as automating component, integration and end-to-end tests 
• Experience with containers and container-based deployment environment (Docker, Kubernetes, etc.) 
• Good experience with using a JVM language (Java/Scala/Kotlin, preferably Java 8+)

We offer

• Financial transparency: the salary is: 28 560 - 35 280 PLN (170 - 210 PLN/h) + VAT​
• Clear wage model. 7N Consultants know exactly what’s 7N fee for its agent role on the project.   
• Professional development support. We subsidize trainings, technical certificates, conference participation, and foreign language classes. Our Career Development Program offers consultants advisory support in career planning. Additionally, 7N Consultants have the opportunity to participate in interpersonal skills training through the 7N Secret Code. Learn more about it here: 7N: 7N – The Secret Code   
• 7N Inspiration Team’s support and an opportunity to join their ranks. They are a group of 7N Consultants who consistently and actively share their professional knowledge and expertise.  
• Cooperation with real experts – the average professional experience of a 7N consultant is 10 years. Check out one of our discussion panels here: Does a Scrum Master need a background in programming (in Polish) 
• Comprehensive and personal project support from one of a 7N Agent. Priority regarding project continuity and quality. See what one 7N Consultant said about the role of 7N Agents and project opportunities.  
• Numerous (10-12 each year) high-end events – both online and offline. See the 7N KickOff 2024 clip here: https://www.youtube.com/watch?v=8MjSywOKZnc
• Healthcare, Benefit Multisport and life insurance subsidies regardless of cooperation model. Access to Mindgram, a psychological support and professional development platform.  
• Professional and quality-oriented recruitment process led by the most experienced recruiters in the industry. Read more about it in the 7N candidate recruitment satisfaction survey (in Polish).