DevOps Engineer (Expert)

We foster a dynamic culture rooted in strong engineering, a sense of ownership, and transparency, empowering our team. As part of the expanding VirtusLab Group, we offer a compelling environment for those seeking to make a substantial impact in the software industry within a forward-thinking organization.

About the role

GCP: Advanced

CI/CD: Advanced

Kubernetes (GKE): Advanced

English: Advanced

Communication skills: Advanced

Beam/Spark/Kafka/Airflow/HBase/Presto: Basic

Graph and Vector databases: Nice to have

Project

Apollo Data Platform

Project Scope

Our client is a NASDAQ-listed company that provides a range of solutions to support Go-To-Market (GTM) strategies. They offer a comprehensive B2B database platform that enables sales and marketing professionals to identify, connect with, and engage qualified prospects effectively. The core mission of our client is to equip every company with a complete, 360-degree view of their ideal customer, enhancing every phase of their GTM strategy and boosting their success in achieving business targets.As a Data Platform Engineer, you’ll lead a small engineering team in an R&D setting and play a crucial role in designing, implementing, and deploying elements of a Data Platform for our customer. We seek an experienced software engineer skilled in developing large-scale data pipelines/systems with strong problem-solving abilities. You’ll lead innovation, swiftly test and validate new concepts, and integrate them ensuring team collaboration.

Tech Stack

Java, Beam, Spark, Kafka, Airflow, Presto, AWS and GCP (Dataflow, Dataproc/EMR, Cloud Function/Lambda, BigQuery, BigTable), Python

Challenges

Design, build, and operate highly scalable and flexible systems that can manage and process billions of records a day and support complex and diversified data pipelinesLeverage cloud computing architectures to support development needs*Track and identify relevant new technologies in the market and push their implementation into our pipelines through research and prototyping*Develop processes and tools to monitor, analyze, maintain, and improve data operation, performance, and usability.

Team

Currently, there are 7 engineers from VL working there in 2 subteams. The team is primarily based in the US and Israel.

What we expect in general

• DevOps Engineering experience in data platform/big data software, with a proven track record of delivering highly scalable and efficient solutions

• Familiarity with streaming/data processing technologies such as Beam, Spark, Kafka, Airflow, HBase, Presto

• Experience in building enterprise-grade software in a cloud-native environment (GCP or AWS) using cloud services

• Experience in system architecture and design

• Experience in designing CI/CD pipelines with Jenkins, Github Actions, or similar tools

• Experience with Kubernetes using GKE/EKS

• Experience with SQL (nice to have)

• Experience with Graph and Vector databases or processing frameworks (nice to have)

• Excellent communication skills and a pragmatic approach to problem-solving

• English at B2+ level

A few perks of being with us

• Building tech community

• Flexible hybrid work model

• Home office reimbursement

• Language lessons

• MyBenefit points

• Private healthcare

• Training Package

• Virtusity / in-house training

And a lot more!