Senior Data Engineer

Job description

Solar Spy is an analytical platform dedicated to support solar farm operations. It processes various types of PV plant data, including image data from aerial inspections, weather data, data from photovoltaic installation infrastructure elements, and more. Through the use of artificial intelligence models, we identify problems, help to optimize maintenance costs, and predict energy production. Our main objective is to maximize green energy production and reduce O&M costs.

As we work with huge amounts of mostly unstructured data, we are looking for a Data Engineer who can help us design and implement a scalable data architecture - capable of processing the data with high throughput in a distributed manner.

What we expect:

🔹 Experience of working with big data, data engineering tools for data ingestion, transformations, querying

🔹 Strong understanding of data structures and algorithms

🔹 Strong understanding of object design and integration patterns

🔹 Proven experience of using python for building data processing platforms/applications

🔹 Experience in building distributed data processing pipelines, multiprocessing applications

🔹 Proficient in NoSql and relational databases, SQL, ORMs

🔹 Experience with messaging systems (e.g.: KafkaMQ)

🔹 Experience in building scalable APIs

🔹 Nice-to-have: experience with one of the leading cloud platforms (Azure, AWS, GCP)

Your responsibilities:

🔸 Performance Optimization: Diagnose and resolve performance issues, ensuring optimal functionality and efficiency

🔸 Build and maintain distributed/multiprocessing data processing pipelines

🔸 Build new features, test, measure the impact, improve based on learnings

🔸 Build solutions at scale and optimize for performance

🔸 Work together with data scientists, ML engineers to serve AI/ML models with high throughput

🔸 Proactively make suggestions on how the team can leverage new technologies and architectures to improve the performance of our data platform

What we offer:

🔹 Stable employment – perm or B2B contract – completely up to you

🔹 Work in a non-corporate environment where you can have an impact on the way your business operates

🔹 Flexible working hours, hybrid work model (the team meets onsite once a week)

🔹 Private healthcare, sports card, training budget