Data Engineer

We are seeking 1-2 Senior and self-motivated Data Engineers to assist in the migration of finance and HR data domains. This role offers the flexibility to work remotely; however, visits to the office in Sweden/Stockholm are encouraged for collaboration.

The Assignment:

Currently, we have a series of ETL jobs managing financial data utilizing the ETL tool Matillion and Snowflake as the database. The data resides in the Azure data lake (backed by Dynamics). The project aims to understand the existing system and migrate to a combination of Airflow, dbt, and Snowflake jobs while adopting a data mesh/data products strategy. This migration will not be a direct mirror, but rather an enhancement of the existing setup. For instance, the current implementation uses Row Level Security (RLS) to control access to sensitive data; the new structure may implement role-level access control instead.

Some of the tasks would include:

• Understand and document current pipelines written in Matillion.

• Write ingestion pipelines using Airflow DAG.

• Simplify and convert mappings into dbt models (some tasks may also run in Airflow if they include Python snippets).

• Create dbt documentation for all models.

• Develop unit tests in dbt for data validation.

• Support the reconciliation process between current and new report outputs from the migrated system.

The most intricate aspect of this role will be to understand and document the existing implementation, as there is currently no available documentation; manual examination of the pipelines will be necessary.

Start: Asap
Workplace: Sweden or remote from Sweden.
Length: 3-6 months