2x Senior Data Engineer – Databricks & AWS RFP Managed Service

Work model: fully remote, full-time 40 hrs/week

Assignment type: B2B (own legal entity required)

Project length: 4 months (+ potential extensions).

Project language: English.

Start Date: January 2026

Our client is a Swedish technology company that provides self-serve advertising platforms for publishers. Their solutions allow advertisers of all sizes to buy, manage, and optimise digital ad campaigns directly, without intermediaries. By automating the ad sales process, the company helps publishers increase efficiency and revenue, while offering advertisers greater transparency and control. The company works with global clients such as Disney, TripAdvisor, and Bloomberg.

The company is moving its environment from BigQuery to AWS to simplify operations, meet compliance requirements (SOC 1), and build a scalable, future-proof data platform.

Key elements of the project:

• Building a Lakehouse on Databricks (AWS) – a central platform combining data collection, transformation, management, and consumption.

• Data migration from MySQL RDS (~50 instances) to S3 → schema harmonisation in the Silver layer (Global Data Schema).

• Automation and simplicity – a metadata-driven framework (configuration instead of coding many pipelines).

• Compliance (SOC 1) – full auditability, RBAC, change control, and regulatory compliance.

• Analytics – integration with AWS QuickSight and optionally Power BI.

• Preparation for AI/ML – architecture ready for integration with machine learning and generative solutions in later phases.

Required Skills & Experience

• Proven hands-on experience in Databricks (PySpark, Workflows, Delta Lake).

• Strong background in SQL.

• Experience with AWS services: S3, IAM, CloudWatch.

• Experience with DynamoDB or other NoSQL databases.

• Infrastructure as Code (Terraform or similar).

• Strong programming skills in Python and familiarity with PySpark best practices.

• CI/CD for data engineering (GitHub Actions, GitLab, or AWS CodePipeline).

• Understanding of Unity Catalog for governance, RBAC, lineage, and audit.

Optional / Nice-to-Have

• AWS DMS (Data Migration Service) for Change Data Capture and migration from MySQL RDS to S3.

• Experience with AWS Lambda and SQS for event-driven architectures.

• Exposure to data quality frameworks (e.g., Delta Live Tables expectations).

• Familiarity with BI tools such as Power BI or QuickSight for validation/testing.

• Experience building metadata-driven ingestion frameworks.

Other details:

This position is remote-friendly, BYOD and involves collaboration with a talented team.