Senior Data Engineer

We are a leading European AI company developing large language models and generative platforms for enterprise and government clients.Our products combine high-performance technologies, data security, and transparency, fully aligned with European regulatory and ethical standards.

As a Data Engineer, you will design, build, and maintain high-performance backend services and data pipelines, enabling our teams to deliver scalable, reliable, and production-ready systems.

Responsibilities:

1. ETL Data Pipelines (Batch & Streaming)

• Develop and operate ETL pipelines to extract, transform, and load data from multiple sources

• Support both batch workloads (large-scale periodic data processing) and streaming workloads (real-time or near-real-time data flows)

• Optimize performance, scalability, and reliability of data processing pipelines

• Collaborate with data engineers and analysts to ensure high-quality, clean, and accessible datasets

1. ML Data Pipelines using Temporal

• Design, implement, and maintain robust ML data pipelines for training, validation, and inference of machine learning models

• Use Temporal or similar tools for workflow orchestration, ensuring reliability, retries, and state management across complex ML workflows

• Collaborate closely with ML engineers and researchers to automate and scale model pipelines

• Ensure pipelines are reproducible, maintainable, and observable

1. Backend Services & Infrastructure

• Design, build, and maintain backend services in Go or Python 

• Work with PostgreSQL and object storage (S3) to store and manage structured and unstructured data

• Deploy and manage services using Kubernetes (K8s) and Helm

• Implement best practices in CI/CD using GitHub Actions

• Apply system design and data modeling principles, handling concurrency and performance optimization

Requirements:

• 3+ years of commercial experience in Go or/and proficient in Python (most of which in a production setup with real customers) 

• Strong knowledge of ETL pipeline development (batch and streaming workloads)

• Experience with Temporal or other asynchronous workflow orchestration tools

• Experience with PostgreSQL and object storage (S3)

• Familiarity with Kubernetes (K8s) and Helm

• Understanding of concurrency patterns and performance optimization in Go

• Experience building and operating ML data pipelines is highly desirable

• Strong collaboration skills and attention to detail

Nice to Have:

• Experience designing APIs/SDKs

• Experience with complex migrations or data model changes

• Knowledge of TDD, DDD, or other development best practices

• Familiarity with resiliency patterns (retries, circuit breakers)

• Experience integrating backend systems with ML models

• Experience with OpenFGA or similar tools

• Familiarity with stakeholder management and brief, concise communication

Technology Stack:

• Core Backend: Python (FastAPI), Go

• Data Storage: PostgreSQL, S3 / object storage

• Workflow Orchestration: Temporal (for ML pipelines)

• ETL: Batch and streaming pipelines

• DevOps & Infrastructure: Kubernetes (K8s), Helm, GitHub Actions

• Internal Tools: TypeScript / Nx for CLI automation (if applicable)