MLOps / Data Engineer – Databricks & AI Pipelines

Key Tasks & Responsibilities

• Design & Develop: Build scalable data platforms on Azure Cloud using Azure Data Factory, Databricks, ADLS, and other Azure services.

• Operationalize AI Solutions: Support deployment and maintenance of Data Science solutions; manage the full lifecycle of AI products with a focus on industrialization and operations.

• Workflow Orchestration: Orchestrate data workflows and machine learning pipelines.

• Monitoring & Resilience: Implement monitoring and backup strategies to ensure solution robustness; respond to failures and prevent recurrence (e.g., Azure Monitor, Log Analytics).

• Data Management: Experience integrating data from multiple sources and managing access with Unity Catalog; familiarity with Key Vault and Purview is a plus.

• CI/CD & Code Management: Build and maintain production-grade pipelines in Spark/PySpark; ensure version control via Git and apply best practices for CI/CD automation.

• Experimentation & Reproducibility: Improve reproducibility and scalability of ML workflows; build frameworks for experimentation efficiency.

• Optimization: Optimize ML solutions for performance and cost efficiency in cloud environments.

• Collaboration: Work closely with data scientists and analysts to deliver efficient data and AI solutions.

• Solution Architecture: Actively contribute to architectural discussions by proposing improvements and identifying technical gaps.

• Continuous Improvement: Stay current with Azure and Databricks technologies to continuously improve workflows and automation.

Qualifications & Competencies

• Education: Bachelor’s degree in Computer Science, Mathematics, Statistics, Engineering or related fields. Master’s is a plus.

• Experience: 3–4+ years of hands-on experience working with cloud-based data platforms and advanced analytics/AI products.

  Programming Skills: Proficiency in Python

  Big Data & ML Pipelines:

• Strong experience with Spark & PySpark

• Experience building ETL/ELT and ML pipelines

  Azure Cloud Expertise:

• Experience with Azure Data Factory, ADLS, SQL services

• Familiarity with Azure Apps, Containers, Storage

• Experience with security and governance (Purview, Unity Catalog, Key Vault)

  DevOps & CI/CD:

• Hands-on experience with CI/CD pipelines, Git-based workflows, and deployment automation

• Experience with Docker

• Tools like SonarQube, flake8 (nice to have)

  Soft Skills:

• Strong problem-solving and communication skills

• Proven ability to collaborate across technical teams

Nice to Have

• Experience with generative AI, forecasting, optimization, or simulation

• Knowledge of SQL/NoSQL, data modeling, and warehousing principles

We offer:

• B2B contract (up to 250 PLN/h net + VAT)

• 100% remote work

• Wide range of projects (internal and international)

• Dedicated certification budget

• Annual evaluation meetings to define an individual development path

• Benefits package

• Integration trips