Data Engineer - Data Bricks

We are looking for a Senior Data Engineer with expertise in DataBricks and Data Engineering to join our team, working with one of the leaders in the automotive industry. The ideal candidate will have a deep understanding of DataBricks, Delta Lake, PySpark, and related technologies.

Job Title: Senior Data Engineer - Data Bricks

Location: Wipro Warsaw Office (Hybrid/2-3 days a week)

Experience: 7+ years

Rate: 1.000-1.200 PLN/day

Key Responsibilities:

• Design, develop, and manage data pipelines in the DataBricks Lakehouse Platform
• Work on Delta Lake, Delta Live Tables, and LakeFlow for batch and streaming data processing
• Utilize PySpark to orchestrate diverse workloads across the full lifecycle, including Delta Live Tables, data transformation, and more
• Leverage Unity Catalog and Delta Sharing for data sharing and catalog management
• Monitor and visualize performance, data quality, and reliability metrics using Delta Lake tables
• Collaborate on workflows and dashboards to track production job performance and data pipelines
• Strong experience in data modelling and BI skills

Required Skills:

• 8+ years of experience in Data Engineering
• Extensive experience with DataBricks, Delta Lake, PySpark, LakeFlow, Delta Live Tables, Unity Catalog, and Delta Sharing
• Strong understanding of DataBricks Lakehouse Solutions
• Proficient in data modeling and business intelligence
• Ability to work with both batch and streaming data on the DataBricks platform

Good to Have:

• Experience with Azure Data Factory or Fivetran
• Familiarity with PowerBI for reporting and data visualization

Notes:

• The ideal candidate will have a data engineering background rather than a data scientist profile.
• In-depth experience with DataBricks tools (not third-party tools) such as PySpark, LakeFlow, Delta Live Tables, Unity Catalog, and Delta Sharing is crucial.
• Familiarity with PowerBI is a plus, but it is not mandatory.
• Strong data modeling and BI skills are essential.

If you are a passionate Data Engineer with extensive experience in DataBricks and Data Engineering, we would love to hear from you!