Senior Data Engineer (AI-Powered Automation)

Senior Data Engineer (AI-Powered Automation)

About project

We are looking for an experienced Senior Data Engineer who will design and maintain efficient, stable data pipelines powering AI-driven customer case automation. You would join our client from the logistics sector.

You will be responsible for acquiring data from the key operational systems, SharePoint pages, and Excel-based business documentation. Your pipelines will deliver structured, validated, and near real-time data to AI prompt-generation modules integrated with LLM and RAG architectures.

This role is not purely ML-focused; however, it plays a mission-critical part in the LLM value chain. The quality, freshness, and reliability of your pipelines will directly impact the accuracy and business value of AI-generated customer responses.

Start: March

End: December 2026 with possibility of extension

100% remote

Job responsibilities

• Design, build, and maintain robust ETL/ELT data pipelines (batch & streaming)

• Integrate data from main internal system, SharePoint, Excel, and other operational systems

• Deliver near real-time, high-quality data for LLM prompt generation

• Implement and optimize workflows in Databricks (Spark, Delta Lake)

• Develop API endpoints and support event-driven architectures (Kafka)

• Collaborate closely with Data Scientists and ML Engineers

• Diagnose and resolve data issues end-to-end

• Ensure data quality, validation, and performance optimization

Expectations

• Strong Python programming skills (clean, testable code, packaging, CI/CD)

• Solid experience in Data Engineering (ETL/ELT, distributed processing)

• Hands-on experience with Databricks (Spark, Delta Lake, workflows/jobs)

• Strong SQL skills (data modeling, complex transformations, optimization)

• Practical experience with AWS services (S3, Glue, Lambda, Step Functions, RDS, ECR)

• Experience with Kafka and event-driven architectures

• Experience building API endpoints (including Databricks/VPC endpoints)

• Strong analytical and debugging skills (data issues end-to-end)

• Very good English communication skills

• Polish language required

  
  

  Additionally, minimum required knowledge in the AI/LLM area:

• Understanding how LLMs use context and prompts (prompt engineering basics)

• Preparing data for prompt generation (extraction, cleaning, validation, standardization)

• Creating structured context windows

• Understanding data flows in RAG (Retrieval-Augmented Generation) systems

• Basic knowledge of embeddings and vector databases

We offer

• Ongoing support from a dedicated agent who is caring about your project continuity, contact with clients, necessary formalities, work comfort, and development

• Career Development Program – advice on career planning based on the latest trends and market needs in IT, including consultations with career agents and mentors

• Access to 7N Learning & Development – a development and educational platform offering webinars, a library of articles and industry reports, and frequent invitations to both one-time and recurring events focused on technical, business, and lifestyle topics

• Spectacular events, both for you (e.g., annual Kick-Off trip, Christmas parties, or Summer Olympics sports events) and for your loved ones (e.g. family picnics),

• Professional growth beyond your current project. You can actively engage in knowledge sharing through the 7N Services offering, designed for 7N clients.

• Connections and access to the expertise of top IT professionals in the market – the average professional tenure of our consultants in Poland is over 10 years,

• A complete benefits package, including funding for medical care, life insurance, sports cards for you and your loved ones, as well as discounts in stores in Poland and abroad.

  
  

About 7N

Constantly searching for projects, difficult rate negotiations, lack of development support – sounds familiar? At 7N, you gain not only stability of contracts but also the personal involvement of a dedicated agent who ensures your professional comfort and continuous access to development initiatives. 

Our mission is to provide stable and rewarding collaborations that drive your success as an IT expert and the success of our clients. We build long-lasting relationships based on Scandinavian values and 30 years of experience creating IT solutions for over 200 organizations.