Data Engineer

We are currently looking for Senior Data Engineer to support an LLM/NPL Project.

Data Engineer must-have experiences with:

· Data management and integration, including Data Mesh, Data Lakes, and integration with external services

· Core cloud concepts, with a special focus on databases (e.g., AWS RDS /Kinesis /Glue /EC2 /EKS /ECS)

· Optimization of NoSQL and SQL databases in a cloud environment

· Software engineering, especially in object-oriented programming (OOP)

· SQL and database query optimization techniques

· Implementing ETL and data ingestion pipelines for both initial and update loads, including batch processing of data (structured and unstructured sources)

· Performing database benchmarking for latency and performance optimization

· Good programming practices, particularly in implementing data cleansers and parsers

· Understands the differences between various database solutions and can recommend appropriate ones, including document databases and vector databases

· Big Data technologies (Hadoop, Spark, or Apache Kafka)

· Message queues and asynchronous processing

Location: 100% remote