Remote Senior Big Data Engineer
Data
Remote Senior Big Data Engineer
Warszawa
Type of work
Undetermined
Experience
Senior
Employment Type
Permanent, B2B
Operating mode
Remote
Tech stack
Hadoop
Kafka
Python
Presto
SQL
Linux
Scala
GCP
Job description
Online interview
Friendly offer
We are looking for Senior Data Engineers to work remotely on a data analytics projects that form a RTB exchange. The Exchange operates over 100k requests/s and all teams constantly face with high volume, low latency and geo-distributed challenges. Our data volume is over 20TB/day, over 200 uncompressed.
This is a 100% remote position, for qualified candidates we help with visa and relocation to Wrocław, Poland.
For B2B contracts we are offering 150-180 PLN/h
The data is used to generate reports, update advertising budges and update our optimization engines, we focus on running as close to real-time as possible with very tight SLAs.
The data is used to generate reports, update advertising budges and update our optimization engines, we focus on running as close to real-time as possible with very tight SLAs.
If you like solving hard and technically challenging problems, join us to use those skills here to create real-time, concurrent, globally distributed systems applications and services.
Note about working hours: It's OK to work in CET/CEST timezone, but there are meetings or other events that will require flexibility on your side
Note about working hours: It's OK to work in CET/CEST timezone, but there are meetings or other events that will require flexibility on your side
If you think you are a good fit even though you don’t meet all requirements – please apply, we are currently filling multiple roles and will do our best to find the best match.
Role details:
- Work on creating and maintaining reliable and scalable distributed data processing systems
- Optimize our jobs that utilize Kafka, Hadoop, Spark, Presto and Kubernetes
- Scale, troubleshoot and fix existing applications and services
- Own a complex set of services and applications
- Focus ensuring that applications run 24/7
- Lead technical discussions leading to improvements in tools, processes or projects
- 70% time – new development; 30% – bug fixing, maintenance
- Work on hybrid infrastructure: GCP + on-prem
- Regular tasks include:
- creating new jobs to ingest different internal and third party data
- develop frameworks for new jobs
- create different consumers for data sent via Kafka and Spark Streaming
Key requirements:
- very good knowledge of Python
- minimum 7 years of relevant professional experience
- Database experience, for example: PostgreSQL, MongoDB, Cassandra, Redis
- Proficiency in Linux
- Willingness to participate in 24/7 on-call rotation (support team members in EST time zone when they sleep)
Bonus experience:
- Real Time Bidding (RTB), AdTech, High Frequency Trading (HFT) experience
- Devops experience (Chef, Jenkins, CircleCI)
- Scala and Java experience
- Kubernetes experience
Client Description
Our Client is a company hiring over 150 employees, headquartered in NYC, the company operates a large RTB exchange and multiple other supporting services.
Client takes great care of engineer happiness and puts great value on technical excellence, they understand that it takes time to do things right and value that over half-backed solutions.
Our Client is a company hiring over 150 employees, headquartered in NYC, the company operates a large RTB exchange and multiple other supporting services.
Client takes great care of engineer happiness and puts great value on technical excellence, they understand that it takes time to do things right and value that over half-backed solutions.
Check similar offers
