Data Engineer
Data
Data Engineer
eSky.pl
Białystok
Type of work
Undetermined
Experience
Mid
Employment Type
B2B
Operating mode
Remote
Tech stack
Python
Grafana
MongoDB
SQL Server
PostgreSQL
Job description
Are you interested in Data Engineering? We are currently looking for a person to join our
newly formed team to help us develop this field. As a team member, you will have the
opportunity to co-create the architecture of processes that feed eSky with data.
We use elements of Agile methodology to organize our work. We use Grafans to monitor the
availability of our solutions. On a daily basis, we use Python, Apache Airflow and Google
Cloud Platform cloud services, integrate data from various sources (REST, NoSQL
databases, relational databases, (S)FTP and more) and place it in our BigQuery data
warehouse.
newly formed team to help us develop this field. As a team member, you will have the
opportunity to co-create the architecture of processes that feed eSky with data.
We use elements of Agile methodology to organize our work. We use Grafans to monitor the
availability of our solutions. On a daily basis, we use Python, Apache Airflow and Google
Cloud Platform cloud services, integrate data from various sources (REST, NoSQL
databases, relational databases, (S)FTP and more) and place it in our BigQuery data
warehouse.
Challenges that await you:
- You will be responsible for the development and maintenance of processes related to data feeding and collection in the BigQuery data warehouse;
- You will also be expected to ensure data quality and consistency;
- Using Python and SQL, you will also program and configure ETL mechanisms to feed the data warehouse;
- You will also be tasked with working with data source owners and data consumers to develop proper practices for feeding the data warehouse, maintaining data consistency and data quality and usage methods;
Expectations:
- Ability to analyze and understand the business domain of problems and translate the business domain into a data model;
- Very good knowledge of Python and SQL Knowledge of containerization issues (Docker)
- Knowledge of ETL/ELT process tools (e.g. Apache Airflow)
Nice to have:
- Knowledge of issues related to cloud services (ideally Google Cloud Platform)
- Knowledge of issues and tools related to data mining, data science, machine learning will be an asset
Technology stack:
- Python
- Google Cloud Platform (BigQuery, Kubernetes Engine, Pub/Sub, Compute Engine,
- DataPrep, Dataflow, Cloud Functions, Cloud Run, Cloud Storage)
- Apache Airflow
- Grafana
- MongoDB
- Postgres
- SQL Server
What we offer:
- We offer a sa lary depending on your knowledge and experience - from 12,000 to 19,000 PLN net per month on B2B. We are open to employment contract.
- Flexible and stable form of employment - B2B or employment contract.
- Work in the IT Department in the Data Engineering cell;
- Close cooperation with developers, business owners and analytical team;
- Friendly and open environment to improve work to get better results
- Benefits in a cafeteria system - including a Multisport card cinema tickets, discount codes in stores, the opportunity to
- Group life insurance and private medical care
- Great working conditions - modern office, well-stocked kitchen, chillout room, best commercial tools, flexible working hours
- Opportunity to work fully remotely