Senior Data Engineer
Other
Senior Data Engineer
Kainos
Gdańsk
Type of work
Undetermined
Experience
Senior
Employment Type
Permanent
Operating mode
Office
Tech stack
Scala / Java
Apache Spark
ETL
Data modeling
Apache Hadoop
Job description
Senior Data Engineer works within a multi-skilled agile team to design and develop large-scale data processing software to meet user needs in demanding production environments.
Most of our work comes through repeat business and direct referrals, which comes down to the quality of our people. The success of our Data Engineering teams means that customers are bringing us an increasing number of exciting data projects using cutting-edge technology to solve real-world problems. We are seeking more high calibre people to join our Data & Analytics capability where you will grow and contribute to industry-leading technical expertise.
Role Description:
- Working to develop data processing software primarily for deployment in Big Data technologies. The role encompasses the full software lifecycle including design, code, test and defect resolution;
- Collaborating with colleagues to resolve design challenges;
- Working with architects to ensure the software supports the high-level architecture and in particular non-functional needs;
- Writing easily testable code including automated unit testing;
- Working with operations teams to ensure operational readiness;
- Advising customers and project managers on the estimated effort and technical implications of user stories and user journeys;
- Continuous improvement and sharing input on data best practice;
- Coaching and mentoring more junior technical staff.
Required Experience:
- Software development experience with distributed data processing technologies including Cloudera’s distribution of Apache Spark;
- Software development experience in Scala;
- Understanding of ETL/ELT data processing pipelines;
- Experience developing multiple large-scale data processing solutions;
- Understanding of contemporary data storage technology, such as document, graph, log stores and other non-relational platforms;
- Data modelling experience with RDBMS and NoSQL data stores.
- Experience coaching and mentoring junior and experienced technical staff;
- Ability to make effective decisions within fast-moving Agile delivery;
- Experienced in continuous integration tools and techniques (e.g. Jenkins);
- Ability to advise architects and other stakeholder on detailed technology and development practice;
- Ability to work collaboratively with others using version control (e.g. Git);
- Clear written and verbal presentation with an open attitude to sharing information;
- Able to prioritise and work to deadlines.
Desirable Skills:
- Software development experience in open source technologies including: JVM languages;
- Software development experience with Cloudera’s distribution of Apache Hadoop;
- Experience of data visualisation and complex data transformations, including ETL tools such as Talend;
- Experience with steaming and event-processing architectures including technologies such as Kafka and change-data-capture products;
- Understanding of unstructured text processing including NLP;
- Understanding of data redaction implementations;
- Participation in development and/or technology communities;
- Open source contributor.
What you get from Kainos:
- Relocation package that covers flights, accommodation, and other relocation costs;
- Private medical insurance at Medicover (possibility to take your family under it);
- 1500 PLN per year to pay for your hobby courses (you can spend it for whatever you want i.e. kitesurfing or Spanish language lessons);
- Gym card Multisport (50% paid by company; possibility to take your family under it);
- Integration and family parties: Santa Claus, Summer BBQ, Kick Off, Pay Day Drinks, Christmas Party, St. Patrick’s Day, and many more;
- Online training portal access — Pluralsight;
- Career Coach taking care of your career progress in Kainos;
- Conferences in Poland and abroad as a speaker or listener (you can prepare for your presentation within working hours);
- 3000 Pounds for Refer-a-friend scheme;
- Possibility to work from home up to 3 days per week;
- Core hours (starting work between 8 and 10 AM);
- 7,5 hours working day;
- Fun rooms, Chill Out room and gym with showers in the office;
- Soft skills training;
- Easy access to the people on managerial levels, real influence on the company’s decisions;
- Very stable employment in a company that cares about employees’ development.