Big Data Architect
Java
Big Data Architect
7N
Warszawa
Type of work
Undetermined
Experience
Senior
Employment Type
B2B
Operating mode
Remote
7N
W 7N zapewniamy najlepszą opiekę ekspertom na rynku IT, chcącym pracować w modelu kontraktorskim, w oparciu o jawne i transparentne zasady finansowe. Pracując z nami zyskujesz wsparcie profesjonalnych Agentów, którzy dbają o ciągłość Twoich kontraktów, komfortową pracę, rozwój zawodowy i niezapomniane integracje. Aktualnie pracujemy z ponad 1700 Konsultantami, którzy realizują projekty dla ponad 160 Klientów z różnych branż. Posiadamy 11 biur w siedmiu krajach: Danii, Polsce, Szwecji, Norwegii, Finlandii, USA oraz Indiach.
Tech stack
Big Data
Java
Spark
Job description
7N is an agent for high-end IT professionals. Over 30 years of operation we have proven that a clear and transparent financial model, collaboration exclusively with experts in their respective fields, and taking good care of them comprise the best possible IT consulting model. We act as an individual agent for our consultants and promote their competences to our clients by offering them a wide range of projects in which they may participate. We add wage transparency, career development support and professional stability. Our main goal is long-term collaboration; therefore the majority of our staff have been with us for many years.
About project
About project
Currently for Our Client - International Pharmaceutical Company we are looking for a talented Big Data Architect who would like to help us transform the way how new therapeutic hypotheses are formed and validated. We work in 3 distributed development work streams delivering value for the scientists from the laboratories located in California, USA. One of the work streams is focused on delivering data engineering platform which is the hub of scientific data from in-house and external organisations. On a very high level our platform should allow to pull the data from the selected sources, expose it to semi automated curation, store harmonised data, validate the data, enhance the data, process the harmonised data to calculate evidences and associations, allow query capabilities of the data.
Responsibilities
- Review the current architecture and confront this with challenges, limitations and new requirements for post pilot release
- Design and implement the solution and justify the choices to the product owner (located in CA, USA)
- Share the knowledge by code/design review, etc.
- Project in therapeutic research area: genetics, biotechnology. Complex domain.
Requirements
- At least 2 years of big data development professional experience
- Experienced with building big data from scratch
- Experienced with architecture and design of big data solutions, being able to justify decisions and choices
- Understand HDFS
- Understand High Performance Computing
- Experienced with MapReduce,YARN and tools like SPARK
- Experienced with NoSQL db like Hive and query engine Presto
- Experienced with big data volumes - single dataset >100GB; aggregates, search on tens of TB - a few PB
- Experienced with different data formats for big data like PARQUET and data compressions
- Experienced with monitoring for data processing
- Experienced with network protocols like nfs, samba, san
- Willing to share knowledge with other team members
- Fluent English
- Comfortable in a distributed work environment
- Nice to have: experienced with JVM tuning
- Nice to have: experienced with RDBMS databases and ETL processing
We offer
- Transparent wage model; disclosed margin for 7N. The aforementioned wages are target wages paid to the consultant for the subcontracted work.
- Stable and long-term collaboration with various client projects.
- Professional freedom; We are one of the few IT companies who do not use non-compete clauses or retention agreements.
- Career development support, training and technical certification subsidies, conference participation, etc.
- Private healthcare and the Benefit Multisport card.
- Collaboration with experts.
- Large client and project portfolio of over 40 companies, prioritizing project continuity and ongoing personal agent support.
- Full integration into the client company structure (e.g. participation in all company events, 7N Kick Off 2018: https://youtu.be/0YoTCeVsB3E and trainings)