Scala/Spark Developer

Project

Our project is building ecosystem around Nordea’s Enterprise Data Lake. Our developers build solution mainly in Scala language, we help them with providing automation and pipelines for testing and deployment.

Works is in agile setup with international teams. Lenght: at least 1 year.

"Key responsibilities

• Develop Scala/Spark
• Programs, scripts, and macros for data extraction, transformation and analysis
• Design and implement solutions to meet business requirements
• Support and maintain existing Hadoop applications and related technologies
• Develop and maintain technical documentation, including data models, process flows and system diagrams
• Description of knowledge and experience
• Experience from Scala/Spark related projects and/or engagements
• Create Scala/Spark jobs for data transformation and aggregation as per the complex business requirements.
• Should be able work in a challenging and agile environment with quick turnaround times and strict deadlines.
• Perform Unit tests of the Scala code Raise PR, trigger build and release JAR versions for deployment via Jenkins pipeline
• Should be familiar with CI/CD concepts and the processes Peer review the code Perform RCA of the bugs raised
• Should have excellent understanding of Hadoop ecosystem
• Should be well versed with below technologieso SQL/HQL(Hive Queries)o Oozieo Jenkinso ETLo Shell scriptingo GITo Splunk